New Orleans Saints coach Sean Payton and defensive coordinator Rob Ryan dislike each other so much that the team may part with at least one of them after the season, according to CBS Sports.

Payton and Ryan have had several spirited discussions on the sideline during games, including last Monday’s loss to the Baltimore Ravens. Ryan is popular with the team but may be let go because of the declining play of the defense this season.

The Saints’ defense improved significantly during Ryan’s first year as coordinator in 2013 but that did not carry over to this year.

New Orleans has had several defensive coordinators in recent years under Payton, including Gregg Williams, Steve Spagnuolo and Ryan. The Saints also have an NFL-high $160 million committed to next year’s salary cap, and several of their higher priced defensive veterans will probably not return.

Payton’s future status is not clear. The Saints are tied for the NFC South lead but are 4-7.