If your divorce agreement provides or divorce order provides that your spouse is either responsible or shares in liability for prior tax returns, then you’re not necessarily going to be responsible for all of the interest and penalties and late fees of a prior return. You will be in part responsible depending on how much the agreement puts on either party. But the important thing to realize is that the IRS is still going to hold you responsible for all of it. They are not bound by your divorce agreement, so you may still need to seek court intervention to enforce that on the other spouse or your divorced spouse.