Why should you be confident about your marriage? You are being naive.
Agree with him, this isn't working, he needs to move out, you need to get divorced because he's not a good husband or father. Let him go, you are pursuing someone who doesn't value you. How does that make you look in his eyes?
When you stand up for yourself is when his feelings start to change.
"What is best for my kids is best for me" Link to quotes: https://www.divorcebusting.com/forums/ubbthreads.php?ubb=showflat&Number=2879712