Your children should be with their mum, it's unfair on the 3 year old. You will have to tell your wife to take care of your children. Of course the 3 yr old is going to miss his mum, a 3 yr old should be with his mum!
Instead of asking her to come back home, tell her she needs to take care of her children. I know you've probably told me this before, but what's her reason for leaving her kids with you?
Next time she tells you how miserable she is feeling, tell her that you are feeling miserable as well but you can't have her yoyoing between the 2 houses because that will really hurt you and the kids. Try not to make her feel guilty about the kids, you don't want her to come home for the wrong reasons!
The only other thing you can do is be a mum and dad to your kids. Make life fun for everyone, especially the 3 yr old. Do you live in the States? Can you get help to look after the kids? Do you work?
I know it's probably alien to you, but you could take the little one to activities at the library and mum and toddler groups. You may find another male there and if not I can guarantee you will get a lot of attention from the ladies. Not in that way, just that they will want to look after you.
Do you go to any single parent groups? They will give you lots of helpful advice as they've been in similar sitches.
Do a search on the internet for free or cheap activities in your area.
Hope this helps, you've got a lot of homework to do now smile

H47 me48
T22 M21
S20 - Got high functioning autism
3/2000 H admits to OW and moves out to live with her
11/2000 H moves back home
2/4/13 H moves out
H tells me he wants D 6/13, but now he can't afford to!