Well, my DH is in the "I like it but in moderation" category. He told me the other night that he's not going back to WDW for at least another 5 years! (I am currently planning a DLR trip....he didn't say he wouldn't go to DLR
! But if he really doesn't want to go to WDW for 5 years, then DS and I will go without him.) But I had a similiar situation with my father. I really wanted my mother to get to experience WDW. I knew that Dad would hate it (hot, expensive, crowded, a lot of walking), but he wouldn't be left home alone. Who would take care of him? He's a big baby! The plan was for all of us to go, when he told my Mom that he was going but would be miserable, "And if I'm miserable, I'm going to make sure that everyone else is miserable too,"
Well, that was it. There was no way that I was letting him ruin my vacation, so he was uninvited. Unfortunately, that meant my Mom stayed home too, but that was because she didn't want to confront him.
It is a difficult situation. If I were you, I might have a talk with DH and see if I could get to the bottom of what his issue is. But you're right, you might just have to leave him behind and try not to let his negativity ruin your fun.