I say absolutely go for it. Universal is so much fun, and the boys will LOVE Islands of Adventure with all of the great rides!

What fun!
I know it might seem like it would be tough to be on Disney property without going to the parks, but I've done it and it's perfectly fine. It's not like you're camping and doing nothing at all. You're going to Universal, and the beaches, etc. I always use Orlando as a base of operations and then from there I hit the beaches (the west coast beaches are my favorite - I LOVE St. Pete) and there's tons of stuff to do in Orlando besides Disney. I'm not sure what your money situation will be, but you could do a dinner at Medieval Times. That's one of my favorite dinner shows. Or there's the Dixie Stampede (YUMMMMM)... a pirate dinner show (might be more up the boys' alley)... there's mini golf, there's a place called the Holy Land Experience (if that is something that interests you). Outlets, good food, shops... Orlando is full of things to do!
Anyway, much as I love Disney (and I DO love Disney), I don't think you're crazy at all and I think the boys will be just fine not doing the parks much, if at all, because it sounds like you'll be keeping them quite busy with other adventures.
Now you're making me REALLY want to go back!