Have you tried soft lowers on the highway crash bars? As long as the roads are dry I ride down and around freezing thorough the winter. Not bad at all with the soft lowers - sucks badly without them. . . . .
I'll second this. I picked up a set of the regular Chillowee chaps for the Star Big Bar, and it made a world of difference. Between those and regular chaps, it should end the heat drain in your legs. The bike still gets plenty of air flow, when the temps get back into the 60's.
You're sitting on a big hot plate, take advantage of it.