I think it has more to do with the fact that at that speed which you're not really used to, everything becomes a potential threat to your continued existence. Hence the only thing that will keep you alive is the concentration of your mind on the road and its hazards, and the willingness of your body to cooperate to instructions. I don't think I'm a thrill seeker in that respect, nor do I get any sort of pleasure from walking close to the edge of a cliff. It's the mental challenge that appeals to me, with the added pressure of the very real consequences should I fail.
This could also be a very good allegory for my life, but let's not go there.