Exes, a threat?

I have been wondering this last three years (when ever I have had time to be single ) and I still haven't found the answer:

What makes it so dangerous that I don't hate any of my exes (I can't say I always understand them, but I surely don't hate them)? Why majority of men seem to find it as a threat? (Remember, majority.)

I know, one of the common excuses is They are afraid that you return back to $EX...

I said excuse and I meant it.
It is nothing but one. It usually is all about their low self-esteem, not believing that they could have and hold me because they aren't something called good-enough. Which makes it interesting... they make me run away because I don't want a man with low self-esteem and because they express distrust towards me and my moral fibre.
So, in certain sense they make their fears true. They prove being not good enough or at least not someone I want to be with.

And, the question I have sometimes asked: if your exes aren't an issue to me why mine should be to you?
I am closing to my thirties, I can't except that my counterpart has no experience (honestly speaking... I prefer them experienced (in many areas *cough*) - divorced is always good *grin* (because, based on experience, having some depth and understanding is more common in divorcees...)), so, why the heck some people seem to expect that someone who is almost 28 would have none..? Or should have none...
I find that way of thinking odd.

If one finds my personality fascinating one should also remember that my exes have been there moulding me as I am as they are my past and my past is part of my life. Life, which has made me who I am, which has made me that person I am. That person who one finds interesting - though it seems that most find me intimidating rather than interesting.