Thats happened to me, but mostly from chestnuts! Imagine that?
My best horse was a 15 yo Gray Polish Arab gelding, had to hop on quick because he'd walk out from underneath you, but he sure did a good job of taking care of me.
And I learned to ride on another gray arab, polish bred too, my aunt's horse.
I fell off a chestnut morgan, got bucked off a chestnut arab...and..got taken on a gallop/jump on a chestnut haflinger. (Not the one I currently own)