If you start the horse right, it can be done at age 2. Just don't ride for forever and be willing to accept that when they're going through a growth spurt that you probably aren't going to get a whole lot done. Also, doing higher level stuff like reining or jumping is going to be harmful. Flat rail work isn't going to be the death knell of your horse.
Heck, the lamest horse in my barn right now is an elderly broodmare who's never been backed.
I'm not sure where people are getting the idea that in the USA horses over the age of 12 are over the hill. They're not, I see lots of older teens and 20 year old horses packing youths and nov. amateurs around even at upper breed level. This is even more true at lower level local and open shows. That's not even getting into recreational riders.