Everything everyone else has said, especially about getting a share or loan first, you not only get horse experience without losing anything, you also get an 'in' to the local horsey community, through the owner or the yard its on. And this is a valuable resource in finding your perfect partner.
A couple of tips, if you go through a private sale always google the phone number, its amazing how many times this will identify a hidden dealer, not that all dealers are bad, but if they are good why hide? Do your research, get a list of attributes, ones that are non negotiable and ones that are flexible and only look at horses that meet most of your requirements, this strategy should in theory stop you seeing anything too unsuitable and falling in love with it. Be open minded about older horses, they are experienced and should know their jobs, and natives and cobs especially tend to age well staying fit for what a novice horse owner would want to do, and realistically you don't want two novices together. All though of course there are exceptions, and people and horses can 'learn together' its a very steep learning curve at the best of times.
I bought my boy when he was 15, he is 18 now and still as fit as a fiddle (touch wood) and is well able to go on long after I have had more than enough. He is a Fell pony, short strong and very sturdy, can hack, jump, dressage all better than me. He was the first one I looked at, and fell in love with him. But because he met most of my short list, this did not turn out to be a disaster, and both he and his lovely owner were vouched for by a member on here. (forever grateful) So if you see a likely advert, bung a link on here you never know you might get some interesting insider info. And also we like being nosy.
Good luck!!