I've had a record player for a while now and in that time have built up a small but solid collection of vinyl records. I initially went to record shops I knew, like Rough Trade in Brick Lane but they were really expensive with albums typically costing from £25 upwards. I found that a far more manageable and fun way is to crate surf at market stalls and charity shops. Some charity shops know that there is a demand for vinyl so do specialist music only shops. I've collected a lot from Oxfam for a great price including some of the Jean Michel Jarre albums from the 70's and 80's Mike Oldfield in his prime. All for a princely £2.99 each, bargain!
I've also recently splurged a little and bought the Three Colours Soundtracks for Kieslowski's masterpeices for £15 each on Amazon. These came with CD recordings included (which I already had but it's a nice touch). My prize is an old pressing of the Fleetwood Mac Rumours album. This only coat me £4.99 and is in pristine condition. All in all I have about 20 records which is not bad by my reckoning. Long live crate surfing and charity shops