I've collected most of my children's books from local used stores. (Goodwill, St. Vincent De Paul, and Montana Rescue Mission) The prices at used stores in my area range from $0.20 to $2.99. I also look in used book stores, but prices are usually higher.
Amazon also has many used books if you are looking for a certain book but don't have the time to search through a store's pile of books.
Garage sales and church bazaars are two more places to find children's books.
Borrow books from your library.
Children love to have stories read to them again and again. Children also become fluent readers by reading books themselves again and again.