I am not sure if I mentioned here how my now 4 year old son had a major aversion to shopping at a very well known and much loved craft supply store, let's call it Michelle's... well because I live in a small town and there aren't many other options I have become very savy at where I can get supplies. Here are my favorite.
Hardware Store - it's my favorite place for paint chips ( great fun for collages), paint brushes , glue and painter's tape.
Grocery Store - paper bags for paper, dish sponge for a paint brush, apples for stamps. The list goes on and on.
Office Supply Store - Paper, stickers, labels, hole punches... the list goes on and on.
GoodWill - This is mostly for books but from time to time I find wonderful puzzles, and pretend play costumes too!
Dollar Stores - Not many of us have a huge budget for crafts and lucky for us you don't have to with such variety at the dollar stores. Most have dedicated craft aisles and on good days you can find name brand art supplies.
See don't fret if there is no store near by devoted to crafts, you can find what you need to have fun at stores you do have ( or rather where your son won't meltdown).