These are traditionnal Japanese sweets
https://en.wikipedia.org/wiki/Wagashi
Some of them you can find in supermarkets for cheap but the higher class ones are indeed pretty expensive. Nearly everything in Japan pretty much exists in three categories, cheap, normal and really expensive ^^
That being said, the tea + sweets set in teahouses on the other hand can be rather cheap though.
Aha natural hot springs are the best! I recommend going to one that's outside so you have a nice balance between hot and cold and can stay longer without getting dizzy too fast.