Sandstone caves, also known as wind caves are formed over long periods of time. Sandstone is formed when layers of sand are compressed and cemented together at the bottom of oceans and other bodies of water. Over time, oceans dry up and become deserts. The earth moves and shifts and this sandstone is eventually exposed to the wind and rain, which patiently shapes it into caves.
Because they are usually small and shallow there is not much to explore inside, but just getting to them can be an adventure in itself. They are often located in deserts or high on cliffs.
In the County of San Diego, California are the Wind Caves of the Anza-Borrego Desert State Park. Other examples in California are located in Castle Rock State Park and there is a cave with Native-American art in it at Painted Cave State Historic Park.