In India, they built a lot of stuff directly into mountains and caves

Here are some interesting examples: https://www.thrillophilia.com/caves-in-maharashtra