In this pic you can get a view of how they are built right into the mountain:

This is a close up of one of the entrances.  To think that this was built in a BCE time is mind boggling.

This shows one of the vast hallways with incredible columns and statues along the sides.  It is reminiscent of the movie Indiana Jones and The Temple of Doom.

Here are some videos giving some more information on the caves and some additional cool scenery.  One of the commenters had this to say in terms of an archaeological explanation in reply to someone on the thread who thinks it was done by aliens:

The kind of explanation that these theorists are giving is exactly the reason why the west was a backward place for most of the history of mankind. Firstly, it is very difficult for them to accept that when there ancestors were running in the jungles, there was civilization in other places of the world.  Secondly, they have no imagination and they are unable to comprehend things that are beyond the western material scope of understanding.  The half-man half-animal creatures are all explained in different Indian philosophies. Also, these caves are not the oldest structures in the Indian subcontinent or even the world. So suggesting aliens is an extremely ignorant argument.

Another commenter had this to say about visiting:

i have visited these caves. they are now being maintained by some organisation from Japan. Simply extraordinary.

In this video they show some cool pics and get into the religious side of things a bit.

However one commenter disagrees:

This video says, hinduism gained in India and buddhism declined after 6th century. I believe this statement is wrong. Hinduism existed in India for a long time even before buddhism or Buddha came into existance. They should rephrase this as Buddhism gained and declined in India between 2 BC and 6 AD or whatever it is.

What do you think about who built these?  Feel free to comment with your ideas.

thanks to Shriram Rajagopalan for the pic