Where can I get cheap apartments just outside of NYC that are still nice and are close to mass transit to NYC?

I do not want to pay the outrageous prices in NYC, yet I still would like a nice apartment that can easily access NYC (via mass transit-- not driving). Where should I be looking for apartments that match this criteria? What mass transit opportunities do they have? Are there any websites that list them? Is there anything in NJ that is comparable in quality to apartment in Manhattan but while being cheap(er) because they are outside of NYC?