Yes, where you go to school does matter in a few different ways...
Good schools typically have better professors. I went to a lesser university for 2 years, and it was actually more difficult than the good school I transferred to. I don't know about Ivy League schools, but as a general rule, I believe better schools are easier once you're there bc the level of instruction is better.
As far as getting a job, going to Harvard vs a state university won't make a lot of difference for most people. If you're in a highly competitive field, yeah, it probably does. If you're going to try to live in a major city, yeah it probably does. But as far as a bachelor's degree in business or accounting or another very general field, it probably won't make a big difference in most cities.