Another recommendation from Cheryl:

Digital History at the University of Houston has a repository of historical music which is in the public domain. They also link to historical music for educational use that is freely available on the web under fair use guidelines.

Pretty cool site with a great variety of music.