How's the library been coming along?

I've been thinking of possibly starting up a similar endeavor in my local area, after I move out of the city and into a more sane outlying area. How large (and topic-diverse) of a book collection do you think one might need in order to make such a project viable?