Update documentation

[xdvom03]

Particular flaws to fix:

• More mathematical background needed. Explanation of bag-of-words model, citations of provided literature, discussion of what we are classing - a "3" found in the wild at random, or as part of a document?
• Crawlers - current successes/failures, interface, models, purpose
• Overall interface, GUI, state management
• Discuss specific folders (not redundantly wrt the folder comments, so at a grander scale) - motivation of classification, attempts to avoid bias, inclusion characteristics
• Input some chronology, explain the process of program evolution

This also serves to slow down the flood of half-made features and solidify some of the basics.

[xdvom03]

A necessary condition is getting a fixed algorithm wrt normalization. http://www.ra.ethz.ch/cdstore/www6/posters/725/web_search.html suggests normalizing documents to the same size, somethign I don't do. However, it also does little against word repetition.

[xdvom03]

Whoops, many of the points raised here are still unresolved. What was resolved was the final point: some stability & fixes. But the main point still stands.

[xdvom03]

Got accidentally re-closed.