add a robots.txt and sitemap.xml

[Orbiter]

Because our site navigation cannot be crawled so easily with a non-javascript enabled web crawler since https://github.com/loklak/loklak_server/issues/394, we need a sitemap which links all available pages.

• implement a sitemap servlet in org.loklak.api.cms.Sitemap.java which returns xml according to https://en.wikipedia.org/wiki/Sitemaps
• the sitemap shall list all pages using the method org.loklak.api.cms.TopMenu.service(); - like (new TopMenu()).service(). Then you get all paths as TopMenu presents that as well.
• add a robots.txt file in https://github.com/loklak/loklak_server/tree/master/html as static file
• the robots.txt must allow all search engines to load all pages, see https://en.wikipedia.org/wiki/Robots_exclusion_standard
• the robots.txt must include a link to the sitemap according to https://en.wikipedia.org/wiki/Sitemaps#Search_engine_submission
• for security, add a crawl-delay of 10 seconds to robots.txt

[shivenmian]

Sorry for postponing this. Had spend quite some time on the accounts system lately, now it's more or less done so I'll try my best to do this by Milestone 5.