Add robots.txt to prevent bots indexing technical content

[flohoff]

We have short url for Articles as /wiki/$article. All the technical stuff like "version history" etc is reachable as /mediawiki/index.php?...

Prevent search engines from indexing technical pages and historic pages

Documentation see here:

https://www.mediawiki.org/wiki/Manual:Short_URL/Prevent_bots_from_crawling_index.php