Custom robots.txt and sidemap.xml

[Kariton]

i have added a custom robots.txt and sitemap.xml because the include ones have no capabilities.

Disable entire path globaly:

params:
  robotsdisallow: 
    - /private/
    - /secret/

or set the robotsdisallow parameter to true per page:

---
title: "Private Page"
robotsdisallow: true
---

[Kariton]

params:
  robotsdisallow: 
    - /authors/

but sub-sites are still in sitemap.xml... needs fix

[Kariton]

after reviewing a few other repos and templates i realized that this kind of customization is better placed within the users files instead of the theme itself.