Add basic robots.txt file

nhsuk / betahealth

Rendering application for beta symptom and condition pages

https://beta.nhs.uk

MIT License

2 stars 8 forks source link

Closed teneightfive closed 7 years ago

teneightfive commented 7 years ago

This change adds a basic robots.txt file. It will prevent 404 errors showing in logs when this file is hit.

It also allows a base file to declare any future definitions for beta.nhs.uk.

tyom commented 7 years ago

Do we want to disallow /elements since it's not really part of the content on that domain?