Closed ItzNotABug closed 6 months ago
This PR adds the basic deny all robots.txt endpoint and a X-Robots-Tag header to all endpoints.
deny all
X-Robots-Tag
Note: It is up to the search engines to honor his request.
This PR adds the basic
deny all
robots.txt endpoint and aX-Robots-Tag
header to all endpoints.Note: It is up to the search engines to honor his request.