Allow customization of robots.txt

ckan / ideas

[DEPRECATED] Use the main CKAN repo Discussions instead:

https://github.com/ckan/ckan/discussions

40 stars 2 forks source link

Allow customization of robots.txt #178

Closed torfsen closed 8 years ago

torfsen commented 8 years ago

Currently, CKAN's robots.txt is a static file and hence can only be changed in CKAN itself. However, sometimes one wants to change robots.txt for a specific extension (for example to specify the location of the sitemap generated by ckanext-sitemap) or for a specific installation.

I therefore suggest to dynamically create robots.txt from a Jinja2 template.

torfsen commented 8 years ago

This has been implemented and merged: ckan/ckan#3164.

ThrawnCA commented 2 years ago

This was reverted in the move to CKAN 2.9 (https://github.com/ckan/ckan/issues/4801), without a clear explanation of why.