search

ansible / docsite

Static HTML and assets for docs.ansible.com
18 stars 15 forks source link

update robots.txt to disallow old urls #227

Closed oraNod closed 8 months ago

oraNod commented 8 months ago

This PR updates the robots.txt file to disallow crawlers on URLs for old projects and for pages that no longer exist.