mailarchive.ietf.org/robots.txt is blocking search engines

hassanakbar4 / mailarchive-tickets

0 stars 0 forks source link

mailarchive.ietf.org/robots.txt is blocking search engines #274

Closed hassanakbar4 closed 2 years ago

hassanakbar4 commented 5 years ago

component_MailArchive: ArchiveContents resolution_fixed type_defect | by nick@foobar.org

Hi,

https://mailarchive.ietf.org/robots.txt is configured to disallow /. This is blocking search engines from indexing the site. Could someone fix the robots.txt file?

thanks,

Nick

Issue migrated from trac:2658 at 2021-09-22 16:55:19 +0500

hassanakbar4 commented 5 years ago

@hassanakbar4 changed priority from n/a to critical

hassanakbar4 commented 5 years ago

@hassanakbar4 changed status from new to closed

hassanakbar4 commented 5 years ago

@hassanakbar4 changed resolution from ` tofixed`

hassanakbar4 commented 5 years ago

@hassanakbar4 commented

Fixed in [886/mailarch]:

Modify robots.txt to allow search engine crawling. Add static browse index to assist crawlers in finding static pages. Fixes #2658