search

WordPress / wordcamp.org

WordCamps are casual, locally-organized conferences covering everything related to WordPress.
https://wordcamp.org
127 stars 69 forks source link

Requests to WordCamp subdomain robots.txt files should not redirect #687

Open dd32 opened 3 years ago

dd32 commented 3 years ago

Describe the bug

per https://meta.trac.wordpress.org/ticket/5814

Requests to robots.txt files on wordcamp.org subdomains, like https://slc.wordcamp.org/robots.txt, redirect to the most recent/current year (e.g., https://slc.wordcamp.org/2019).

Instead, this should return an 'empty' robots.txt file, as follows:

User-agent: *

dd32 commented 3 years ago

FWIW Here's the current default WordPress robots.txt contents that we could also put in it's place: https://slc.wordcamp.org/2019/?robots=1

Including the Sitemap at least for the latest year event might be worthwhile.

jonoalderson commented 3 years ago

Ah, interesting.

Can we make the earlier sitemaps available, too? Would be nice to do something like:

User-agent: *
Sitemap: https://slc.wordcamp.org/2019/wp-sitemap.xml
Sitemap: https://slc.wordcamp.org/2018/wp-sitemap.xml
Sitemap: https://slc.wordcamp.org/2017/wp-sitemap.xml
...