The marketing team is concerned that the many pages of the archive site are dominating Code for
America search results. It was decided that we'll take some steps to prevent the archive sites from
showing up on search engines, especially on Google. This adds a robots.txt file with the instruction for
bots to not crawl the pages of the archive site.
The marketing team is concerned that the many pages of the archive site are dominating Code for America search results. It was decided that we'll take some steps to prevent the archive sites from showing up on search engines, especially on Google. This adds a robots.txt file with the instruction for bots to not crawl the pages of the archive site.