Use robots.txt file to disable preview site

DOI-ONRR / onrr.gov-site

We will use this repo to manage our work on the onrr.gov website

11 stars 3 forks source link

Use robots.txt file to disable preview site #3066

Closed eelzi-ONRR closed 5 months ago

eelzi-ONRR commented 5 months ago

Figure out and implement changes to robots.txt file in order to remove preview site from search results

eelzi-ONRR commented 5 months ago

This might be a bigger issue - thinking through it more we also should indicate canonical URLs so lessen the number of results that come up form the old site

eelzi-ONRR commented 5 months ago

@greensomes robots-dev.txt added via PR #3092

eelzi-ONRR commented 5 months ago

@greensomes robots-dev.txt added via PR #3092