search

hbz / digitalisiertedrucke

Implements http://digitalisiertedrucke.de/
0 stars 0 forks source link

Set up robots.txt to avoid indexing of test.digitalisiertedrucke.de #22

Closed fsteeg closed 7 years ago

fsteeg commented 7 years ago

Also beta.digitalisiertedrucke.de?

fsteeg commented 7 years ago

Deployed to staging: http://test.digitalisiertedrucke.de/robots.txt

Checked with http://tool.motoricerca.info/robots-checker.phtml

acka47 commented 7 years ago

+1

acka47 commented 7 years ago

I guess we should also disallow crawling beta.