Added x-robots-tag: none. Ref #30

spatie / robots-txt

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

https://spatie.be/en/opensource/php

MIT License

219 stars 36 forks source link

Closed ihah closed 3 years ago

ihah commented 3 years ago

Ref #30

Based on google docs X-Robots-Tag none directive should have priority in any case.

freekmurze commented 3 years ago

Thanks!