Implement Allow directive

spatie / robots-txt

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

https://spatie.be/en/opensource/php

MIT License

219 stars 36 forks source link

Implement Allow directive #18

Closed BenMorel closed 4 years ago

BenMorel commented 5 years ago

I don't have a strong incentive to work on this one right now, but I'm opening this issue to keep reminded that this library should support this directive.

See: http://robots-txt.com/ressources/robots-txt-allow/

spatie-bot commented 4 years ago

Dear contributor,

because this issue seems to be inactive for quite some time now, I've automatically closed it. If you feel this issue deserves some attention from my human colleagues feel free to reopen it.

BenMorel commented 4 years ago

Still relevant. How do I reopen?

spatie-bot commented 4 years ago

Dear contributor,

because this issue seems to be inactive for quite some time now, I've automatically closed it. If you feel this issue deserves some attention from my human colleagues feel free to reopen it.