spatie / robots-txt

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers
https://spatie.be/en/opensource/php
MIT License
219 stars 36 forks source link

Fixes (see commit msg) #13

Closed RobinDev closed 5 years ago

RobinDev commented 5 years ago

Fix #10 #11 #12

RobinDev commented 5 years ago

Thanks for your review.

I will rebuild my changes trying to integrate your comments. I was equally wrong on my interpretation of parsing a robots.txt, i read again google docs and I will do a lot of change in my commit "Fix Wildcard check in Robots.txt and Headers".

I update PR before monday.

RobinDev commented 5 years ago

Except for the loop on tmp var, I think I resolved every point in your code review.

RobinDev commented 5 years ago

Do you think it's ok now ?

mabasic commented 5 years ago

If it does what I think that it does, it would reduce my code by a few lines. Awesome! When will it be implemented?

spatie-bot commented 5 years ago

Dear contributor,

because this pull request seems to be inactive for quite some time now, I've automatically closed it. If you feel this pull request deserves some attention from my human colleagues feel free to reopen it.