search

bopoda / robots-txt-parser

PHP class for parse all directives from robots.txt files according to specifications
http://robots.jeka.by
MIT License
44 stars 17 forks source link

crowndiamond.org/robots.txt why empty rules parsed? #40

Closed bopoda closed 5 years ago

bopoda commented 5 years ago

Now it works.