bopoda / robots-txt-parser

PHP class for parse all directives from robots.txt files according to specifications
http://robots.jeka.by
MIT License
44 stars 17 forks source link

Avoid possible Compilation failed errors when run regexp for isUrlAllow #58

Closed bopoda closed 4 years ago

bopoda commented 4 years ago