issues
search
bopoda
/
robots-txt-parser
PHP class for parse all directives from robots.txt files according to specifications
http://robots.jeka.by
MIT License
44
stars
17
forks
source link
Save User-Agent as is without removing whitespaces Fixes #5
#6
Closed
bopoda
closed
7 years ago
bopoda
commented
7 years ago
Resolve #5
Resolve #5