issues
search
bopoda
/
robots-txt-parser
PHP class for parse all directives from robots.txt files according to specifications
http://robots.jeka.by
MIT License
44
stars
17
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Optimize isUrlAllow()
#18
LeMoussel
closed
7 years ago
2
Some robots.txt files for robustness testing
#17
LeMoussel
closed
7 years ago
1
Typo on directive
#16
LeMoussel
opened
7 years ago
6
Remove asterisk (*) at the end of path.
#15
LeMoussel
opened
7 years ago
3
IsUrlAllow Google test
#14
LeMoussel
closed
7 years ago
8
Sorted robots.txt
#13
LeMoussel
closed
7 years ago
2
Poor performance isUrlAllow() function
#12
LeMoussel
closed
7 years ago
6
init RobotsTxtValidator with isUrlAllow method and phpunit
#11
bopoda
closed
7 years ago
0
need to have 2 methods (isUrlAllow, isUrlDisallow) to check is url allow or not
#10
bopoda
closed
7 years ago
1
Check valid path
#9
LeMoussel
opened
7 years ago
6
Get Url sitemaps: added getSitemaps method. Resolve #7
#8
bopoda
closed
7 years ago
3
Get Url sitemaps
#7
LeMoussel
closed
7 years ago
3
Save User-Agent as is without removing whitespaces Fixes #5
#6
bopoda
closed
7 years ago
0
Suppress space in User_Agent
#5
LeMoussel
closed
7 years ago
1
improve new line function and add some http tests
#4
bopoda
closed
9 years ago
0
Запоминаем абсолютные пути как есть
#3
bopoda
closed
9 years ago
0
Не обрабатывать громадные файлы robots.txt
#2
bopoda
closed
7 years ago
1
Правильно обрабатывать абсолютные urls в директивах allow/disallow
#1
bopoda
closed
9 years ago
1
Previous