bopoda robots-txt-parser issues

bopoda / robots-txt-parser

PHP class for parse all directives from robots.txt files according to specifications

http://robots.jeka.by

MIT License

44 stars 17 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Parsing rules is platform dependent

#68 Emosewaj opened 3 months ago
0
Init php.yml for github actions

#67 bopoda opened 7 months ago
0
tests: Get rid of phpunit deprecations, migrate phpunit configuration

#66 bopoda closed 7 months ago
0
add supported php 8

#65 tacman closed 7 months ago
1
remove $ so gitclip works

#64 tacman closed 7 months ago
1
Wrong boolean value in RobotsTxtValidator.php?

#63 scott8035 opened 11 months ago
1
Fix Crawl-Delay issue

#62 bopoda closed 3 years ago
0
Fixed deprecation

#61 lyrixx closed 3 years ago
1
Fix deprecation about usort

#60 lyrixx closed 3 years ago
0
Crawl Delay of 0 will get ignored

#59 offsky closed 3 years ago
1
Avoid possible Compilation failed errors when run regexp for isUrlAllow

#58 bopoda closed 4 years ago
0
Fix travisCI builds

#57 bopoda closed 4 years ago
0
Fix of not allowed char

#56 phenixiim closed 5 years ago
3
Added composer.json file and fixed file structure

#55 bopoda closed 5 years ago
0
Fixed error when a directive is empty

#54 lyrixx closed 5 years ago
0
Could you publish this package to packagist ?

#53 lyrixx closed 5 years ago
6
fix .travis.yml use trusty Ubuntu 14.04

#52 bopoda closed 7 years ago
0
Resolve Issue #43

#51 LeMoussel closed 7 years ago
3
Directive with '+' not handled correctly, escape '+', resolve #48

#50 bopoda closed 7 years ago
0
Add test cases according to Google robots.txt tester

#49 bopoda closed 7 years ago
0
Directive with '+' not handled correctly

#48 LeMoussel closed 7 years ago
1
Handle list of user-agents

#47 bopoda closed 7 years ago
0
Order of precedence for google user agents

#46 kennylajara closed 3 years ago
7
Update README.md (Solve #42)

#45 kennylajara closed 7 years ago
1
Add phpunit tests with cases from t1gor/Robots.txt-Parser-Class

#44 bopoda closed 7 years ago
0
Problems with iconv()

#43 LeMoussel closed 7 years ago
2
Language and version

#42 kennylajara closed 7 years ago
3
Fixing Typo

#41 kennylajara closed 7 years ago
1
crowndiamond.org/robots.txt why empty rules parsed?

#40 bopoda closed 5 years ago
1
RobotsTxtParser significantly improve performance (~1000x)

#39 bopoda closed 7 years ago
0
Rename tests use prefix Test

#38 bopoda closed 7 years ago
0
Fix phpunit tests, compatible with different phpunit versions

#37 bopoda closed 7 years ago
0
No user agent

#36 LeMoussel closed 7 years ago
3
Add tests for invalid encoding robots.txt

#35 bopoda closed 7 years ago
0
Rules not handled correctly if dont't find User Agent

#34 LeMoussel closed 7 years ago
0
Resolve iconv() notice error

#33 LeMoussel closed 7 years ago
1
Rule: host not handled

#32 LeMoussel closed 7 years ago
1
Get relative url

#31 LeMoussel closed 7 years ago
1
isUrlAllow() not handled correctly if no path in Url.

#30 LeMoussel closed 7 years ago
1
Handle noindex directive

#29 LeMoussel closed 7 years ago
2
Handle noindex directive

#28 LeMoussel closed 7 years ago
0
Rules not handled correctly

#27 LeMoussel closed 7 years ago
6
Rule: noindex not handled

#26 LeMoussel closed 7 years ago
1
Remove comments in content

#25 LeMoussel closed 7 years ago
1
Check valid content

#24 LeMoussel opened 7 years ago
2
RobotsTxtParser make values case-sensitive

#23 bopoda closed 7 years ago
0
add test cases from google specification

#22 bopoda closed 7 years ago
0
add phpunit tests from google robots examples

#21 bopoda closed 7 years ago
0
Poor perfomance when parsing large file.

#20 LeMoussel closed 7 years ago
2
add 2 tests to check isUrlAllow

#19 bopoda closed 7 years ago
0