issues
search
Folyd
/
robotstxt
A native Rust port of Google's robots.txt parser and matcher C++ library.
https://crates.io/crates/robotstxt
Apache License 2.0
88
stars
13
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Panics when slicing within char boundaries
#5
samuelfekete
opened
3 years ago
0
Document test dependencies
#4
let4be
opened
3 years ago
3
Why does allowed_by_robots and one_agent_allowed_by_robots parse robotstxt for each request?
#3
let4be
opened
3 years ago
2
panics when robots_txt is not valid
#2
iyzana
closed
3 years ago
2
crashes for deviantart robots.txt
#1
iyzana
closed
3 years ago
2