search

Folyd / robotstxt

A native Rust port of Google's robots.txt parser and matcher C++ library.
https://crates.io/crates/robotstxt
Apache License 2.0
88 stars 13 forks source link

Panics when slicing within char boundaries #5

Open samuelfekete opened 3 years ago

samuelfekete commented 3 years ago

The parser can panic if it tries to slice a string within char boundaries. 

thread '<unnamed>' panicked at 'byte index 58198 is not a char boundary; it is inside '\u{a0}'
...
/root/.cargo/registry/src/github.com-1ecc6299db9ec823/robotstxt-0.3.0/src/parser.rs:169:57