rust-bakery / nom

Rust parser combinator framework
MIT License
9.18k stars 792 forks source link

`tag_no_case` panicks while matching certain unicode capitalized characters. #1719

Open DelSkayn opened 6 months ago

DelSkayn commented 6 months ago

Test case

fn main() {
    let _ = nom::bytes::complete::tag_no_case::<_,_,nom::error::Error<&str>>("k")("K");
}

The tag_no_case function can panic whenever a character in a to be matched string lowercases to a character with a shorter byte pattern.

In the above example the K to be matched is actually the kelvin sign with the UTF-8 byte pattern of 0xE2 0x84 0xAA. This character has a the normal single byte k as its lowercase character and therefore matches the tag.

The tag_no_case implementation, after matching a string to the tag, splits the given string with the length of the tag: https://github.com/rust-bakery/nom/blob/e87c7da9fa2ddb943306369f6c6d9e914256edbc/src/bytes/mod.rs#L76

This assumes that the length in bytes of the matched characters is the same as the tag it was matched to. But, as in the above example, this is not always the case, resulting in the function sometimes trying to split a string outside of a character boundary and panicking