Alexhuszagh / rust-lexical

Fast numeric to- and from-string conversion routines.
Other
296 stars 37 forks source link

[BUG] Regression with Panic for Format API #157

Closed Alexhuszagh closed 1 month ago

Alexhuszagh commented 1 month ago

Description

When attempting to parse feature floats without the compact feature with digit separators, there is a panic due to Self::IS_CONTIGUOUS. This seems to be an incorrect attempt to parse multi-digit separators and a test must be introduced to avoid regressions.

Alexhuszagh commented 1 month ago

A minimal failing test case is:

#[test]
#[cfg(feature = "format")]
fn f64_exponent_consecutive_digit_separator_test() {
    const FORMAT: u128 = rebuild(format::PERMISSIVE)
        .exponent_internal_digit_separator(true)
        .exponent_consecutive_digit_separator(true)
        .digit_separator(num::NonZeroU8::new(b'_'))
        .build();
    let options = Options::new();
    assert!(f64::from_lexical_with_options::<FORMAT>(b"31.01e7__1", &options).is_ok());
    assert!(f64::from_lexical_with_options::<FORMAT>(b"31.01e_71", &options).is_err());
    assert!(f64::from_lexical_with_options::<FORMAT>(b"31.01e71_", &options).is_err());
}

Here, the integer and fraction digits are consecutive, so it should be able to use multi-digit optimizations, even if the exponent is not. It seems to be using is_contiguous on the byte and not the actual external iterator.