akubera / bigdecimal-rs

Arbitrary precision decimal crate for Rust
Other
302 stars 73 forks source link

sqrt_with_context() doesn't respect rounding mode? #133

Closed jorisguex closed 1 month ago

jorisguex commented 3 months ago

The following code:

fn main() {
    let val = bigdecimal::BigDecimal::new(225.into(), 2);
    let prec = std::num::NonZero::new(1).unwrap();
    println!("sqrt({val}) with prec={prec}:");
    println!("Actual: {}", val.sqrt().unwrap());
    for rounding_mode in [bigdecimal::RoundingMode::Up, bigdecimal::RoundingMode::Down] {
        let res = val.sqrt_with_context(&bigdecimal::Context::new(prec, rounding_mode)).unwrap();
        println!("{rounding_mode:?}: {res}");
    }
}

produces the following output:

sqrt(2.25) with prec=1:
Actual: 1.500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Up: 1
Down: 1

I would have expected that when using a Context with RoundingMode::Up, the output value would be 2 instead of 1. The docs say "Take the square root of the number, using context for precision and rounding", so it seems like this is either a bug or I am doing something wrong?

akubera commented 3 months ago

Thanks for reporting it. The sqrt implementation is only rounding down (truncating) and only uses the context to set the precision.

I think I'll have time to check this next weekend.

danilopedraza commented 1 month ago

It seems like this was fixed in 8d38646, ed6cfc3, and fcdf76b

Should we close this?

akubera commented 1 month ago

Yup