search

paupino / rust-decimal

Decimal number implementation written in pure Rust suitable for financial and fixed-precision calculations.
https://docs.rs/rust_decimal/
MIT License
976 stars 179 forks source link

Pow overflows #668

Open rawhuul opened 1 month ago

rawhuul commented 1 month ago

Hi there, I am trying to calculate sigmoid of &[Decimal] here, which fails while using Decimal but works fine after being converted to f64, is this expected behaviour or a bug?

const E: f64 = 2.71828182845904523536028747135266250_f64; // 2.7182818284590451f64

let euler = Decimal::from_i128_with_scale(27182818284590451, 16);

let sgm = |d: &Decimal| -> Decimal { Decimal::ONE / (Decimal::ONE + euler.powd(-*d)) };

let sgf = |d: &Decimal| -> Decimal {
  let f: f64 = (*d).try_into().unwrap();
  Decimal::from_f64(1.0 / (1.0 + f64::powf(E, -f))).unwrap()
};

self.iter().map(sgf).collect()
# Output while using closure sgm
thread 'tests::test_float64data' panicked at ~\.cargo\registry\src\index.crates.io-6f17d22bba15001f\rust_decimal-1.35.0\src\maths.rs:290:21:
Pow overflowed

# Output while using function: sfm
[0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.731058578630005, 0.731058578630005, 0.731058578630005, 0.731058578630005, 0.731058578630005, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.880797077977882]
paupino commented 1 month ago

This is unfortunately related to this issue: https://github.com/paupino/rust-decimal/issues/431

The long story short is that the way that powd works is it tries to calculate the exponent and then reduce the number back to the correct scale (rounding if necessary). Currently this uses multiplication to derive this - of which behind the scenes multiplication temporarily expands the decimal to 192 bits before shrinking back to 96 bits. Because of this, every "iteration" within powd effectively expands and then shrinks - which of course is not effective and causes the overflow.

What should happen is that the calculation should expand to 192 bits (or more) but not scale back to 96 bits until the very last opportunity. This would help resolve situations like you're seeing above.

Overall, the fix isn't difficult, however requires some critical path refactoring to allow for more lenient multiplication steps.