cmpute / dashu

A library set of arbitrary precision numbers implemented in Rust.
Apache License 2.0
74 stars 9 forks source link

Improve precision-computation for binary FBig oparations #24

Closed MamfTheKramf closed 1 year ago

MamfTheKramf commented 1 year ago

The precision of the result of a binary oparation between two FBigs (e.g. add) is the maximum of the precision of the arguments, as stated in the documentation. But this leads to some probably undesired behaviour. For example in this code:

let a = Decimal::from_str_native("0.25").unwrap();
let b = Decimal::from_str_native("1000").unwrap();
println!("a: {}", a);           // prints "a: 0.25"
println!("b: {}", b);           // prints "b: 1000"
println!("a + b: {}", a + b);   // prints "a + b: 1000"

let a = Real::ONE << 8;
let b = Real::from_str_native("0.101101").unwrap();
println!("a: {}", a);           // prints "a: 100000000"
println!("b: {}", b);           // prints "b: 0.101101"
println!("a + b: {}", a + b);   // prints "a + b: 100000000"

Maybe the precision-computation for such oparations should be changed such that no loss occurs.

cmpute commented 1 year ago

This is intentionally designed as so. The precision of "0.25" and "1000" are 3 and 4 respectively, so the result will have a precision of 4 by default. If you want unlimited precision, you can set the precision to zero. Try:

let a = Decimal::from_str_native("0.25").unwrap().with_precision(0).value();
let b = Decimal::from_str_native("1000").unwrap().with_precision(0).value();
println!("a + b: {}", a + b);

let a = Real::ONE << 8; // Real::ONE already has a precision of 0
let b = Real::from_str_native("0.101101").unwrap().with_precision(0).value();
println!("a + b: {}", a + b);

Maybe I should add a separate function to explicitly mark the number has unlimited precision.

MamfTheKramf commented 1 year ago

If this is intentional, then everything is alright. I just wanted to make sure that this isn't some behaviour that was supposed to be fixed by simply choosing the max-precision.