cmpute / dashu

A library set of arbitrary precision numbers implemented in Rust.
Apache License 2.0
74 stars 9 forks source link

`Display` impl for `FBig` gives wrong result #48

Closed Shoeboxam closed 6 months ago

Shoeboxam commented 6 months ago

The following code outputs 0.1:

println!("{}", FBig::<Up>::try_from(0.5)?);

However, the internal representation is correct. This passes and outputs 1 * 2 ^ -1 (prec: 53):

assert_eq!(FBig::<Up>::try_from(0.5)?.to_f64().value(), 0.5);
println!("{:?}", FBig::<Up>::try_from(0.5)?);
cmpute commented 6 months ago

If you use FBig with default generic params, then the representation is in binary digits. That is, 0.1 is the floating point notation in base 2, which is equivalent to 0.5 in base 10. This is a desired behavior (documented here), though I will make it clearer in the user guide.

To diplay the number in decimal digits, you can convert it to DBig using to_decimal(), or convert it to f32/f64

Shoeboxam commented 6 months ago

Ah, sorry, I wasn't expecting floats to print in base-2, but I guess it makes sense to print in the same B. Thanks for the lib as always!

cmpute commented 6 months ago

The major reason I don't want to print in base-10 is that, base conversion can be pretty expensive, and I think it's better to make the conversion explicit,