search

basvandijk / scientific

Arbitrary-precision floating-point numbers represented using scientific notation
BSD 3-Clause "New" or "Revised" License
73 stars 40 forks source link

Integer overflow in exponent #62

Open ksvanhorn opened 6 years ago

ksvanhorn commented 6 years ago

scientific 1 (maxBound::Int) * scientific 1 (maxBound::Int) evaluates to 1.0e-2.

basvandijk commented 6 years ago

Yes, this is unfortunate. Scientific uses an Int to represent the exponent so it suffers from the same integer overflows that Int suffers from. I've an old TODO to switch the exponent to Integer to make Scientific truly arbitrary precision.

basvandijk commented 6 years ago

I started doing some work on this on the integer-exponent branch.

mbj commented 2 years ago

I'd prefer we throw an impure error instead of producing wrong values, till we can get arbitrary precision would a patch be accepted that calls Prelude(error) on any operation that would result in corruption?