Rounding error

[Kaven01]

Hello, I found something that seems like a bug in BigNum library to me. I have current version, updated today.

So I have following code, that simulates an operation I have to do. There are printed results in comments. :

let toround = BigFloat(2) / BigFloat(100) print(toround.description) //0.0199999999999999999996

let pow = BigFloat(10).power(BigInt(2)) var res = toround * pow print (res.description) //1.999999999999999999959

res = res.rounded(.toNearestOrAwayFromZero) print (res.description) //1.0

res = res / pow print (res.description) //0.0099999999999999999998

Problem is the third result, just after .rounded() . 1.999999999999999999959 should have been rounded to 2.0, not 1.0 . Am I doing something wrong, or can you fix the bug, please?

[mgriebling]

This is a typical misunderstanding people have with floating point numbers. Some numbers just cannot be exactly represented as binary fractions. Your example of 0.02 is one such number. Decimal floating point numbers (as used in your hand-held calculator) use Decimal floating point math in which case 0.02 is exact and operations on such a number are also exact.

The number 0.02 in binary fractional (IEEE 754 32-bit format) = 00111110010011001100110011001101 which is exactly equal to 0.0199999995529651641845703125. The next higher binary fraction would be 00111100101000111101011100001011 which is exactly 0.02000000141561031341552734375. So, you see, it will never be possible to represent 0.02 exactly, unless you use decimal floating point numbers which are stored entirely differently.

I know you're thinking, with enough bits, eventually I'll be able to represent 0.02. Unfortunately, the answer is no. There are no finite combinations of bits in a binary floating-point fractional format that can ever represent this number.

Let's try it out with the BigNum library. Here is the 512-bit version of 0.02: .019999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999. You can try other resolutions but they'll all have the same issue. Good luck.

BTW, BigRat (big rational numbers) can exactly represent 0.02 as 2/100 or 1/50.

[dankogai]

The explanation above by @mgriebling is 💯.