Closed SeaOfCrystalGreens closed 4 months ago
Sorry, which calculation do you think is incorrect?
You are not going to find any errors here in basic arithmetic operations. This is a reliable, time-tested library.
Note that precision refers to number of significant digits not decimal places.
Comparison of : (2-(2-(1/3)) === (1/3)
Decimal.set({ precision: 3, rounding: 5 })
let n1 = new Decimal("1")
let n2 = new Decimal("2")
let n3 = new Decimal("3")
let a = n1.div(n3)
let b = n2.sub(n2.sub(n1.div(n3)))
console.log(a.equals(b))
Before code change, results in false. After equality holds.
It looks like it holds after decimal places are of equal length (erroneously increased precision).
If all works as intended then I do apologize for opening issue. I'll try to understand it more carefully.
1 / 3
to a precision of 3 significant digits is 0.333
.
2 - 0.333
is exactly 1.667
, but rounded to a precision of 3 significant digits it is 1.67
.
2 - 1.67 = 0.33
exactly.
0.333 != 0.33
, as expected.
Normally, a much greater precision is used for calculations (for example, the default precision is 20) and then toSignificantDigits
or toDecimalPlaces
or toFixed
etc. is used to round the result to the desired final precision.
.sub method seems to calculate to precision one less than precision set for Decimal object
after changing line https://github.com/MikeMcl/decimal.js/blob/7f01abd83d5632005d5f97ddbea41e069ee69a74/decimal.js#L1296 to
in https://github.com/MikeMcl/decimal.js/blob/7f01abd83d5632005d5f97ddbea41e069ee69a74/decimal.js#L1264
it seems to start calculating correctly
Although maybe it is intended behavior.
This is ephemery fix. Intended only to highlight issue.
If unintended, what parts of code might cause shortening of precision? To what extend it might be in other parts of code?