decimalExpansion gives out unexpected values

chokokatana commented 1 year ago

I've tried to use the BDouble for storage but at some point I need to use decimalExpansion to format the value to end users, and I was getting really weird values. In a debug session I got the following:

(lldb) po percent
▿ 7450921770987/7450936270987
  - sign : false
  ▿ numerator : 1 element
    - 0 : 7450921770987
  ▿ denominator : 1 element
    - 0 : 7450936270987
  - magnitude : 0.0
  - _precision : 4

(lldb) po percent.decimalDescription
"0.99910"

(lldb) po percent.decimalExpansion(precisionAfterDecimalPoint: 2)
"0.910"

(lldb) po percent.decimalExpansion(precisionAfterDecimalPoint: 0)
"0.0"

(lldb) po percent.decimalExpansion(precisionAfterDecimalPoint: 7)
"0.9999981"

I'm pretty certain a value nearly 1 should not round to 0 when displayed to end users. Is decimalExpansion not the function I'm expecting and should I be using something else? I had expected the value to round to either 1.0 or 0.9.

mkrd commented 1 year ago

Can you set the "rounded" parameter to false and check again? I think the problem might have something to do with rounding at a given precision level.

But still that doesn’t make sense why for example the case with 2 doesn’t give at least two 9's

mkrd commented 1 year ago

Okay I got it, in the case of precision=2, it computes 0.999 so since the third 9 is >= 5, the second 9 is rounded up, but there is a bug so it is replaced with 10 and the number becomes 0.910

mkrd commented 1 year ago

Could be solved easily by checking the last digit of the derived BInt before converting to a string, if .divmod(10).remainder >= 5 then take .divmod(10).quotient + 1, else .divmod(10).quotient. Only after that take the Bint.description and prune 0's from the end if needed (so if 999 is rounded to 1000, then 1.000 has to remove the last zero

mkrd / Swift-BigInt

decimalExpansion gives out unexpected values #72