tc39 / proposal-decimal

Built-in exact decimal numbers for JavaScript
http://tc39.es/proposal-decimal/
497 stars 18 forks source link

FormatNumericToString does not work with all Decimal128 values #169

Closed jessealama closed 2 months ago

jessealama commented 4 months ago

The FormatNumericToString AO delegates its work to the ToRawFixed and ToRawPrecision AOs after checking the sign of its argument and, for negative arguments, taking the absolute value. There were a couple of bits where Decimal128 values were not getting properly handled in FormatNumericToString; this PR fixes that. We also make clear that, when Decimal128 values are given to the ToRaw{Fixed,Precision} AOs, the solution to the equation to be solved is the cohort and quantum, so essentially no work needs to be done.

In other words, given this change, checking the correctness of FormatNumericToString on decimal arguments is reduced to checking that the ToRawFixed and ToRawPrecision AOs correctly handle decimals. Those are correct, AFAICS. If there are any bugs, though, I'm happy to work on them in this PR.

CC @sffc

littledan commented 4 months ago

Could you give an example of the bug, and what the output is before and after this fix?

jessealama commented 4 months ago

Could you give an example of the bug, and what the output is before and after this fix?

Here are a couple of examples of where we did the wrong thing before and how we do the right thing now:

  1. With -0.0𝔻, the old algorithm incorrectly replaced this with 0 (the mathematical value), thereby losing the trailing zero. This example can be generalized for any number of trailing zeroes. The same goes for +0.0𝔻 and any number of trailing zeroes you wish.
  2. With any negative Decimal128 value, e.g., « -1.2, -1 »𝔻 (which is just a fancy way of expressing -1.2), we would end up passing a negative Decimal128 value to the ToRawFixed resp. ToRawPrecision AOs, which require a non-negative Intl mathematical value.

AFAIK ToRawFixed and ToRawPrecision do the right thing with Decimal128 values.