Normalization in other implementations

[sffc]

This week's slides discuss precedent for decimals from other programming languages. I did some quick research on the listed programming languages to see if they support trailing zeros.

Java: YES

https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html

The rounding mode determines how any discarded trailing digits affect the returned result. … stripTrailingZeros() Returns a BigDecimal which is numerically equal to this one but with any trailing zeros removed from the representation.

C#: YES

https://learn.microsoft.com/en-us/dotnet/api/system.decimal?view=net-8.0

The scaling factor also preserves any trailing zeros in a Decimal number. Trailing zeros do not affect the value of a Decimal number in arithmetic or comparison operations. However, trailing zeros might be revealed by the ToString method if an appropriate format string is applied.

Python: YES

https://docs.python.org/3/library/decimal.html

The decimal module incorporates a notion of significant places so that 1.30 + 1.20 is 2.50. The trailing zero is kept to indicate significance. This is the customary presentation for monetary applications. For multiplication, the “schoolbook” approach uses all the figures in the multiplicands. For instance, 1.3 1.2 gives 1.56 while 1.30 1.20 gives 1.5600.

Ruby: Wasn't able to determine for sure in the short amount of time I took to research, but I think it does not support them. Ruby defines decimals as: "arbitrary-precision floating point decimal arithmetic". When I play with them in irb I can't seem to get a trailing zero to roundtrip.

Postgresql: YES

https://www.postgresql.org/docs/8.1/datatype.html https://stackoverflow.com/questions/26920157/postgresql-adds-trailing-zeros-to-numeric

An example of the data I am inserting is 0.75 in the numeric(9,3) column and the value stored is 0.750. Another example for the numeric(9,4) column: I insert the value 12 and the DB is holding 12.0000.

@jessealama

[sffc]

I didn't go this deep but it would be interesting to see how the prior art handles the "middle ground" proposal suggested in the slide deck, namely whether lessThan and equals pay attention to the trailing zeros.

[jirkamarsik]

• Java
  • compareTo uses mathematical value and doesn't care about trailing zeroes (https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html#compareTo-java.math.BigDecimal-).
  • equals cares about trailing zeroes (https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html#equals-java.lang.Object-).
  • toString preserves trailing zeroes (https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html#toString--).
• C#
  • Comparison and equality operators compare by mathematical value only (i.e. they don't care about trailing zeroes).

    Trailing zeros do not affect the value of a Decimal number in arithmetic or comparison operations.

  • ToString can expose trailing zeroes.

    However, trailing zeros might be revealed by the ToString method if an appropriate format string is applied.

• Python
  • Comparison and equality operators compare by mathematical value only (i.e. they don't care about trailing zeroes).

    >>> Decimal("1.20") < Decimal("1.2") False >>> Decimal("1.20") == Decimal("1.2") True

  • Decimal objects also expose compare and compare_total methods.
  • compare is like the comparison operators, compares by mathematical value only (https://docs.python.org/3/library/decimal.html#decimal.Decimal.compare).
  • compare_total imposes a total ordering on Decimal values and takes trailing zeroes into account when comparing values (https://docs.python.org/3/library/decimal.html#decimal.Decimal.compare_total).
  • str (Python's toString) preserves trailing zeroes.

    >>> str(Decimal("1.20")) '1.20'

• Ruby
  • Ruby does seem to normalize all decimal values.

So in conclusion, Java, C# and Python's behavior is closest to the "middle ground" proposal. The primary comparison operators care only about the mathematical value and ignore trailing zeroes. The string conversion exposes trailing zeroes. On the other hand, Ruby adopts the always-normalize strategy.

On top of that, Python offers a compare_total method that can provide a total ordering of the decimal values by taking into account trailing zeroes and Java's equals method compares both value and scale, so that BigDecimals which are equal according to equals are indistinguishable using the other BigDecimal methods.

[sffc]

In Temporal we had similar questions about what fields to consider in .compare and .equals; what we landed on was .compare being more lenient and .equals being more strict, meaning .compare() == 0 doesn't imply .equals().

https://github.com/tc39/proposal-temporal/issues/523

@ljharb @ptomato

[jessealama]

Just to update this discussion with the status quo:

• We propose =, ≠, <, ≤, >, and ≥ that all compare mathematical values;
• We propose a compare method that works on all decimal values (even NaNs and infinities) and compares the full decimal data, allowing the programmer to distinguish mathematically equal but distinct-in-Decimal128-world values. The range will be { -1, 0, 1, NaN } (...the latter because of NaN pollution).

[sffc]

It looks like Python has a compare_total function

$ python3
Python 3.11.8 (main, May 26 2024, 00:40:16) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal
>>> Decimal("1.00") == Decimal("1.0")
True
>>> Decimal("1.00") < Decimal("1.0")
False
>>> Decimal("1.00").compare_total(Decimal("1.0"))
Decimal('-1')

[nicolo-ribaudo]

The answer for Postgresql is "NO", not "YES". Postgres does not store the precision together with the number, and instead it's part of the type of the column (so, there is no difference between trying to store 12.0 or 12.00 in any given column).