Using 0 (unlimited) precision with a scale yields 0 when the number of digits exceeds the scale

[joffrey-bion]

Describe the bug

When using 0 (unlimited) precision with a scale, I get 0 (with weird exponents) each time I have numbers that have more digits than the scale.

To Reproduce

val mode = DecimalMode(roundingMode = RoundingMode.ROUND_HALF_AWAY_FROM_ZERO, scale = 4)
println(BigDecimal.fromInt(123, decimalMode = mode)) // 1.23E+2 (OK)
println(BigDecimal.fromInt(1234, decimalMode = mode)) // 1.234E+3 (OK)
println(BigDecimal.fromInt(12345, decimalMode = mode)) // 0.0E+4 (why?)
println(BigDecimal.fromInt(12345678, decimalMode = mode)) // 0.0E+16 (why?)

Expected behavior

In this setup, I would expect the scale to be the only thing that's bounded, so that if digits after the decimal point keep going, we round them to the scale. I would expect 12345 to yield 1.2345E+4 and 12345678 to yield 1.2345678E+7.

Platform

• JVM

[joffrey-bion]

Interestingly, when directly using a decimal number, it doesn't give 0:

val mode = DecimalMode(roundingMode = RoundingMode.ROUND_HALF_AWAY_FROM_ZERO, scale = 4)
println("12345678.1234567".toBigDecimal(decimalMode = mode)) // 1.23456781235E+7 (OK)

[ionspin]

Hi @joffrey-bion, thanks for reporting, I'll look into this.

[mihbor]

I'm getting a potentially related issue when dividing with DecimalMode with unlimited precision and a fixed scale: Dividing 200.5230146281866563757715403019550414480770532190116073 by 1111.99009855 with DecimalMode(roundingMode = RoundingMode.ROUND_HALF_CEILING, scale = 8) yields: 0.00000000

Platform

• JS Version
• 0.3.8

Workaround: use big but limited decimalPrecision, e.g. DecimalMode(decimalPrecision = 64, roundingMode = RoundingMode.ROUND_HALF_CEILING, scale = 8)