ionspin / kotlin-multiplatform-bignum

A Kotlin multiplatform library for arbitrary precision arithmetics
Apache License 2.0
350 stars 41 forks source link

Invalid decimal mode when using scale #161

Closed ionspin closed 3 years ago

ionspin commented 3 years ago

Describe the bug When using scale a big decimal is created with invalid precision. This later causes certain calculations to fail. To Reproduce

            val mode = DecimalMode(4, RoundingMode.ROUND_HALF_AWAY_FROM_ZERO, 3)
            val a = 500.toBigDecimal(decimalMode = mode)
            println("A exponent: ${a.exponent}")
            println("A precision: ${a.precision}")
            println("A scale: ${a.scale}")
            println("A significand: ${a.significand}")

Results in:

A exponent: 2
A precision: 4
A scale: 3
A significand: 500000 //Note that significand is actually 6 digits, not 4 as required by precision

Expected behavior Decimal mode and scale operatoins behavior is cleared up, and all calculations are correct

Additional context Issue was originally reported via email.