search

ionspin / kotlin-multiplatform-bignum

A Kotlin multiplatform library for arbitrary precision arithmetics
Apache License 2.0
339 stars 40 forks source link

0 precision is unintuitive #257

Closed sproctor closed 1 year ago

sproctor commented 1 year ago

Describe the bug The docs claim that 0 means unlimited precision. This isn't true when scale is set. 0 precision is worthless. My suggestion is that when precision is 0 and scale is set in the DecimalMode for an operation, default to max(decimalPrecision, other.decimalPrecision).

ionspin commented 1 year ago

Precision turned out to be an awful API, my idea is to completely drop it in 0.4 and just keep scale or if no scale set unlimited precision

ionspin commented 1 year ago

I'm closing this as https://github.com/users/ionspin/projects/2 and initial issue #259 has been created to tackle DecimalMode/Scale API.