cockroachdb / apd

Arbitrary-precision decimals for Go
https://pkg.go.dev/github.com/cockroachdb/apd/v2
Apache License 2.0
622 stars 34 forks source link

Context.Rem fails for Rem(100E100, 10) #134

Open NoamTD opened 2 months ago

NoamTD commented 2 months ago

I attempted to run the following test:

func TestRem(t *testing.T) {
    a := apd.New(100, 100)
    b := apd.New(10, 1)
    var rem apd.Decimal
    _, err := apd.BaseContext.Rem(&rem, a, b)
    if err != nil {
        t.Fatal(err)
    }

    if !rem.IsZero() {
        t.Fatal("expected zero")
    }
}

I would expect this test to pass, but it fails:

=== RUN   TestRem
    rem_test.go:72: division impossible
--- FAIL: TestRem (0.00s)

FAIL

apd.BaseContext.Rem returns division impossible, which (I believe) is an incorrect result in this case.

mvdan commented 2 months ago

For some added context, we wanted to use Rem to be able to precisely tell if one apd.Decimal number was a multiple of another. We instead ended up with the following code, requiring two apd operations (Quo + Modf) instead of one (Rem):

_, err := apdCtx.Quo(&d, x, y)
if err != nil {
    return false, err
}
var frac apd.Decimal
d.Modf(nil, &frac)
return frac.IsZero(), nil