exponent too large to convert to mpfr2 (32-bit)

vbraun commented 4 years ago

This happens often, but not always when testing on 32-bit:

File "src/sage/tensor/differential_form_element.py", line 328, in sage.tensor.differential_form_element.DifferentialForm
Failed example:
    form2.diff().degree()
Expected:
    2
Got:
    exception: exponent too large to convert to mpfr2
**********************************************************************
1 item had failures:
   1 of  28 in sage.tensor.differential_form_element.DifferentialForm
    [287 tests, 1 failure, 0.69 s]

Upstream: Not yet reported upstream; Will do shortly.

CC: @wbhart @fredrik-johansson @EmmanuelCharpentier

Component: basic arithmetic

Author: Volker Braun, Erik Bray

Branch: dc3df3a

Reviewer: Erik Bray, Volker Braun, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/28817

vbraun commented 4 years ago

Branch: u/vbraun/exponent_too_large_to_convert_to_mpfr2__32bit

vbraun commented 4 years ago

comment:2

The "exception: exponent too large to convert to mpfr" is coming from arb. It should have called flint_abort which is just a define for abort and terminated the process, but for some reason it didnt. The final 2 is printed, too.

New commits:

6859145 Fix some places where arb/mpfr conversion of python ints can break on Py3

vbraun commented 4 years ago

Commit: 6859145

vbraun commented 4 years ago

comment:3

Backtrace:

    /home/buildbot/slave/sage_git/build/local/lib/libarb.so.2(arf_get_mpfr+0x13d)[0xd0b5657d]
    /home/buildbot/slave/sage_git/build/local/lib/libarb.so.2(arb_get_interval_mpfr+0x85)[0xd0bc29d5]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/rings/real_arb.cpython-37m-i386-linux-gnu.so(+0x16d4e)[0xd0ac6d4e]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/rings/complex_arb.cpython-37m-i386-linux-gnu.so(+0x48d29)[0xd0a6ad29]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/rings/complex_arb.cpython-37m-i386-linux-gnu.so(+0x4949f)[0xd0a6b49f]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/structure/coerce_maps.cpython-37m-i386-linux-gnu.so(+0xd3c5)[0xf5f533c5]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/structure/coerce_maps.cpython-37m-i386-linux-gnu.so(+0x17b97)[0xf5f5db97]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/structure/parent.cpython-37m-i386-linux-gnu.so(+0x1326e)[0xf646426e]
    /home/buildbot/slave/sage_git/build/local/lib/libpython3.7m.so.1.0(+0xc37b3)[0xf7cca7b3]
    /home/buildbot/slave/sage_git/build/local/lib/libpython3.7m.so.1.0(+0x713a7)[0xf7c783a7]
    /home/buildbot/slave/sage_git/build/local/lib/libarb.so.2(arf_get_mpfr+0x13d)[0xd0b5657d]
    /home/buildbot/slave/sage_git/build/local/lib/libarb.so.2(arb_get_interval_mpfr+0xa7)[0xd0bc29f7]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/rings/real_arb.cpython-37m-i386-linux-gnu.so(+0x16d4e)[0xd0ac6d4e]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/rings/complex_arb.cpython-37m-i386-linux-gnu.so(+0x48d29)[0xd0a6ad29]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/rings/complex_arb.cpython-37m-i386-linux-gnu.so(+0x4949f)[0xd0a6b49f]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/structure/coerce_maps.cpython-37m-i386-linux-gnu.so(+0xd3c5)[0xf5f533c5]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/structure/coerce_maps.cpython-37m-i386-linux-gnu.so(+0x17b97)[0xf5f5db97]
    /home/buildbot/slave/sage_git/build/local/lib/python3.7/site-packages/sage/structure/parent.cpython-37m-i386-linux-gnu.so(+0x1326e)[0xf646426e]
[...]

vbraun commented 4 years ago

comment:4

The error comes from evaluating

exp(cos(x)).subs({x: CIF(-50.3035873016558,- 25.1526638467043)})

when testing whether a certain coefficient of the differential form is zero. This value cannot be represented in 32-bit mpfr, we catch the exception and continue testing at a different point. I think this part just works as intended.

The problem is simply that arb writes 'exception: exponent too large to convert to mpfr2' to its stderr, which may or may not be captured in the doctest stderr stream.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

aabb1ce Fix some places where arb/mpfr conversion of python ints can break on Py3

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 6859145 to aabb1ce

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7ee4a81 Silence the arb error message

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from aabb1ce to 7ee4a81

vbraun commented 4 years ago

Author: Volker Braun

embray commented 4 years ago

comment:8

Ticket retargeted after milestone closed

vbraun commented 4 years ago

comment:9

This happens almost always on the buildbot...

embray commented 4 years ago

comment:10

Looks to me like the right fix, though it could be a bit surprising that some of these methods accept Python ints only if they're within intmax, even if they could work with arbitrary sized ints.

embray commented 4 years ago

comment:11

This pattern from sage.rings.integer might be good to use elsewhere as well:

        if is_small_python_int(b):
            tmp = b
            if (tmp & 1) == 0:
                raise ValueError("Jacobi symbol not defined for even b.")
            return mpz_kronecker_si(self.value, tmp)
        if not isinstance(b, Integer):
            b = Integer(b)

embray commented 4 years ago

comment:12

Yeah, for example ComplexBall.pow is giving inconsistent results between 2^65 and int(2^65). Fix+test incoming...

embray commented 4 years ago

comment:13

Can you try this?

New commits:

`00d1d86`	`Fix some places where arb/mpfr conversion of python ints can break on Py3`
`e7b90fe`	`Silence the arb error message`
`d5a1c22`	`Trac #28817: Fix use of large Python ints in these case and include regression tests.`

embray commented 4 years ago

Changed branch from u/vbraun/exponent_too_large_to_convert_to_mpfr2__32bit to public/exponent_too_large_to_convert_to_mpfr2__32bit

embray commented 4 years ago

Changed commit from 7ee4a81 to d5a1c22

vbraun commented 4 years ago

Changed author from Volker Braun to Volker Braun, Erik M. Bray

vbraun commented 4 years ago

comment:14

Thanks, looks good to me

vbraun commented 4 years ago

Reviewer: Erik M. Bray, Volker Braun

vbraun commented 4 years ago

comment:15

I'm getting:

sage -t --long --warn-long 36.1 src/sage/rings/real_arb.pyx
**********************************************************************
File "src/sage/rings/real_arb.pyx", line 2876, in sage.rings.real_arb.RealBall.__pow__
Failed example:
    RBF(e)**(-1r)
Expected:
    [0.367879441171442 +/- ...e-16]
Got:
    [1.792143500744e-187 +/- 5.50e-200]
**********************************************************************
File "src/sage/rings/real_arb.pyx", line 2888, in sage.rings.real_arb.RealBall.__pow__
Failed example:
    RBF(e)^int(2^65)
Expected:
    [+/- ...e+16022638320587143290]
Got:
    [+/- 2.79e+16022638320587144275]
**********************************************************************
1 item had failures:
   2 of  14 in sage.rings.real_arb.RealBall.__pow__
    [560 tests, 2 failures, 0.23 s]
----------------------------------------------------------------------
sage -t --long --warn-long 36.1 src/sage/rings/real_arb.pyx  # 2 doctests failed
----------------------------------------------------------------------

I also got yet another exception: exponent too large to convert to mpfr2, so I'll patch out the other occurence in arb as well.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from d5a1c22 to daa6f8c

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

daa6f8c Remove remaining "exponent too large to convert to mpfr"

vbraun commented 4 years ago

comment:18

Erik, are you going to work on the test failures that your patch introduced? If not I suggest we move it to a different ticket...

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from daa6f8c to dc3df3a

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

dc3df3a Remove remaining "exponent too large to convert to mpfr"

vbraun commented 4 years ago

comment:20

Eric's commit is moved to #29137

dimpase commented 4 years ago

Upstream: Not yet reported upstream; Will do shortly.

dimpase commented 4 years ago

comment:21

not reported upstream?

dimpase commented 4 years ago

comment:22

reported: https://github.com/fredrik-johansson/arb/issues/302

dimpase commented 4 years ago

comment:23

As usual, the question is whether it's really needed to deviate from upstream here.

vbraun commented 4 years ago

comment:24

I don't have time to always check each 32-bit buildbot whether it tripped over this issue again, so unless you want to do that job its needed.

dimpase commented 4 years ago

comment:25

OK, good. Hopefully the arb patch won't be needed after Fredrik improves API...

dimpase commented 4 years ago

Changed reviewer from Erik M. Bray, Volker Braun to Erik M. Bray, Volker Braun, Dima Pasechnik

vbraun commented 4 years ago

Changed branch from public/exponent_too_large_to_convert_to_mpfr2__32bit to dc3df3a

mkoeppe commented 4 years ago

Changed commit from dc3df3a to none

mkoeppe commented 4 years ago

Changed reviewer from Erik M. Bray, Volker Braun, Dima Pasechnik to Erik Bray, Volker Braun, Dima Pasechnik

mkoeppe commented 4 years ago

Changed author from Volker Braun, Erik M. Bray to Volker Braun, Erik Bray

sagemath / sage

exponent too large to convert to mpfr2 (32-bit) #28817