enclosure regressions with arb 2.13.0

[videlec]

While moving to arb 2.13.0 in Sage (see upstream ticket #24927) I noted the following enclosure regressions

**********************************************************************
File "complex_arb.pyx", line 2798, in sage.rings.complex_arb.ComplexBall.hypergeometric
Failed example:
    CBF(2+3*I).hypergeometric([1/4,1/3],[1/2])
Expected:
    [0.7871684267473 +/- 7.34e-14] + [0.2749254173721 +/- 9.23e-14]*I
Got:
    [0.7871684267473 +/- 7.38e-14] + [0.2749254173721 +/- 9.22e-14]*I
**********************************************************************
File "complex_arb.pyx", line 2800, in sage.rings.complex_arb.ComplexBall.hypergeometric
Failed example:
    CBF(2+3*I).hypergeometric([1/4,1/3],[1/2],regularized=True)
Expected:
    [0.4441122268685 +/- 3.96e-14] + [0.1551100567338 +/- 5.75e-14]*I
Got:
    [0.4441122268685 +/- 3.98e-14] + [0.1551100567338 +/- 5.77e-14]*I
**********************************************************************
File "complex_arb.pyx", line 3648, in sage.rings.complex_arb.ComplexBall.legendre_P
Failed example:
    CBF(1/2).legendre_P(5)
Expected:
    [0.08984375000000000 +/- 4.5...e-18]
Got:
    [0.0898437500000000 +/- 7.29e-17]
**********************************************************************
File "complex_arb.pyx", line 3686, in sage.rings.complex_arb.ComplexBall.legendre_Q
Failed example:
    CBF(-10).legendre_Q(5, 325/100)
Expected:
    [-83825154.36008 +/- 4.94e-6] + [-34721515.80396 +/- 5.40e-6]*I
Got:
    [-83825154.36008 +/- 5.02e-6] + [-34721515.80396 +/- 5.42e-6]*I

[fredrik-johansson]

Thanks for reporting these. I guess it should be possible to bisect to find the specific commits where the enclosures got larger?

[arbguest]

On the arb development branch (https://github.com/fredrik-johansson/arb/commit/512ada96748dc64d29b8113e80d60f78a528c3f6), for these four examples with prec=53 and printing with digits=13 I see

(0.7871684267473 + 0.2749254173721j)  +/-  (6.38e-14, 4.54e-14j)
(0.4441122268685 + 0.1551100567338j)  +/-  (3.59e-14, 2.55e-14j)
(0.08984375 + 0j)  +/-  (7.29e-17, 0j)
(-83825154.36008 - 34721515.80396j)  +/-  (2.2e-06, 2.02e-06j)

indicating that the enclosures of the first, second, and fourth examples have tightened since 2.12 and 2.13, but that the enclosure for the third example is still regressed to its 2.12 state.

I didn't do a git bisection search for the commit responsible for the regression, but if arb_hypgeom/legendre_p_ui_rec.c in the development branch is replaced with https://github.com/fredrik-johansson/arb/blob/0021661d618a7d392db8a7292230c593d849b59b/arb_hypgeom/legendre_p_ui_rec.c then I recover the interval

(0.08984375 + 0j)  +/-  (4.55e-18, 0j)

reported by the original poster for 2.12, and if I replace that file with https://github.com/fredrik-johansson/arb/blob/3959e07b5a4caf4673e9304f3fa1273c2f4d726e/arb_hypgeom/legendre_p_ui_rec.c then I get back the interval

(0.08984375 + 0j)  +/-  (7.29e-17, 0j)

reported for 2.13 and for the development branch, suggesting that the commit https://github.com/fredrik-johansson/arb/commit/3959e07b5a4caf4673e9304f3fa1273c2f4d726e was likely responsible. I tried to git checkout old commits but they didn't build because of the since-fixed "-r and -pie may not be used together" issue.

[fredrik-johansson]

The enclosures have tightened because some code improved significantly, but it's possible that the problem that caused the initial small regression is still present somewhere else. It would be nice to track it down, though I don't think it's a high priority. (Things like #153 are more important.)

The specific regression for Legendre polynomials is not much of an issue. We could perhaps increase the number of guard bits again if benchmarking shows that there is no slowdown for G-L nodes. Here we could also improve things further by making evaluation of Legendre polynomials exact at exact floating-point constants (again, ideally without slowdown). Basic idea: inspect the exponent and bit length of x and do some quick test to see if the numerator and denominator of P_n(x) would both be smaller than 2^(2*prec). In that case use an exact version of the basecase recurrence or hypergeometric series.