python / cpython

The Python programming language
https://www.python.org
Other
62.38k stars 29.96k forks source link

`doctest` GitHub Actions job exits with `2` failing all recent PRs #101639

Closed arhadthedev closed 1 year ago

arhadthedev commented 1 year ago

The Docs / Doctest (pull_request) check started to fail recently with the following job log:

make: Entering directory '/home/runner/work/cpython/cpython/Doc'
make[1]: Entering directory '/home/runner/work/cpython/cpython/Doc'
mkdir -p build

Missing the required blurb or sphinx-build tools.
Please run 'make venv' to install local copies.

make[1]: *** [Makefile:48: build] Error 1
make[1]: Leaving directory '/home/runner/work/cpython/cpython/Doc'
Testing of doctests in the sources finished, look at the results in build/doctest/output.txt
make: *** [Makefile:1[2](https://github.com/python/cpython/actions/runs/4113297888/jobs/7099253477#step:9:2)[7](https://github.com/python/cpython/actions/runs/4113297888/jobs/7099253477#step:9:8): doctest] Error 1
make: Leaving directory '/home/runner/work/cpython/cpython/Doc'
Error: Process completed with exit code 2.

@colorfulappl found out that:

It seems like numpy build failed while running

make -C Doc/ PYTHON=../python venv

The error message is

Error:             numpy/random/_mt19937.c:8320:53: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
             8320 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
                  |                                                     ^~

It looks like gh-101292 is the cause. We need to provide some sort of compatibility shim or a deprecation period until numpy introduces version-specific #if guards.

Yhg1s commented 1 year ago

I think the fundamental problem here is that the doc build/test process uses the newly built Python and third-party dependencies from PyPI. I don't think it's bad to test that certain things work in the new Python, but expecting installable packages for the target Python is a bit problematic for two reasons: the Python we're building may be too new for the packages (as in this case), or the Python may be too old (for example when building docs for an older Python that the dependencies have already stopped supporting).

hugovk commented 1 year ago

One of the Sphinx extensions we're using just released (in 0.8.0) support for generating images via Matplotlib, which depends on NumPy: https://github.com/wpilibsuite/sphinxext-opengraph/pull/88

Even though we're not using the image generation there (yet, we may do later), it still attempts the install and import.

The quick fix is to pin sphinxext-opengraph to 0.5.9.

Alternatively, for the benefit of downstream redistributors, we did take care that if the extension isn't installed, it won't be used:

https://github.com/python/cpython/blob/79903240480429a6e545177416a7b782b0e5b9bd/Doc/conf.py#L27-L33

For doctest, we could also make use of this and only install the key essentials for the test.

arhadthedev commented 1 year ago

Fixed by gh-101642.