Closed oliverob closed 5 months ago
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:
See also https://github.com/python/cpython/issues/117173 for a similar report.
@oliverob what is the command line and or minimal reproducer to see this issue in action? Looks like you're in the middle of some Docker build?
I also started getting this problem yesterday with my repo2docker-action workflow, which is essentially the workflow on your README.md under where it says 'Cache builds on mybinder.org'. (This is my workflow that gives what looks like the same error https://github.com/acpennlab/statistics-resampling-online/blob/master/.github/workflows/binder-master.yml).
...
#6 1.539 (10/22) Installing gcc (12.2.1_git20220924-r4)
#6 2.237 (11/22) Installing libstdc++-dev (12.2.1_git20220924-r4)
#6 2.488 (12/22) Installing musl-dev (1.2.3-r5)
#6 2.550 (13/22) Installing libc-dev (0.7.2-r3)
#6 2.553 (14/22) Installing g++ (12.2.1_git20220924-r4)
#6 2.751 (15/22) Installing make (4.3-r1)
#6 2.756 (16/22) Installing fortify-headers (1.1-r1)
#6 2.762 (17/22) Installing patch (2.7.6-r9)
#6 2.766 (18/22) Installing build-base (0.5-r3)
#6 2.769 (19/22) Installing curl (8.5.0-r0)
#6 2.774 (20/22) Upgrading python3 (3.10.13-r0 -> 3.10.14-r0)
#6 3.277 (21/22) Installing pkgconf (1.9.4-r0)
#6 3.283 (22/22) Installing python3-dev (3.10.14-r0)
#6 3.538 Executing busybox-1.35.0-r29.trigger
#6 3.568 OK: [63](https://github.com/acpennlab/statistics-resampling-online/actions/runs/8402090076/job/23011109592#step:2:63)0 MiB in [76](https://github.com/acpennlab/statistics-resampling-online/actions/runs/8402090076/job/23011109592#step:2:76) packages
#6 DONE 4.2s
#7 [3/7] RUN python3 -m pip install --upgrade wheel setuptools
#7 0.635 Traceback (most recent call last):
#7 0.635 File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
#7 0.635 return _run_code(code, main_globals, None,
#7 0.635 File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
#7 0.636 exec(code, run_globals)
#7 0.636 File "/usr/lib/python3.10/site-packages/pip/__main__.py", line 31, in <module>
#7 0.636 sys.exit(_main())
#7 0.636 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 68, in main
#7 0.636 command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
#7 0.636 File "/usr/lib/python3.10/site-packages/pip/_internal/commands/__init__.py", line 114, in create_command
#7 0.636 module = importlib.import_module(module_path)
#7 0.636 File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
#7 0.636 return _bootstrap._gcd_import(name[level:], package, level)
#7 0.636 File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#7 0.636 File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#7 0.637 File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
#7 0.637 File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
#7 0.637 File "<frozen importlib._bootstrap_external>", line 8[83](https://github.com/acpennlab/statistics-resampling-online/actions/runs/8402090076/job/23011109592#step:2:83), in exec_module
#7 0.637 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#7 0.637 File "/usr/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 16, in <module>
#7 0.637 from pip._internal.cli.req_command import (
#7 0.637 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 21, in <module>
#7 0.454 command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
#7 0.454 File "/usr/lib/python3.10/site-packages/pip/_internal/commands/__init__.py", line 114, in create_command
#7 0.454 module = importlib.import_module(module_path)
#7 0.454 File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
#7 0.455 return _bootstrap._gcd_import(name[level:], package, level)
#7 0.455 File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#7 0.455 File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#7 0.455 File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
#7 0.455 File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
#7 0.455 File "<frozen importlib._bootstrap_external>", line 883, in exec_module
#7 0.455 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#7 0.455 File "/usr/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 16, in <module>
#7 0.455 from pip._internal.cli.req_command import (
#7 0.455 File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 21, in <module>
#7 0.455 from pip._internal.index.package_finder import PackageFinder
#7 0.455 File "/usr/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 33, in <module>
#7 0.456 from pip._internal.req import InstallRequirement
#7 0.456 File "/usr/lib/python3.10/site-packages/pip/_internal/req/__init__.py", line 8, in <module>
#7 0.456 from .req_install import InstallRequirement
#7 0.456 File "/usr/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 44, in <module>
#7 0.456 from pip._internal.operations.install.wheel import install_wheel
#7 0.456 File "/usr/lib/python3.10/site-packages/pip/_internal/operations/install/wheel.py", line 39, in <module>
#7 0.456 from pip._vendor.distlib.scripts import ScriptMaker
#7 0.456 File "/usr/lib/python3.10/site-packages/pip/_vendor/distlib/scripts.py", line 16, in <module>
#7 0.456 from .compat import sysconfig, detect_encoding, ZipFile
#7 0.456 File "/usr/lib/python3.10/site-packages/pip/_vendor/distlib/compat.py", line 83, in <module>
#7 0.456 import xmlrpc.client as xmlrpclib
#7 0.456 File "/usr/lib/python3.10/xmlrpc/client.py", line 138, in <module>
#7 0.457 from xml.parsers import expat
#7 0.457 File "/usr/lib/python3.10/xml/parsers/expat.py", line 4, in <module>
#7 0.457 from pyexpat import *
#7 0.457 ImportError: Error relocating /usr/lib/python3.10/lib-dynload/pyexpat.cpython-310-x[86](https://github.com/acpennlab/statistics-resampling-online/actions/runs/8402090076/job/23011109592#step:2:86)_64-linux-gnu.so: XML_SetReparseDeferralEnabled: symbol not found
#7 ERROR: process "/bin/sh -c python3 -m pip install --upgrade wheel setuptools" did not complete successfully: exit code: 1
------
> [3/7] RUN python3 -m pip install --upgrade wheel setuptools:
0.456 from pip._vendor.distlib.scripts import ScriptMaker
0.456 File "/usr/lib/python3.10/site-packages/pip/_vendor/distlib/scripts.py", line 16, in <module>
0.456 from .compat import sysconfig, detect_encoding, ZipFile
0.456 File "/usr/lib/python3.10/site-packages/pip/_vendor/distlib/compat.py", line 83, in <module>
0.456 import xmlrpc.client as xmlrpclib
0.456 File "/usr/lib/python3.10/xmlrpc/client.py", line 138, in <module>
0.457 from xml.parsers import expat
0.457 File "/usr/lib/python3.10/xml/parsers/expat.py", line 4, in <module>
0.457 from pyexpat import *
0.457 ImportError: Error relocating /usr/lib/python3.10/lib-dynload/pyexpat.cpython-310-x86_64-linux-gnu.so: XML_SetReparseDeferralEnabled: symbol not found
------
Dockerfile:5
--------------------
3 | RUN apk add --no-cache curl build-base python3 python3-dev py3-pip
4 |
5 | >>> RUN python3 -m pip install --upgrade wheel setuptools
6 |
7 | # https://stackoverflow.com/a/41651363/16[95](https://github.com/acpennlab/statistics-resampling-online/actions/runs/8402090076/job/23011109592#step:2:95)486
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 -m pip install --upgrade wheel setuptools" did not complete successfully: exit code: 1
Error: Docker build failed with exit code 1
Looking at the discussion at https://github.com/python/cpython/issues/117173 is this related to the upgrade from python 3.10.14? Is it possible to specify the python version as 3.10.13 until this is fixed? I've looked back at the last time the workflow passed and I see that python 3 wasn't updated there:
...
#6 3.042 (17/21) Installing patch (2.7.6-r9)
#6 3.048 (18/21) Installing build-base (0.5-r3)
#6 3.050 (19/21) Installing curl (8.5.0-r0)
#6 3.056 (20/21) Installing pkgconf (1.9.4-r0)
#6 3.060 (21/21) Installing python3-dev (3.10.13-r0)
#6 3.290 Executing busybox-1.35.0-r29.trigger
#6 3.325 OK: 630 MiB in 76 packages
I was able to reproduce the issue now. It's a bug in the dependencies of Alpine package python3 version 3.10.14-r0. It needs libexpat >=2.6.0 at runtime but forgets to pull a recent enough version in. When libexpat 2.5.0 is installed, this will produce the crash that you observed above. For a reproducer Dockerfile
:
FROM quay.io/jupyterhub/repo2docker:main@sha256:7ffa7a5d29942403d10524f933e7742d6d5f0835a82489e1978ffcf85df306e5
RUN apk add python3-dev
RUN test "$(python3 --version)" = "Python 3.10.14"
RUN apk update && apk policy libexpat | grep -B1 lib/apk/db/installed | grep -F 2.5.0
RUN ls -l /usr/lib/libexpat.so.*
RUN grep -F XML_SetReparseDeferralEnabled /usr/lib/libexpat.so.*
The last command will fail.
CC @ncopa
Also, the same issue using the repo2docker-action workflow to build our docker image: https://github.com/CryoInTheCloud/hub-image/actions/runs/8395354983/job/23011294112?pr=109. Thank you so much for your help!
This will be resolved by https://github.com/jupyterhub/repo2docker/pull/1332 that updates to use alpine:3.19, and this issue isn't showing up in either alpine:3.18 or 3.19, so I suggest we let it be resolved like that.
Is this resolved now? If someone is able to, check that would be great - the local tests are broken currently (#117)
I restarted this failing build (https://github.com/CryoInTheCloud/hub-image/pull/109) and it seems to be fixed now. Thanks for the investigation and merge, @consideRatio.
For others, if restarting your build doesn't fix the issue, please re-open.
When I try to run this action I get the following error:
This was not happening yesterday so not sure what has changed between then and now