benfogle / crossenv

Cross-compiling virtualenv for Python
MIT License
108 stars 22 forks source link

pip failed to install numpy when version>1.17.5 #99

Closed AeolianRenne closed 2 years ago

AeolianRenne commented 2 years ago

I used to be using numpy==1.17.5 on Python 3.8.10 and it worked well . Then I was trying to use numpy==1.19.2. However, it failed with the error that " AttributeError: 'zipimporter' object has no attribute 'create_module'". I found the function in site.py.

class CrossenvPatchLoader(importlib.abc.Loader): def init(self, original, patch): self.original = original self.patch = patch

def create_module(self, spec):
    return self.original.create_module(spec)

def exec_module(self, module):
    self.original.exec_module(module)
    _patch_module(module, self.patch)

# runpy module expects a source loader, should we try to run 'python -m
# sysconfig'. This has extra methods, so we'll keep it happy with whatever
# it wants.
def __getattr__(self, name):
    return getattr(self.original, name)

Is it because that the CrossenvPatchLoader has no support for newer version of numpy yet? I tried other versions like 1.18.0, which also failed. It seems that only version<=1.17.5 can be successfully installed.


ERROR: Command errored out with exit status 1: command: /home/cpython/build/python3/Python-3.8.10/venv/cross/bin/python3 /tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-zgs4d9hg/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://mirrors.sjtug.sjtu.edu.cn/pypi/web/simple -- 'setuptools<49.2.0' wheel 'Cython>=0.29.21' cwd: None Complete output (55 lines): Traceback (most recent call last): File "/home/cpython//build/python3/Python-3.8.10/build-pfb/install/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/cpython//build/python3/Python-3.8.10/build-pfb/install/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip/main.py", line 29, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip/_internal/cli/main.py", line 9, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip/_internal/cli/autocompletion.py", line 10, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip__.zip/pip/_internal/cli/main_parser.py", line 8, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip/_internal/cli/cmdoptions.py", line 23, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip/_internal/cli/parser.py", line 12, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip.zip/pip/_internal/configuration.py", line 21, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "", line 259, in load_module File "/tmp/pip-standalone-pip-zwjr964a/env_pip__.zip/pip/_internal/exceptions.py", line 7, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 657, in _load_unlocked File "", line 556, in module_from_spec File "/home/cpython/build/python3/Python-3.8.10/venv/lib/site.py", line 64, in create_module return self.original.create_module(spec) AttributeError: 'zipimporter' object has no attribute 'create_module'

benfogle commented 2 years ago

I can't seem to reproduce this on my end, and I have a few questions:

  1. The trace you've provided mentions "env_pip.zip". Are you loading any modules from zip files?
  2. Are build-python and host-python the same version? From within the cross environment, what is the output of build-python --version
  3. What command was used to create the cross environment?

It looks to me like something tried to load a module from a zip file, but it got confused about what methods were available. zipimporter did indeed get a create_module function, but only in Python 3.10. Otherwise it should have been defaulting to the legacy loader. Offhand, I can't think of anything other than a version mismatch between Python versions that could cause that.

AeolianRenne commented 2 years ago

Thank you for your response!

  1. The pip install process does download a zip file. It is the source code of numpy 1.19.2.
    Collecting numpy==1.19.2 Downloading https://mirror.sjtu.edu.cn/pypi-packages/bf/e8/15aea783ea72e2d4e51e3ec365e8dc4a1a32c9e5eb3a6d695b0d58e67cdd/numpy-1.19.2.zip (7.3 MB) Installing build dependencies ... error
  2. The build-python and host python are both 3.8.10.
  3. I was doing this on another project about operating system and it seems that they have made changes to both python and cross-env in order to build and use them on their platform.

I guess it is more likely the problem of my environment or settings. I will try other methods to install numpy. Thanks a lot!