GrahamDumpleton / wrapt

A Python module for decorators, wrappers and monkey patching.
BSD 2-Clause "Simplified" License
2.04k stars 230 forks source link

Wrapt 1.13.x fails to install properly on Arm #198

Closed timbrown5 closed 2 years ago

timbrown5 commented 2 years ago

We recently started getting errors when installing one of our internal tools. The errors only happened on ARM and a subset of arm systems - 18.04 fails [Python 3.6], but 20.04 works [Python 3.8]).

2021-11-16 09:35:01+0000  Collecting wrapt<2,>=1.10 (from deprecated->redis>=3.5.3->graphcore-view==0.4.2)
2021-11-16 09:35:01+0000    Downloading https://files.pythonhosted.org/packages/eb/f6/d81ccf43ac2a3c80ddb6647653ac8b53ce2d65796029369923be06b815b8/wrapt-1.13.3.tar.gz (48kB)
2021-11-16 09:35:02+0000      Complete output from command python setup.py egg_info:
2021-11-16 09:35:02+0000      Traceback (most recent call last):
2021-11-16 09:35:02+0000        File "<string>", line 1, in <module>
2021-11-16 09:35:02+0000        File "/tmp/pip-build-8qyvnw37/wrapt/setup.py", line 42, in <module>
2021-11-16 09:35:02+0000          ext_modules=[] if disable_extensions else extensions
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 129, in setup
2021-11-16 09:35:02+0000          return distutils.core.setup(**attrs)
2021-11-16 09:35:02+0000        File "/usr/lib/python3.6/distutils/core.py", line 121, in setup
2021-11-16 09:35:02+0000          dist.parse_config_files()
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 494, in parse_config_files
2021-11-16 09:35:02+0000          ignore_option_errors=ignore_option_errors)
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/config.py", line 106, in parse_configuration
2021-11-16 09:35:02+0000          meta.parse()
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/config.py", line 382, in parse
2021-11-16 09:35:02+0000          section_parser_method(section_options)
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/config.py", line 355, in parse_section
2021-11-16 09:35:02+0000          self[name] = value
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/config.py", line 173, in __setitem__
2021-11-16 09:35:02+0000          value = parser(value)
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/config.py", line 430, in _parse_version
2021-11-16 09:35:02+0000          version = self._parse_attr(value)
2021-11-16 09:35:02+0000        File "/usr/lib/python3/dist-packages/setuptools/config.py", line 305, in _parse_attr
2021-11-16 09:35:02+0000          module = import_module(module_name)
2021-11-16 09:35:02+0000        File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
2021-11-16 09:35:02+0000          return _bootstrap._gcd_import(name[level:], package, level)
2021-11-16 09:35:02+0000        File "<frozen importlib._bootstrap>", line 994, in _gcd_import
2021-11-16 09:35:02+0000        File "<frozen importlib._bootstrap>", line 971, in _find_and_load
2021-11-16 09:35:02+0000        File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
2021-11-16 09:35:02+0000      ModuleNotFoundError: No module named 'wrapt'
2021-11-16 09:35:02+0000      
GrahamDumpleton commented 2 years ago

Ensure you have latest pip and setuptools versions installed. I understand this as possibly being related to those packaging tools. Note that you also need to have a compiler available so C extensions can be compiled. You may get more debug information from pip by using the -v -v -v option. And yes I mean to list -v multiple times to get extra level of debug.

avegao commented 2 years ago

Same fail in Apple M1 with macOS 12.0.1

GrahamDumpleton commented 2 years ago

To add to the other suggestions, which had no feedback on to say whether it helped or not, try explicitly running:

pip uninstall wrapt

multiple times until it says not present and can't be uninstalled.

Then run:

pip install -v -v -v --no-cache-dir --no-binary ":all:" wrapt

Supply the output from running this command if subsequently still not available.

The --no-binary ":all:" is to try and make sure it isn't being confused by wheels for other platform variants. Thus it should use the wrapt-X.Y.Z.tar.gz file.

Eg.,

  Downloading wrapt-1.13.3.tar.gz (48 kB)
     |████████████████████████████████| 48 kB 6.7 MB/s

It should be automatically using the wrapt-X.Y.Z.tar.gz file as none of the wheel files should match these platforms.

Really need more information to try and determine if this is an issue with packaging or not.

eldritchideen commented 2 years ago

The key to building Wrapt on Arm is to ensure that pip and setuptools are updated to the latest version. Without this it fails with a gcc command line argument error.

After running pip install --upgrade pip setuptools and running pip install -v -v -v --no-cache-dir --no-binary ":all:" wrapt the output is:

(venv) [ec2-user testing]$ pip install -v -v -v --no-cache-dir --no-binary ":all:" wrapt
Using pip 21.3.1 from /home/ec2-user/testing/venv/lib/python3.9/site-packages/pip (python 3.9)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-wme6foz0
Created temporary directory: /tmp/pip-req-tracker-2_c810wq
Initialized build tracking at /tmp/pip-req-tracker-2_c810wq
Created build tracker: /tmp/pip-req-tracker-2_c810wq
Entered build tracker: /tmp/pip-req-tracker-2_c810wq
Created temporary directory: /tmp/pip-install-axyhrvzv
1 location(s) to search for versions of wrapt:
* https://pypi.org/simple/wrapt/
Fetching project page and analyzing links: https://pypi.org/simple/wrapt/
Getting page https://pypi.org/simple/wrapt/
Found index url https://pypi.org/simple
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/wrapt/ HTTP/1.1" 200 45762
[Lines trimmed]
Skipping link: not a file: https://pypi.org/simple/wrapt/
Given no hashes to check 39 links for project 'wrapt': discarding no candidates
Collecting wrapt
  Created temporary directory: /tmp/pip-unpack-jhjcreaw
  Starting new HTTPS connection (1): files.pythonhosted.org:443
  https://files.pythonhosted.org:443 "GET /packages/eb/f6/d81ccf43ac2a3c80ddb6647653ac8b53ce2d65796029369923be06b815b8/wrapt-1.13.3.tar.gz HTTP/1.1" 200 48871
  Downloading wrapt-1.13.3.tar.gz (48 kB)
     |████████████████████████████████| 48 kB 37.2 MB/s            
  Added wrapt from https://files.pythonhosted.org/packages/eb/f6/d81ccf43ac2a3c80ddb6647653ac8b53ce2d65796029369923be06b815b8/wrapt-1.13.3.tar.gz#sha256=1fea9cd438686e6682271d36f3481a9f3636195578bab9ca3382e2f5f01fc185 to build tracker '/tmp/pip-req-tracker-2_c810wq'
  Running setup.py (path:/tmp/pip-install-axyhrvzv/wrapt_e62beb8699784a8c8a57a2cbe80e368d/setup.py) egg_info for package wrapt
  Created temporary directory: /tmp/pip-pip-egg-info-llqvtn10
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-llqvtn10/wrapt.egg-info
  writing manifest file '/tmp/pip-pip-egg-info-llqvtn10/wrapt.egg-info/SOURCES.txt'
  writing manifest file '/tmp/pip-pip-egg-info-llqvtn10/wrapt.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
  Source in /tmp/pip-install-axyhrvzv/wrapt_e62beb8699784a8c8a57a2cbe80e368d has version 1.13.3, which satisfies requirement wrapt from https://files.pythonhosted.org/packages/eb/f6/d81ccf43ac2a3c80ddb6647653ac8b53ce2d65796029369923be06b815b8/wrapt-1.13.3.tar.gz#sha256=1fea9cd438686e6682271d36f3481a9f3636195578bab9ca3382e2f5f01fc185
  Removed wrapt from https://files.pythonhosted.org/packages/eb/f6/d81ccf43ac2a3c80ddb6647653ac8b53ce2d65796029369923be06b815b8/wrapt-1.13.3.tar.gz#sha256=1fea9cd438686e6682271d36f3481a9f3636195578bab9ca3382e2f5f01fc185 from build tracker '/tmp/pip-req-tracker-2_c810wq'
Created temporary directory: /tmp/pip-unpack-t_zksh_p
Skipping wheel build for wrapt, due to binaries being disabled for it.
Installing collected packages: wrapt
  Created temporary directory: /tmp/pip-record-2rks7m45
    Running command /home/ec2-user/testing/venv/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-axyhrvzv/wrapt_e62beb8699784a8c8a57a2cbe80e368d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-axyhrvzv/wrapt_e62beb8699784a8c8a57a2cbe80e368d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2rks7m45/install-record.txt --single-version-externally-managed --compile --install-headers /home/ec2-user/testing/venv/include/site/python3.9/wrapt
    running install
    /home/ec2-user/testing/venv/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build/lib.linux-aarch64-3.9
    creating build/lib.linux-aarch64-3.9/wrapt
    copying src/wrapt/__init__.py -> build/lib.linux-aarch64-3.9/wrapt
    copying src/wrapt/decorators.py -> build/lib.linux-aarch64-3.9/wrapt
    copying src/wrapt/importer.py -> build/lib.linux-aarch64-3.9/wrapt
    copying src/wrapt/wrappers.py -> build/lib.linux-aarch64-3.9/wrapt
    running build_ext
    creating build/temp.linux-aarch64-3.9
    creating build/temp.linux-aarch64-3.9/src
    creating build/temp.linux-aarch64-3.9/src/wrapt
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/ec2-user/testing/venv/include -I/usr/local/include/python3.9 -c src/wrapt/_wrappers.c -o build/temp.linux-aarch64-3.9/src/wrapt/_wrappers.o
    gcc -pthread -shared build/temp.linux-aarch64-3.9/src/wrapt/_wrappers.o -o build/lib.linux-aarch64-3.9/wrapt/_wrappers.cpython-39-aarch64-linux-gnu.so
    running install_lib
    creating /home/ec2-user/testing/venv/lib/python3.9/site-packages/wrapt
    byte-compiling /home/ec2-user/testing/venv/lib/python3.9/site-packages/wrapt/__init__.py to __init__.cpython-39.pyc
    byte-compiling /home/ec2-user/testing/venv/lib/python3.9/site-packages/wrapt/decorators.py to decorators.cpython-39.pyc
    byte-compiling /home/ec2-user/testing/venv/lib/python3.9/site-packages/wrapt/importer.py to importer.cpython-39.pyc
    byte-compiling /home/ec2-user/testing/venv/lib/python3.9/site-packages/wrapt/wrappers.py to wrappers.cpython-39.pyc
    running install_egg_info
    running egg_info
    writing manifest file 'src/wrapt.egg-info/SOURCES.txt'
    Copying src/wrapt.egg-info to /home/ec2-user/testing/venv/lib/python3.9/site-packages/wrapt-1.13.3-py3.9.egg-info
    running install_scripts
    writing list of installed files to '/tmp/pip-record-2rks7m45/install-record.txt'
    Running setup.py install for wrapt ... done
Successfully installed wrapt-1.13.3

Without upgrading setuptools to the latest this fails to build the C extension.

GrahamDumpleton commented 2 years ago

Version 1.14.0 now has ARM on Linux binary wheels, so should hopefully now install for some Python versions at least if don't have a C compiler installed.