pyca / pynacl

Python binding to the Networking and Cryptography (NaCl) library
https://pynacl.readthedocs.io/
Apache License 2.0
1.05k stars 231 forks source link

Building wheels #746

Closed ghost closed 2 years ago

ghost commented 2 years ago

I'm using Python3.6 on Ubuntu and am unable to find a wheel that works for me - https://pypi.org/simple/pynacl/

We are now forced to build a wheel when installing:

09:55:28  Collecting pynacl>=1.0.1 (from paramiko>=1.10.1->paramiko-expect->qiutils==1.18.14->-r requirements.txt (line 6))
09:55:28    Using cached https://artifactory.delphix.com/artifactory/api/pypi/delphix-virtual-pypi/packages/packages/a7/22/27582568be639dfe22ddb3902225f91f2f17ceff88ce80e4db396c8986da/PyNaCl-1.5.0.tar.gz
09:55:29  Building wheels for collected packages: pynacl
09:55:29    Running setup.py bdist_wheel for pynacl: started
09:55:29    Running setup.py bdist_wheel for pynacl: finished with status 'error'
09:55:29    Complete output from command /var/tmp/jenkins_agents/regression.jenkins/workspace/generate-jobs/tmp.preseed_ws/.venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-jkezhjlb/pynacl/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp9bhx6mtwpip-wheel- --python-tag cp36:
09:55:29    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
09:55:29       or: -c --help [cmd1 cmd2 ...]
09:55:29       or: -c --help-commands
09:55:29       or: -c cmd --help
09:55:29    
09:55:29    error: invalid command 'bdist_wheel'

but that doesn't seem to be a build requirement. The pyproject.toml file mentions that the setup_requirements list in setup.py must match the pyproject.toml file but it does not - https://github.com/pyca/pynacl/blob/main/pyproject.toml#L2


What should the fix be?

Short term fixes for me might be to:

kitterma commented 2 years ago

Try installing the python3-setuptools package. I don't use Ubuntu, but the get their package splits from Debian and that's what I would recommend there.

Scott K

ghost commented 2 years ago

Hey Scott, thanks for the super quick reply. That sounds like it could be a workaround but I think it would be a workaround not a real fix.

I may not have been super clear but the way I see it there are two bugs here.

  1. A pre-built wheel is not being exported that works with Ubuntu + Python3.6. This forces someone who wants to use this package to install it from source.

  2. Because of 1 we now need to build the package from source. We are unable to install from source though because of https://github.com/pyca/pynacl/commit/44fb453931c611d4822343debe902cca8e1c8afe which contradicts the comment on https://github.com/pyca/pynacl/blob/main/pyproject.toml#L2

I could be wrong about my assessment though, its a Saturday and I've only briefly looked at this


I could work around this by installing the implicit wheel dependency but I don't think that that should be considered a permanent fix

reaperhulk commented 2 years ago

Looks like a few things here. First, Python3.6 on Ubuntu x86_64 is definitely covered by our existing wheels. However, to receive a wheel you'll need to make sure you're up-to-date with your pip version.

For compiling, wheel is a declared dependency for our build system, but it's not something that's a runtime dep. We will actually be entirely removing setup_requires shortly (as we've done in pyca/cryptography). The message there is out of date though, thank you for pointing that out. An up-to-date pip will also properly detect the wheel dependency (and setuptools minimum version) we declare within the pyproject.toml and successfully install.

ghost commented 2 years ago

Sure enough upgrading to a not-horrendously-old-version of pip allowed me to find the wheel.


@reaperhulk thanks for your help with this, I imagine this is something that you've seen a lot of while working on these widely used packages. I'll close this out and hopefully anyone else who runs into this will stop using pip 9.0.2 ;)

mendhak commented 2 years ago

Can confirm, future searchers - in our case pynacl was being built during a pip install a Docker container (and failing of course) instead of a wheel being downloaded. We are using the Airflow docker image + paramiko, and pynacl is pulled in as a requirement.

I couldn't get it to work with the latest pip but specifically 20.2.4:

pip install --upgrade pip==20.2.4

Because Airflow also has a limitation

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4 pip upgrade --pip==20.2.4 or, in case you use Pip 20.3, you need to add option --use-deprecated legacy-resolver to your pip install command.