Lack of wheel in distribution causes it to break builds targeting separate platforms

cmancone commented 1 year ago

Granted, some of this may be specific to my process, but the lack of a wheel in your distributions is rather unusual, so I figured I'd raise an issue for this.

I work heavily in Lambdas but my build machines are typically ubuntu (e.g.: public build servers for Github actions). As a result, when packaging my applications, I have to tell pip to target a separate build. This is a normal use case for pip, and I've never had trouble with this before using jwcrypto. To reproduce this issue, dump the following line into a file called requirements.txt:

jwcrypto==1.5.0 ; python_version >= "3.8" and python_version < "4.0"

And then ask pip to build a package for this requirements.txt file (this specifically targets the platform used by AWS Lambda):

pip3 install --platform manylinux2014_x86_64 -r requirements.txt -t ./package --implementation cp --only-binary=:all:

You'll be met with the following error message:

ERROR: Could not find a version that satisfies the requirement jwcrypto==1.5.0 (from versions: 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.6.0, 0.7, 0.8, 0.9, 0.9.1, 1.0)
ERROR: No matching distribution found for jwcrypto==1.5.0

Which was very confusing for me for quite a while, until I went and actually looked at the files in your distributions in pypi. You can see that 1.5.0 only has a .tar.gz file:

https://pypi.org/project/jwcrypto/1.5.0/#files

While 1.0 (the last version that pip recognizes) has a wheel file:

https://pypi.org/project/jwcrypto/1.0/#files

So, it seems like there was a point in time where you followed the standard build process for your package, but it looks like that changed after 1.0. Presumably you ditched the wheel because your package is pure python, so it's not technically necessary - that is, until it is :)

simo5 commented 1 year ago

I change how pypi publishing happens in this commit https://github.com/latchset/jwcrypto/commit/d83b316bd847335be5fdfe2e51f402527c9e7862 from a manual process to an automatic one on release.

I am not sure what broke and I am not a pypi expert, so if you know how to fix it to make it better for you I will access patches here.

Note though that jwcrypto is pure python and fully depends on python cryptography which releases wheels, so I think it may make more sense to figure out if you can pull a cryptography wheel and just lay jwcrypto on top.

cmancone commented 1 year ago

So the root issue here is the fact that you're using sdist rather than (something like) bdist. See this article:

https://dev.to/icncsx/python-packaging-sdist-vs-bdist-5ekb

The main question I would have for you is, "how would you feel about switching to a more comprehensive build/packaging system?". Namely, poetry.

simo5 commented 1 year ago

Quite frankly I do not care what system is used, so anything that works is theoretically fine.

However:

I do not want to have to deal with new dependencies if avoidable, and I certainly do not want to have to become an expert in a new build system to be able to use it.
I do not want things too young that require a lot of churn (see above) or that may break existing users.

Fundamentally a build/packaging system that doesn't get out of the way and requires constant care (which includes becoming rapidly obsoleting and requiring replacement) is not a system I care for.

achamayou commented 9 months ago

I don't think switching build systems is necessary for this, there is a suitable setuptools extension. As far as I can tell, the following steps work:

pip install wheel
python setup.py bdist_wheel

That creates a wheel under dist:

$ ls dist/
jwcrypto-1.5.0-py3-none-any.whl

schinnaswamy commented 6 months ago

Hi,

We are using AWS serverless platform and using Serverless-python-requirments plugin to create a python-requirments layer build from ManyLinux_2014_aarch64 platform and bundle all the python project dependencies (binary wheel distribution) as a layer and use the layer while running serverless AWS Lambda.

JWCrypto is one of dependent library and missing binary wheel distributions and the deployment is currently failing due to below PYPI error

Its same issue reported by @cmancone

Stderr: ERROR: Could not find a version that satisfies the requirement jwcrypto==1.5.4 (from versions: 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.6.0, 0.7, 0.8, 0.9, 0.9.1, 1.0)
ERROR: No matching distribution found for jwcrypto==1.5.4

Could you release the binary distribution as well as a part of your recent release 1.5.4