search

rrmeister / pyQuEST

Python interface for the Quantum Exact Simulation Toolkit (QuEST)
MIT License
17 stars 6 forks source link

Numpy 2.0 binary incompatibility #10

Open lochsh opened 4 months ago

lochsh commented 4 months ago

The dependency specified for numpy is >=1.20, but the pyquest wheels released on pypi.org are not compatible with numpy 2.0, which is an ABI-breaking release.

See numpy's advice for building compatible wheels: https://numpy.org/doc/stable/dev/depending_on_numpy.html#numpy-2-abi-handling.

It would be great to have pyquest wheels that are compatible with numpy ^2.0. Note that Numpy 2.0 drops support for Python 3.8. Python 3.8 is due to be end-of-lifed later this year, but if you wanted to keep supporting it then I think you could build your Python 3.8 wheels with 1.x numpy.

rrmeister commented 4 months ago

Hi Hannah,

thanks for drawing my attention to this. It's quite a tricky topic, because a lot of other packages (e.g. scipy) depend on specific numpy versions, so support should be carefully considered. There is a comment about this buried deep in the commit history (496fc98) saying

Moving from numpy 1.19 to 1.20 introduces binary incompatibility of numpy.ndarray. Isolated builds will use the latest version of numpy, i.e. 1.20. If 1.19 is installed, this will lead to pyQuEST (compiled with version 1.20) to be incompatible with the installed numpy version. By manually specifying all versions to be >=1.20, these problems can be avoided at the cost of breaking compatibility with envs requiring numpy 1.19. These cases can still manually specify 1.19 as the required version in setup.py and pyproject.toml.

I will be more explicit about this in the readme, but the unfortunate truth is that because we need to compile and link to a specific numpy version, there will never be wide support across many numpy versions for the releases on PyPI as long as numpy introduces breaking changes (which can be as little as modifying the size of the header of a data structure). I will consider moving to numpy 2.0 soon though.

Hope this helps.

Cheers! Richard

TerraVenil commented 4 months ago

Hi, @lochsh. A different issue related to the numpy version also was highlighted here https://github.com/rrmeister/pyQuEST/issues/9. So we just decided to use in our project the following workaround ["numpy >= 1.20, < 2.0.0"] at least to prevent installing the latest version of numpy.

jackturnerer commented 4 months ago

Hi, I had the same issue installing from pypi, where numpy>=1.20 is specified but it seems 2.0.0 (latest, installed on my system) is incompatible. My workaround was setting up a virtual env and installing an older numpy version on this (but newer than 1.20), and then running pip install pyquest. I have checked and this works right up to numpy 1.26.4 (newest before 2.0.0). Hope this is helpful!