search

biocore / biom-format

The Biological Observation Matrix (BIOM) Format Project
http://biom-format.org
Other
89 stars 95 forks source link

Supporting NumPy 2.0 #950

Closed qiyunzhu closed 4 months ago

qiyunzhu commented 4 months ago

See here. To test whether this patch will work, we need to wait until h5py 3.11.0 is released. At present, it will say:

  File "biom/_filter.pyx", line 1, in init biom._filter
    # -----------------------------------------------------------------------------
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
wasade commented 4 months ago

Thanks! I looked at a few of the CI runs, and they looked to be using numpy 1.0. Is there a standard practice for updating CI to test both numpy 1 and 2? If not, can that be added? At the moment, I don't see an indication that we pass tests with 2.0, and this seems like an important distinction to be defensive about in CI

qiyunzhu commented 4 months ago

@wasade NumPy 2.0 hasn't been released yet, but one can install it by pip install numpy --pre or pip install numpy==2.0.0rc1. I have played around it and found some ways to set up a local test environment. It is feasible that we add this to CI. However, I think that an easier way is that we wait until NumPy 2.0 and relevant dependencies (like Pandas and h5py) in their compatible versions are released. This will make the CI much simpler.

qiyunzhu commented 4 months ago

Looks like h5py 3.11.0 was just released. This is the main unblocker. Now we need to wait for the release of Pandas 2.2.2.

wasade commented 4 months ago

Okay, thanks! So I think then this PR is "draft" NumPy 2.0 support but we cannot encode as such in CI. Could a changelog note be added along those lines, nothing likely 2.0 support but not yet part of our testing matrix?

qiyunzhu commented 4 months ago

@wasade Sounds good! Changelog updated.

wasade commented 4 months ago

Thanks!