search

cgohlke / imagecodecs

Image transformation, compression, and decompression codecs
https://pypi.org/project/imagecodecs
BSD 3-Clause "New" or "Revised" License
117 stars 23 forks source link

NumPy 2.0 support #100

Closed jakirkham closed 4 months ago

jakirkham commented 5 months ago

NumPy 2.0 is coming out soon ( https://github.com/numpy/numpy/issues/24300 ). NumPy 2.0.0rc1 packages for conda & wheels came out 3 weeks back ( https://github.com/numpy/numpy/issues/24300#issuecomment-2030603395 )

To prepare for NumPy 2.0, it might be worthwhile to start testing against NumPy 2 in CI

NumPy has put out a migration guide. More details are in the release notes

If imagecodecs make use of NumPy's C API (and produces wheels that use it), having a release of imagecodecs with wheels built against NumPy 2.0.0rc1 would be helpful to ensure NumPy 1 & 2 compatible wheels (as wheels built against NumPy 1 won't be compatible with NumPy 2). More details in this NumPy 2 ABI doc

Also as NumPy is tracking ecosystem support for NumPy 2.0, it would be helpful to share imagecodecs current support status in issue (with any plans): https://github.com/numpy/numpy/issues/26191

jakirkham commented 5 months ago

If imagecodecs make use of NumPy's C API...

It looks like this may be the case based on lines like this

https://github.com/cgohlke/imagecodecs/blob/2ddadafd413c8b035b8f6339e8ea603bb416f011/imagecodecs/_shared.pyx#L44

cgohlke commented 5 months ago

I am aware of the numpy 2 transition. The development version of imagecodecs already passes all tests possible. For CI to succeed, some test dependencies such as numcodecs and blosc 1 need numpy 2 compatible releases. The next imagecodecs release is also waiting for updates of some dependencies: liblzma, Little-CMS...

jakirkham commented 5 months ago

Thanks Christoph! 🙏

Ok that's good to know

Took a look at Numcodecs in issue ( https://github.com/zarr-developers/numcodecs/issues/521 ). It only uses NumPy's Python API (and a very limited set at that), which was unaffected by the new version. Also ran tests locally for good measure. So think it should already work (though please let us know if you see issues)

With Blosc 1, it appears to only use NumPy for testing. Searching the Blosc 1 code more generally, it has some benchmarks and docs that use NumPy, but nothing else AFAICT. Is there somewhere else where NumPy comes up in Blosc 1?

Regarding the other dependencies, do these use NumPy as well (IOW are these Python wrappers of libraries) or are these just the C/C++ libraries?

cgohlke commented 5 months ago

You are right. Blosc 1 does not link to numpy (it was blosc2, which now has a numpy compatible release), neither does numcodecs (it was zfpy that failed, which is optional). Looks like imagecodecs should be good to build and test on CI then. I'll try on the weekend.

jakirkham commented 5 months ago

Ah ok. Good to know about Blosc 2

Also good catch with Numcodecs' optional zfpy. Appears to be tracked in upstream issue: https://github.com/LLNL/zfp/issues/210

cgohlke commented 4 months ago

Fixed in v2024.6.1.