search

bioio-devs / bioio

Image reading, metadata management, and image writing for Microscopy images in Python
https://bioio-devs.github.io/bioio/OVERVIEW.html
BSD 3-Clause "New" or "Revised" License
53 stars 4 forks source link

Remove numpy<2.0.0 restriction #70

Open pgarrison opened 3 months ago

pgarrison commented 3 months ago

Ideally bioio should support the latest numpy.

I checked with the ruff rule for numpy 2.0 migration, which is not (cannot possibly be) 100% comprehensive and found no issues in bioio, bioio-base, and bioio-ome-zarr. Does anyone (@SeanLeRoy) know of anything that would be broken by numpy 2?

If necessary, individual bioio plugins could retain the numpy<2.0.0 restriction until they are updated for compatibility.

BrianWhitneyAI commented 1 month ago

Currently fails when unpinning numpy (upgraded to 2.1.1) (Python 3.10) 

2024-09-25 09:41:09.904 [error] pytest test discovery error for workspace:  /Users/brian.whitney/Desktop/Repos/bioio 
 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

Interrupted: 4 errors during collection
 Check Python Test Logs for more details.
2024-09-25 09:41:09.906 [error] pytest test discovery error for workspace:  /Users/brian.whitney/Desktop/Repos/bioio 
 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

Interrupted: 4 errors during collection
 Check Python Test Logs for more details.
2024-09-25 09:41:09.906 [info] 
==================================== ERRORS ====================================
_________ ERROR collecting bioio/tests/writers/test_ome_tiff_writer.py _________
bioio/tests/writers/test_ome_tiff_writer.py:14: in <module>
    from bioio.writers import OmeTiffWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
_________ ERROR collecting bioio/tests/writers/test_ome_zarr_writer.py _________
bioio/tests/writers/test_ome_zarr_writer.py:13: in <module>
    from bioio.writers import OmeZarrWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
________ ERROR collecting bioio/tests/writers/test_timeseries_writer.py ________
bioio/tests/writers/test_timeseries_writer.py:12: in <module>
    from bioio.writers.timeseries_writer import TimeseriesWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
__________ ERROR collecting bioio/tests/writers/test_two_d_writer.py ___________
bioio/tests/writers/test_two_d_writer.py:12: in <module>
    from bioio.writers.two_d_writer import TwoDWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
=========================== short test summary info ============================
ERROR bioio/tests/writers/test_ome_tiff_writer.py - ValueError: numpy.dtype s...
ERROR bioio/tests/writers/test_ome_zarr_writer.py - ValueError: numpy.dtype s...
ERROR bioio/tests/writers/test_timeseries_writer.py - ValueError: numpy.dtype...
ERROR bioio/tests/writers/test_two_d_writer.py - ValueError: numpy.dtype size...
!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!
===================== 2 tests collected, 4 errors in 0.16s =====================
pgarrison commented 1 month ago

@BrianWhitneyAI That error message reminds me of errors that can come up when using pandas<2 with numpy>=2