Compatibility issue with NumPy 2.0.0

[pingretispec]

File "/test.py", line 4, in <module>
    import rawpy
  File "/usr/local/lib/python3.11/site-packages/rawpy/__init__.py", line 5, in <module>
    import rawpy._rawpy
  File "rawpy/_rawpy.pyx", line 1, in init rawpy._rawpy
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Numpy v2.0.0 rawpy v.0.21.0

[aniketpanjwani]

Try uninstalling 2.0.0 and installing the latest 1.x version - that fixed it for me:

pip uninstall numpy
pip install numpy==1.26.4

[FirefoxMetzger]

👋 I ran into this issue while working on a rawpy plugin for ImageIO this week, Unfortunately, pinning numpy<2.0.0 is not an option for us, as it would force our downstream libraries into the same pin.

I don't know the intricacies of rawpy's codebase but assuming that it doesn't use any obscure/nitch numpy features this should be solvable by a re-compile and re-release of rawpy for numpy>=2 without any code changes. Is there anything we can do to make this happen? (merging the ImageIO plugin will be blocked until this issue is resolved).

[letmaik]

@FirefoxMetzger Releasing a new version built against numpy 2 sounds reasonable. People stuck on numpy 1 will have to pin their rawpy version. Do you want to give this a try in a fork and submit a PR once it's working for you?

[FirefoxMetzger]

I've build rawpy locally against Numpy 2.0 on my Windows machine using python 3.11.

When I run pytest I get the following set of warnings. @letmaik Are these expected or did the switch to numpy v2.0 create them?

``` ================================================================================================= warnings summary ================================================================================================== rawpy\enhance.py:24 c:\users\sebastian\documents\coding-projects\rawpy\rawpy\enhance.py:24: UserWarning: OpenCV not found, install for faster processing (error: No module named 'cv2') warnings.warn('OpenCV not found, install for faster processing (error: ' + str(e) + ')') test/test_basic.py: 16 warnings C:\Users\Sebastian\Documents\Coding-Projects\rawpy\test\test_basic.py:284: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if StrictVersion(np.__version__) <= StrictVersion('1.10'): test/test_basic.py::testFileOpenAndPostProcess test/test_basic.py::testFileOpenAndPostProcess test/test_basic.py::testFileOpenAndPostProcess test/test_basic.py::testFileOpenAndPostProcess test/test_basic.py::testFoveonFileOpenAndPostProcess test/test_basic.py::testFoveonFileOpenAndPostProcess test/test_basic.py::testSRawFileOpenAndPostProcess test/test_basic.py::testSRawFileOpenAndPostProcess C:\Users\Sebastian\Documents\Coding-Projects\rawpy\test\test_basic.py:277: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if im.dtype == np.uint16 and StrictVersion(imageio.__version__) <= StrictVersion('0.5.1'): test/test_basic.py::testThumbExtractJPEG C:\Users\Sebastian\Documents\Coding-Projects\rawpy\test\test_basic.py:140: DeprecationWarning: Starting with ImageIO v3 the behavior of this function will switch to that of iio.v3.imread. To keep the current behavior (and make this warning disappear) use `import imageio.v2 as imageio` or call `imageio.v2.imread` directly. img = imageio.imread(thumb.data) test/test_basic.py::testBadPixelRepair test/test_basic.py::testFindBadPixelsNikonD4 C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (4096), equivalent to an approximate bitdepth of 12.0. image, footprint, out, mask, n_bins = _preprocess_input( test/test_basic.py::testBadPixelRepair C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (4091), equivalent to an approximate bitdepth of 12.0. image, footprint, out, mask, n_bins = _preprocess_input( test/test_basic.py::testBadPixelRepair C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (4093), equivalent to an approximate bitdepth of 12.0. image, footprint, out, mask, n_bins = _preprocess_input( test/test_basic.py::testBadPixelRepair C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (3936), equivalent to an approximate bitdepth of 11.9. image, footprint, out, mask, n_bins = _preprocess_input( test/test_basic.py::testFindBadPixelsNikonD4 C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (3900), equivalent to an approximate bitdepth of 11.9. image, footprint, out, mask, n_bins = _preprocess_input( test/test_basic.py::testFindBadPixelsNikonD4 C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (3892), equivalent to an approximate bitdepth of 11.9. image, footprint, out, mask, n_bins = _preprocess_input( test/test_basic.py::testFindBadPixelsNikonD4 C:\Users\Sebastian\Documents\Coding-Projects\rawpy\.venv\Lib\site-packages\skimage\filters\rank\generic.py:332: UserWarning: Bad rank filter performance is expected due to a large number of bins (3922), equivalent to an approximate bitdepth of 11.9. image, footprint, out, mask, n_bins = _preprocess_input( -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================================================================================= 22 passed, 34 warnings in 30.90s ========================================================================================== ```

[letmaik]

Fixed with the latest rawpy release.