search

geopandas / pyogrio

Vectorized vector I/O using OGR
https://pyogrio.readthedocs.io
MIT License
258 stars 22 forks source link

Compatibility with Numpy 2.0 #379

Closed martinfleis closed 2 months ago

martinfleis commented 3 months ago

I just noticed that pyogrio installed in geopandas' dev CI env, where we install nightly numpy 2 and pyogrio from source, does not work. We get this error

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.1.0.dev0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<string>", line 1, in <module>
  File "/home/runner/work/geopandas/geopandas/geopandas/tools/_show_versions.py", line 154, in show_versions
    proj_info = _get_C_info()
  File "/home/runner/work/geopandas/geopandas/geopandas/tools/_show_versions.py", line 61, in _get_C_info
    import pyogrio
  File "/home/runner/micromamba/envs/test/lib/python3.11/site-packages/pyogrio/__init__.py", line 9, in <module>
    from pyogrio.core import (
  File "/home/runner/micromamba/envs/test/lib/python3.11/site-packages/pyogrio/core.py", line 21, in <module>
    from pyogrio._io import ogr_list_layers, ogr_read_bounds, ogr_read_info

See the log within the Check and Log Environment section here. I am not sure why is that as it is installed from source against numpy 2 but it seems to be something worth our attention.

martinfleis commented 3 months ago

Ignore me, it seems that we first build pyogrio against 1.26 and then download 2.0 nightly in CI...

jorisvandenbossche commented 3 months ago

It's still caused by pyogrio itself that it gets build with an older version of numpy, because pyogrio pins to oldest-supported-numpy. I did work on shapely to ensure build compatibility with the upcoming numpy 2.0 (https://github.com/shapely/shapely/issues/1972), but forgot we need similar work for pyogrio. (and ideally we would already had a release of pyogrio upper pinning numpy)

jorisvandenbossche commented 3 months ago

Now, it seems that while we build against numpy at the cython level, it seems we don't actually use anything from numpy's C API, as far as I can see. So it might be easier to just remove numpy as a build dependency alltogether -> https://github.com/geopandas/pyogrio/pull/381