UDST / pandana

Pandas Network Analysis by UrbanSim: fast accessibility metrics and shortest paths, using contraction hierarchies :world_map:
http://udst.github.io/pandana
GNU Affero General Public License v3.0
385 stars 84 forks source link

Binary incompatibility error with older versions of NumPy #164

Open smmaurer opened 3 years ago

smmaurer commented 3 years ago

We've been seeing a "binary incompatibility" error when running Pandana in combination with older versions of NumPy. This only affects copies of Pandana installed from Pip.

For example:

$ conda create -n test python=3.6 numpy=1.13 pandas requests scikit-learn pytables
$ conda activate test
$ pip install pandana
$ python examples/simple_example.py
...
Traceback (most recent call last):
  File "examples/simple_example.py", line 23, in <module>
    import pandana.network as pdna
  File "/Users/maurer/anaconda/envs/test/lib/python3.6/site-packages/pandana/__init__.py", line 1, in <module>
    from .network import Network
  File "/Users/maurer/anaconda/envs/test/lib/python3.6/site-packages/pandana/network.py", line 7, in <module>
    from .cyaccess import cyaccess
  File "src/cyaccess.pyx", line 1, in init pandana.cyaccess
    cimport cython
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

This can show up in VMs that have an old copy of NumPy pre-installed, or when using other libraries that require older versions of NumPy.

Diagnosis

What's happening is that the binary installers we recently created for Pip (PR #157) were compiled against current versions of NumPy (Pandana uses NumPy C datatypes), which don't have full backward compatibility. I actually tested for this at the time, but didn't go back far enough with NumPy versions. Whoops.

Solution

Users can resolve this by upgrading NumPy in the affected environment, if feasible. You could also install Pandana with Conda, or compile it from source code with the older version of NumPy installed.

Going forward, we'll specify build requirements using the new pyproject.toml standard, and make sure all binary installers (wheels) support the appropriate range of NumPy versions. https://numpy.org/devdocs/user/depending_on_numpy.html

We can also patch the Pip installers for older versions of Pandana (v0.4.4 and v0.5). The procedure would be to make patch branches in GitHub and update the build requirements, then recreate the wheels. Iterate the filenames before uploading to PyPI: pandana-0.4.4-... -> pandana-0.4.4-1-....

PyMap commented 3 years ago

@smmaurer here an example trying to install pandana on google colab environment. NumPy default version is 1.19.5.