Error with older numpy

[danielk333]

Using numpy==1.18.5 resulted in the following error

RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):

[...]

    import digital_rf as drf
  File "/home/danielk/venvs/sorts/lib/python3.7/site-packages/digital_rf/__init__.py", line 4, in <module>
    from .digital_rf_hdf5 import *
  File "/home/danielk/venvs/sorts/lib/python3.7/site-packages/digital_rf/digital_rf_hdf5.py", line 36, in <module>
    from . import _py_rf_write_hdf5, digital_metadata, list_drf
ImportError: numpy.core.multiarray failed to import

and upgrading to numpy==1.20.3 fixed the problem.

[ryanvolz]

Yeah, you generally can't use a numpy that's older than what it was compiled with. Did you install with pip? What platform? Hopefully I can sort this out by fixing the PyPI package.

[danielk333]

This was on a Ubuntu 16 LTS (I desperately need to update that machine when I get time, but that should be independent of this issue).

I installed with pip install digital_rf, but numpy already existed in my virtualenv due to another package being installed prior that listed an old numpy version as a dependency. Hence, I assume that the setup.py ( https://github.com/MITHaystack/digital_rf/blob/master/python/setup.py#L212 ) saw the existing version and did not update it or state a conflict.

So yeah, I think that a simple update of the that line and reupload to PyPI would fix it?

[ryanvolz]

It's not the setup.py per-se, because for people installing from source it doesn't matter what version of numpy is used. I use a particular setup (https://github.com/ryanvolz/digital_rf-wheels) to compile the packages for upload to PyPI, and I usually try to use the oldest numpy available to avoid this problem. I'll look to see why that didn't happen in this case and fix it.

[ryanvolz]

It should have been built against numpy=1.14, so I'm still not sure what happened and why it's broken. These wheels for PyPI are a pain to set up and I haven't done anything more than plug in new versions over the past couple years, so perhaps it is time to re-assess the tooling and see if a better method exists now.

[danielk333]

Oh I see! It was quite some time since I setup a automatic packing pipeline so Ill just say good luck!

Only thing that might be of use is that I once had the problem of the package not having the same requirement in the core meta data as was used in the tests, which created a whole lot of unnecessary debugging.

[ryanvolz]

It seems like pip has gone and done something different from earlier versions that silently uses the newest numpy version for building: https://github.com/pypa/pip/issues/9542. Apparently anyone building wheels with pip now by default has no control over what versions of the dependencies are used; only what's specified in pyproject.toml matters, unless you turn off build isolation. So the proper thing to do from their perspective is for me to use oldest-supported-numpy in pyproject.toml, which I can absolutely do but will require a new release. I guess the workaround would be to turn off build isolation for the wheel builds. I'll probably try to put together a new release soon.

[ryanvolz]

I have uploaded new wheels for digital_rf==2.6.6 (with build number 1) that should work with the oldest available numpy wheels.

[danielk333]

Oh boy, good job catching that. Thanks! (will keep a mental note for the next time I need to build wheels)