search

tilmantroester / pyhmcode

Augmented halo model for accurate non-linear matter power spectrum calculations
MIT License
4 stars 4 forks source link

Installation issue with `pip` on `anaconda`. #6

Closed nikfilippas closed 1 year ago

nikfilippas commented 1 year ago

As this package is not released on anaconda, I used my environment's pip to install it. It installs fine but it complains about a numpy version mismatch on import:

>>> import pyhmcode
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nick/anaconda/lib/python3.9/site-packages/pyhmcode/__init__.py", line 2, in <module>
    from . import _pyhmcode
ImportError: numpy.core.multiarray failed to import

The two versions of numpy it is complaining about are 2 releases apart (one is v16 while the other one is v14?). Not sure if this is a phmcode-related problem, but no other modules complain about my numpy version (currently using 1.21.5). Unfortunately I can't update numpy because np.int and np.float later go through their deprecation cycle and many of the other modules I use which rely on these will break.

I don't think pip is picking up another version of numpy while building pyhmcode because there is only one version of it installed. Also, which pip is /home/nick/anaconda/bin/pip and numpy.__path__ is ['/home/nick/anaconda/lib/python3.9/site-packages/numpy'].

Force-reinstalling the latest version of numpy (the one which breaks other modules) doesn't work. Force-reinstalling 1.21 doesn't work either. Removing every trace of pyhmcode and reinstalling doesn't work. Downloading and building from setup.py using pip yields the same issue.

tilmantroester commented 1 year ago

It could be that during the build, f90wrap uses numpy 1.23 (see https://github.com/jameskermode/f90wrap/blob/master/pyproject.toml#L12). The build environments can be separate from the installation environments, but these build systems are somewhat black magic to me.

nikfilippas commented 1 year ago

Any ideas on how to work around this? I have previously managed to install pyhmcode on another machine with no problem.

tilmantroester commented 1 year ago

The installation is broken in weird ways for me too. What seems to be working for me right now is manually installing f90wrap first (pip install f90wrap). But I don't understand why that's working.

nikfilippas commented 1 year ago

f90wrap is already installed though. That's really odd...

nikfilippas commented 1 year ago

Update: Turns out I had two versions of Numpy (v1.21 & v1.23) installed in parallel in my environment. One through conda, and the other one was automatically installed with one of the pip-only supported packages. This caused f90wrap to use the wrong version to build pyhmcode, which in the end caused the problem on import.

I force-removed every trace of Numpy from my environment, re-installed Numpy v1.23 using the conda (mamba) installer, then f90wrap, then pyhmcode, and it is now working as intended. May I suggest considering releasing pyhmcode on anaconda to completely avoid this issue in the future.

tilmantroester commented 1 year ago

That might explain why it was suddenly working for me again too. I'll look into a conda package but I'm also considering just porting the relevant parts (HMCode, HMx) to python to avoid these headaches in the future.