"NumPy is not present" when installing via 'pip'

[yaronetokayer]

Dear Dr. Vasiliev,

I am encountering the following set of issues when attempting to install agama.

About my machine: I am running macOS Sonoma 14.5 with the Apple M2 Pro chip. I use pynev to manage my python versions. I am in a virtual environment that uses Python 3.12.4. I manage my packages with homebrew.

1) When I run 'pip install agama', I get the following error:

(halos-py3) yaronetokayer@Yarones-MacBook-Pro-2 ~ % pip install agama
Collecting agama
  Using cached agama-1.00.tar.gz (2.0 kB)
  Installing build dependencies ... done
Running /private/var/folders/35/s1f1jlrj2_v9zq2h14brv2xm0000gn/T/pip-install-1uibrj0i/agama_9c48701dfc7e4bc981ea5c8c26c6ffba/setup.py
Downloading the latest version of package from github
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: agama

    ==== Checking supported compiler options and available libraries ====

  Building wheel for agama (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for agama (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]

          ==== Checking supported compiler options and available libraries ====

      clang: error: unsupported option '-fopenmp'
      clang: error: unsupported option '-fopenmp'
      test.cpp:1:10: fatal error: 'omp.h' file not found
      #include <omp.h>
               ^~~~~~~
      1 error generated.
      ld: library 'gomp' not found
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      ld: library 'omp' not found
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      clang: error: unknown argument: '-qno-opt-dynamic-align'
      c++ -fPIC -Wall -O2 test.cpp -o test.out
      c++ -fPIC -Wall -O2 test.cpp -o test.out -fopenmp -Werror -Wno-unknown-pragmas
      c++ -fPIC -Wall -O2 test.cpp -o test.out
      c++ -fPIC -Wall -O2 test.cpp -o test.out -I/opt/homebrew/opt/libomp/include
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include test.cpp -o test.out -lgomp -Xpreprocessor -fopenmp -Werror -Wno-unknown-pragmas
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include test.cpp -o test.out -lomp -Xpreprocessor -fopenmp -Werror -Wno-unknown-pragmas
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include test.cpp -o test.out -L/opt/homebrew/opt/libomp/lib -lomp -Xpreprocessor -fopenmp -Werror -Wno-unknown-pragmas
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include -Xpreprocessor -fopenmp -L/opt/homebrew/opt/libomp/lib -lomp test.cpp -o test.out -std=c++11
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include -Xpreprocessor -fopenmp -std=c++11 -L/opt/homebrew/opt/libomp/lib -lomp test.cpp -o test.out -arch arm64
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include -Xpreprocessor -fopenmp -std=c++11 -arch arm64 -L/opt/homebrew/opt/libomp/lib -lomp test.cpp -o test.out -qno-opt-dynamic-align
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include -Xpreprocessor -fopenmp -std=c++11 -arch arm64 -L/opt/homebrew/opt/libomp/lib -lomp test.cpp -o test.out -Werror -Wno-missing-field-initializers
      c++ -fPIC -Wall -O2 -I/opt/homebrew/opt/libomp/include -Xpreprocessor -fopenmp -std=c++11 -arch arm64 -Wno-missing-field-initializers -L/opt/homebrew/opt/libomp/lib -lomp test.cpp -o test.out -Werror -Wno-cast-function-type
      error: NumPy is not present - python extension cannot be compiled
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for agama
Failed to build agama
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (agama)
(halos-py3) yaronetokayer@Yarones-MacBook-Pro-2 ~ % 

Note that this happens both when numpy 2.0.0 is installed and when I downgrade numpy. As an aside, a colleague who is attempting to install py-sphviewer on his Linux machine is encountering a similar issue, where he gets ModuleNotFoundError: No module names 'numpy'

In case it's relevant, I have both libomp and open-mp as packages installed via homebrew...

2) Since this wasn't working, I tried a manual installation of Agama, and after tinkering around with the Makefile.local, was able to get 'make' to run without fatal errors and do a pip install ./ from that local directory. However, once I did that, anytime I tried to import agama from python, my python kernel shut down. I got a zsh: segmentation error. This happened both when I would do pip install ./ from Agama-master, and when I would just add the Agama-master directory to the PYTHONPATH... In case it's helpful, I've attached my Makefile and Makefile.local files, which I modified.

Thank you for your help!

agama_makefile_copy.txt agama_makefile_local_copy.txt

[eugvas]

hmm, this is strange – the error "Numpy is not present" appears when the setup.py script tries to import numpy and encounters an ImportError. It looks like running pip install inside a pyenv does not pick up the correct environment, can you try running python setup.py install instead? (it may complain about this being an outdated procedure, but so far it has worked on all recent version of python). A side note: the script correctly picked up OpenMP from homebrew; the two identical error messages resulted from various attempts to link against it, but finally it has found the correct configuration, as evidenced by the "-fopenmp" flags appearing in the later lines. Another note is that on my machine (nearly identical setup, but using Anaconda rather than pyenv) I didn't manage to get it working with NumPy 2.0 due to conflicting versions of OpenMP (same problem as in #39); need further investigation. However, if you're not using Anacoda (which comes with its own libomp), you might be fine.

[yaronetokayer]

Thank you!

I ran python setup.py install like you recommended from the Agama-master and the script ran without raising any issues related to NumPy (this was with Numpy 2.0.0). It eventually finished and I am able to import agama into my python session!

A few notes and follow up questions (to the extent that they make sense - it's possible the questions will betray my own lack of understanding...): 1) While running the script, I got a few messages about gsl and eigen not being found, even though I do have those packages installed through homebrew. I replied Y to install them now, but will this cause problems with having conflicting GSL and eigen installations? Or will the ones it installed just stay local in my Agama-master directory?

I was also asked about cvxopt and unsio, but I didn't already have those on my machine.

2) When I run pip list, agama 1.0 is listed twice. Is that normal?

3) While I run pip list, cvxopt 1.3.2 is shown, but not unsio. Does this mean that one of these was installed in my environment site-packages, while the other was only installed in the Agama-master directory?

[eugvas]

great that it worked, so I need to investigate why pip does not correctly capture the current pyenv setting.

• 1. While running the script, I got a few messages about gsl and eigen not being found, even though I do have those packages installed through homebrew. I replied Y to install them now, but will this cause problems with having conflicting GSL and eigen installations? Or will the ones it installed just stay local in my Agama-master directory? I was also asked about cvxopt and unsio, but I didn't already have those on my machine. *

in general, the setup script first tries to use existing versions of these libraries, but if for some reason this failed (I'm not sure why this happened in your case), it will download and compile them. I should make it clear that this does not interfere in any way with the existing ones, because these libraries (GSL, Eigen and UNSIO) are put into the extras folder of the Agama distribution, and thus are only visible to the make program that compiles Agama. These are C or C++ libraries, hence not visible to pip (or any package manager – conda, homebrew, macports, etc., for that matter). CVXOPT is a different story, because this is a Python extension module, and hence it gets installed in the same virtual environment as Agama itself (and should appear in pip list).

• 2. When I run pip list, agama 1.0 is listed twice. Is that normal? *

I'm not sure why this happens, but I've also seen it occasionally; seems to cause no harm.

• 3. While I run pip list, cvxopt 1.3.2 is shown, but not unsio. Does this mean that one of these was installed in my environment site-packages, while the other was only installed in the Agama-master directory? *

yes – see above.

[yaronetokayer]

As an FYI, you mentioned that after running python setup.py install instead of pip install ./, "it may complain about this being an outdated procedure, but so far it has worked on all recent version of python":

When I run pip from my pyenv virtual environment, I am indeed being given the warning you predicted:

DEPRECATION: Loading egg at /Users/yaronetokayer/.pyenv/versions/3.12.4/envs/halos-py3/lib/python3.12/site-packages/agama-1.0-py3.12-macosx-14.5-arm64.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330

We're currently at pip 24.1.2, so it looks like at some point another work around will be needed.

Thank you again for your help! I have been running agama without issue from my environment.

[eugvas]

The installation script has been updated, and usually works fine when invoked both as python setup.py install (deprecated) or as pip install .; however, in some cases pip may decide to use "isolated build environment", in which case you get the error "Numpy is not present". If the packages setuptools, wheel and of course numpy are installed in the current virtual environment, this seems to prevent this problem, but if it appears, the solution is to use a flag --no-build-isolation for pip.