Python wrapper on Mac Catalina 10.15

[ShazAlvi]

Similar issues have been faced by other people, as mentioned in #416 , #356, #349 and #229 (I think all of them are somewhat related issues). But when I import classy in ipython, I continue to get this error,

Python 3.9.7 (default, Sep 16 2021, 08:50:36) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from classy import Class
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-b845b2c38d61> in <module>
----> 1 from classy import Class

ImportError: dlopen(/Users/infn/opt/anaconda3/envs/BasePython/lib/python3.9/site-packages/classy.cpython-39-darwin.so, 2): Symbol not found: _GOMP_loop_nonmonotonic_dynamic_next
  Referenced from: /Users/infn/opt/anaconda3/envs/BasePython/lib/python3.9/site-packages/classy.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/infn/opt/anaconda3/envs/BasePython/lib/python3.9/site-packages/classy.cpython-39-darwin.so

I have followed the remedies given in the issues I mentioned above but they do not solve my problem. I installed gcc-11 using homebrew and I am working in Anaconda environment with python 3.9. Here is the make edit, CC = gcc-11

classy_ext = Extension("classy", [os.path.join(classy_folder, "classy.pyx")],
                           include_dirs=[nm.get_include(), include_folder],
                           libraries=liblist,
                           library_dirs=[root_folder, GCCPATH],
                           extra_link_args=['-lgomp', '-Wl,-rpath,/usr/local/opt/gcc/lib/gcc/11/'])

I have already verified that the path mentioned here /usr/local/opt/gcc/lib/gcc/11/ contains libgomp.dylib, libgomp.a, libgomp.1.dylib and libgomp.spec. It still gives the same error. In issue #356, one person recommended to delete the file ~/.local/lib/python2.7/site-packages/classy.so but file does not exists (which I understand should exist). Perhaps this is part of the problem.

Is there something else that I can try? At the moment, I am trying to do the same in Python 2.7 and to see if I can make it to work there.

[ShazAlvi]

I was unable to solve the issue but I was able to do things in the conda Python2.7 environment. However the installation seems to have problem in the Python wrapper which I describe in issue #463

[gplynch619]

I was able to get the classy working on macOS Monterey 12.1, running on the Apple M1 Pro. I suspect my fix might work for you as well despite the difference in versions.

Roughly speaking, there are at least two reasons the wrapper build might fail. The first is that it could be that your .so files were built for a different architecture than your python executable (say, if you installed an x86-64 version of python, but your .so files were compiled for an arm64 architecture). If this is the case, the error message you get it related but different than the one you have posted. I mention this only for others who might come across the same problem.

If you have ensured that the architectures of your .so files match what the system expects, then it is still possible you will get the Symbol not found error you mention. I believe what is happening here is that your python package manager (I will assume here that it is conda, as that is what I used) is calling the wrong compiler when it needs to create .so files when installing packages. Very briefly, different compilers will sometimes name functions differently behind the scenes. So if CLASS was compiled with a different compiler than conda is using to build its .so files, the linker will not be able to match symbols when it needs to. (I am not entirely clear on the details of the compilation process so I welcome clarifications).

The solution is to ensure that everything is being compiled with the same compiler. This is how I did so.

1. Install gcc using homebrew. I will call the location it installs to /path/to/gcc-11. For me, this was under /opt/homebrew/Cellar
2. Next we want to create a fresh python environment using conda, because we want to force packages to install with a specific compiler. Run conda create --name classy python=3.9 and then conda activate classy to activate the new environment.
3. Now we need to make some changes to the CLASS Makefile. Change the CC environment variable to point towards gcc installed with homebrew by setting CC = /path/to/gcc-11. To be safe, we want to make sure the Makefile knows which python we are using. Set PYTHON =/path/to/conda/envs/classy/bin/python. I'm not strictly sure this step is necessary if we have already activate the environment.
4. Now we want to install the necessary packages using the same compiler we used for CLASS. Run env CC=/path/to/gcc-11 conda install numpy scipy cython. This will install the appropriate packages.
5. Now run make clean && make to first clean your directory and rebuild CLASS.
6. Run cd python to change into the python directory. We want to make a few changes to setup.py. In the definition of GCCPATH_STRING, change 'gcc' to '/path/to/gcc-11', and likewise in the definition of MVEC_STRING.
7. Now we are ready to finish setting up the wrapper. Run env CC=/path/to/gcc-11 python setup.py install --user. This should properly set up the wrapper.

You can check that everything worked by running python -c "import classy as Class" and hopefully getting no errors. This environment is now set up to use classy.

These are the steps that worked for me, so I believe they should be sufficient. However I am not sure if they are all necessary. It may be easier to compile everything with clang as is done in #405 but I have not tried that yet.