Built OK but cannot import on Mac

[zoeye859]

Hi,

I have seen past issues on compiling sigpyproc3 on Mac. Here I share my experience with a successful compiling, but my problem is I cannot import the installed sigpyproc (sadly). Please give some instructions.

1. Install OpenMP

brew install libomp

2. Install clang-omp using homebrew:

brew install llvm

Add llvm binaries to your path using :

echo 'export PATH="/usr/local/opt/llvm/bin:$PATH"' >> ~/.bash_profile

echo 'export PATH="/usr/local/include:$PATH"' >> ~/.bash_profile

echo 'export PATH="/usr/local/lib:$PATH"' >> ~/.bash_profile

3. Test clang usgae:

clang -fopenmp hello.c -o hello -L /usr/local/lib/

./hello

You can create any simple hello.c file here to test -fopenmp and clang

4. linking

ln -s /usr/local/Cellar/gcc/10.2.0/lib/gcc/10/libgomp.spec /usr/local/lib/libgomp.spec
ln -s /usr/local/Cellar/gcc/10.2.0/lib/gcc/10/libgomp.1.dylib /usr/local/lib/libgomp.1.dylib
ln -s /usr/local/Cellar/gcc/10.2.0/lib/gcc/10/libgomp.dylib /usr/local/lib/libgomp.dylib
ln -s /usr/local/Cellar/gcc/10.2.0/lib/gcc/10/libgomp.a /usr/local/lib/libgomp.a

5. Installation

> pip3 install git+https://github.com/FRBs/sigpyproc3

Collecting git+https://github.com/FRBs/sigpyproc3
  Cloning https://github.com/FRBs/sigpyproc3 to /private/var/folders/62/chn0plln2b37czw0t47n9kd80000gn/T/pip-req-build-83fgta5g
  Running command git clone -q https://github.com/FRBs/sigpyproc3 /private/var/folders/62/chn0plln2b37czw0t47n9kd80000gn/T/pip-req-build-83fgta5g
Requirement already satisfied (use --upgrade to upgrade): sigpyproc==0.5.1 from git+https://github.com/FRBs/sigpyproc3 in /usr/local/lib/python3.8/site-packages
Requirement already satisfied: pybind11>=2.6.0 in /usr/local/lib/python3.8/site-packages (from sigpyproc==0.5.1) (2.6.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.8/site-packages (from sigpyproc==0.5.1) (1.19.4)
Requirement already satisfied: tqdm in /usr/local/lib/python3.8/site-packages (from sigpyproc==0.5.1) (4.53.0)
Building wheels for collected packages: sigpyproc
  Building wheel for sigpyproc (setup.py) ... done
  Created wheel for sigpyproc: filename=sigpyproc-0.5.1-cp38-cp38-macosx_10_15_x86_64.whl size=138411 sha256=6eaf5ceb8639cf1b00d76989f1a6755f6289257663a9f978dbf02329df934f96
  Stored in directory: /private/var/folders/62/chn0plln2b37czw0t47n9kd80000gn/T/pip-ephem-wheel-cache-f8hrvu4u/wheels/16/24/22/1cf298bc509480534c02d09f5529f91c47cb10053eba7b6a12
Successfully built sigpyproc

Problem:

I don't know whether it is installed properly, so I checked available python3 libraries:

>help("modules")

I can find that sigpyproc is inside the list.

However, I cannot import sigpyproc:

> python3
Python 3.8.5 (default, Jul 21 2020, 10:48:26) 
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from sigpyproc.Readers import FilReader
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/sigpyproc/__init__.py", line 1, in <module>
    from sigpyproc.Readers import FilReader
  File "/usr/local/lib/python3.8/site-packages/sigpyproc/Readers.py", line 6, in <module>
    from sigpyproc.Utils import File
  File "/usr/local/lib/python3.8/site-packages/sigpyproc/Utils.py", line 5, in <module>
    import sigpyproc.libSigPyProc as lib
ImportError: dlopen(/usr/local/lib/python3.8/site-packages/sigpyproc/libSigPyProc.cpython-38-darwin.so, 2): Symbol not found: ___kmpc_for_static_fini
  Referenced from: /usr/local/lib/python3.8/site-packages/sigpyproc/libSigPyProc.cpython-38-darwin.so
  Expected in: flat namespace
 in /usr/local/lib/python3.8/site-packages/sigpyproc/libSigPyProc.cpython-38-darwin.so
>>> 

I am stuck here. Any advice? Thank you very much!

Best,
Zoe

[zoeye859]

Hi,

I just tried to uninstall the library and reinstall it again:

> pip3 uninstall sigpyproc
Uninstalling sigpyproc-0.5.1:
  Would remove:
    /usr/local/lib/python3.8/site-packages/sigpyproc-0.5.1.dist-info/*
    /usr/local/lib/python3.8/site-packages/sigpyproc/*
Proceed (y/n)? y
  Successfully uninstalled sigpyproc-0.5.1

Reinstall:

> pip3 install git+https://github.com/FRBs/sigpyproc3
Collecting git+https://github.com/FRBs/sigpyproc3
  Cloning https://github.com/FRBs/sigpyproc3 to /private/var/folders/62/chn0plln2b37czw0t47n9kd80000gn/T/pip-req-build-3ot4mcjk
  Running command git clone -q https://github.com/FRBs/sigpyproc3 /private/var/folders/62/chn0plln2b37czw0t47n9kd80000gn/T/pip-req-build-3ot4mcjk
Requirement already satisfied: pybind11>=2.6.0 in /usr/local/lib/python3.8/site-packages (from sigpyproc==0.5.1) (2.6.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.8/site-packages (from sigpyproc==0.5.1) (1.19.4)
Requirement already satisfied: tqdm in /usr/local/lib/python3.8/site-packages (from sigpyproc==0.5.1) (4.53.0)
Building wheels for collected packages: sigpyproc
  Building wheel for sigpyproc (setup.py) ... done
  Created wheel for sigpyproc: filename=sigpyproc-0.5.1-cp38-cp38-macosx_10_15_x86_64.whl size=138411 sha256=b80316636c80212af3fb38246ae38b030b48c2765775d4a9f029e1fbf3de0799
  Stored in directory: /private/var/folders/62/chn0plln2b37czw0t47n9kd80000gn/T/pip-ephem-wheel-cache-5r9zekzt/wheels/16/24/22/1cf298bc509480534c02d09f5529f91c47cb10053eba7b6a12
Successfully built sigpyproc
Installing collected packages: sigpyproc
  Attempting uninstall: sigpyproc
    Found existing installation: sigpyproc 0.1.1
    Uninstalling sigpyproc-0.1.1:
      Successfully uninstalled sigpyproc-0.1.1
Successfully installed sigpyproc-0.5.1

This time I get the message Successfully installed sigpyproc-0.5.1

However, when I tried to import the library in python3, I obtained the same error:

% python3
Python 3.8.5 (default, Jul 21 2020, 10:48:26) 
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from sigpyproc.Readers import FilReader
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/sigpyproc/__init__.py", line 1, in <module>
    from sigpyproc.Readers import FilReader
  File "/usr/local/lib/python3.8/site-packages/sigpyproc/Readers.py", line 6, in <module>
    from sigpyproc.Utils import File
  File "/usr/local/lib/python3.8/site-packages/sigpyproc/Utils.py", line 5, in <module>
    import sigpyproc.libSigPyProc as lib
ImportError: dlopen(/usr/local/lib/python3.8/site-packages/sigpyproc/libSigPyProc.cpython-38-darwin.so, 2): Symbol not found: ___kmpc_for_static_fini
  Referenced from: /usr/local/lib/python3.8/site-packages/sigpyproc/libSigPyProc.cpython-38-darwin.so
  Expected in: flat namespace
 in /usr/local/lib/python3.8/site-packages/sigpyproc/libSigPyProc.cpython-38-darwin.so
>>> 

Your help would be very much appreciated!

Best, Zoe

[pravirkr]

Hi,

Thanks for pointing out this error. It should work now with #11

Just install OpenMP and FFTW with brew. I suppose this will install the files into /usr/local/Cellar/ and symbolically link them to /usr/local. The setup.py now includes this path /usr/local for compilation. So, no need to export paths now.

I have not much experience with macOS and have only tested this fix on a fresh macOS environment with Github CI. So, feel free to test this fix and let me know the results. I will merge to the main repo later.

brew install libomp fftw
pip install git+https://github.com/FRBs/sigpyproc3.git@macOS_fix

@telegraphic, Do you think this solves the macOS issues?

[telegraphic]

@pravirkr those __kmpc symbols do appear to come from openmp, so agreed that a closer look at openmp is a first port of call. And +1 for installing those via brew if using system or homebrew Python. @zoeye859 did mention installing libomp though... Could be a path issue / versionitis, finding incorrect openmp install?

If using anaconda/miniconda python, I would instead try using conda install -c conda-forge fftw openmp.

[telegraphic]

Ah, just looked at #11 which looks like you're now handling different compile flags for MacOS libomp. That seems like it should fix it!

[pravirkr]

@telegraphic yeah, conda is also a good option.

@zoeye859 closing the issue now. Feel free to re-open the issue in case the fix doesn't work well.

[zoeye859]

pip3 install git+https://github.com/FRBs/sigpyproc3.git@macOS_fix

This fixed my problem. Thank you for the joint efforts!

Best, Zoe