Setup Issue

[ronniyjoseph]

OSError                                   Traceback (most recent call last)
<ipython-input-1-01c7974b7be9> in <module>
----> 1 import corrcal

~/Sync/PhD/Projects/CorrCal_UKZN_Development/corrcal/__init__.py in <module>
----> 1 from .corrcal import *

~/Sync/PhD/Projects/CorrCal_UKZN_Development/corrcal/corrcal.py in <module>
      3 import numpy as np
      4 
----> 5 from . import wrapper
      6 
      7 __all__ = ['Sparse2Level', 'get_chisq', 'get_chisq_dense', 'get_gradient',

~/Sync/PhD/Projects/CorrCal_UKZN_Development/corrcal/wrapper.py in <module>
     20 # This will not be needed once we migrate to CFFI.
     21 LIB_LOC = glob(site.getsitepackages()[0] + "/corrcal/c_corrcal*.so")[0]
---> 22 mylib = ctypes.cdll.LoadLibrary(LIB_LOC)
     23 
     24 sparse_matrix_vector_multiplication = mylib.sparse_mat_times_vec_wrapper

~/Software/anaconda3/lib/python3.7/ctypes/__init__.py in LoadLibrary(self, name)
    440 
    441     def LoadLibrary(self, name):
--> 442         return self._dlltype(name)
    443 
    444 cdll = LibraryLoader(CDLL)

~/Software/anaconda3/lib/python3.7/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    362 
    363         if handle is None:
--> 364             self._handle = _dlopen(self._name, mode)
    365         else:
    366             self._handle = handle

OSError: /home/ronniyjoseph/Software/anaconda3/lib/python3.7/site-packages/corrcal/c_corrcal.cpython-37m-x86_64-linux-gnu.so: undefined symbol: omp_get_thread_num

[piyanatk]

I think your C compiler is not linked to openmp when you run pip install. What is your version of gcc?

[ronniyjoseph]

hmm I have gcc version 7.4.0

[steven-murray]

If you're on OSX, the default clang compiler does not link to openmp any more, which is very weird and annoying. You can either install your own gcc with macports or homebrew or whatever, or you can install gcc with conda, and use that.

[ronniyjoseph]

Yes, I think @piyanatk ran into this issue earlier.

So I don't get this issue if I compile the corrcal C library myself (with all linking options as provided in the original code), and do the import manually etc.

So I've been trying to find out if there is a way I can force setup to tell me what compiling command it using, but to no avail. Any suggestions?

[piyanatk]

@ronniyjoseph try

pip install -vvv .

This should print the compiler command that pip is using.

[piyanatk]

@steven-murray The thing is that @ronniyjoseph is on Linux though, which is why this is very confusing. I have no issue fresh install it on enterprise.

[steven-murray]

Oh OK, that is super strange then. What OS are you on @ronniyjoseph ? Yes, the -vvv option will print out A LOT. Sometimes I've found that the -fopenmp flag has to be set in two places in the setuptools.Setup compilation command. So I have it in extra_compile_args and also extra_link_args. Is that the case here?

[piyanatk]

We are only using extra_compile_args. @ronniyjoseph, Can you try adding extra_link_args with -fopenmp flag to setup.py. It should be added somewhere between line 31 to 34.

[piyanatk]

@ronniyjoseph Were you able to resolve the issue?

[ronniyjoseph]

I am indeed on ubuntu. I probably have to add extra extra flags. But even when I add all flags from the original compile command I get nada.

So for now I went back to manually compiling and setting the path myself

[piyanatk]

From #32, I suspect that @ronniyjoseph did python setup.py install. Are you?