search

cosmicrays / hermes

HERMES is a publicly available computational framework for the line of sight integration over galactic radiative processes which creates sky maps in the HEALPix-compatibile format.
GNU General Public License v3.0
22 stars 9 forks source link

Error reading the libraries #15

Closed dariograsso closed 3 years ago

dariograsso commented 3 years ago

I get this error message running the notebook in spite all required libraries are correctly installed:

ImportError Traceback (most recent call last)

in ----> 1 from pyhermes import * 2 from pyhermes.units import * 3 4 import astropy.units as u 5 ImportError: dlopen(/Users/dariograsso/Codes/hermes/build/pyhermes.so, 2): Symbol not found: __PyThreadState_Current Referenced from: /Users/dariograsso/Codes/hermes/build/pyhermes.so Expected in: flat namespace in /Users/dariograsso/Codes/hermes/build/pyhermes.so
adundovi commented 3 years ago

Hi Dario, thanks for your report. These kind of errors could come from mixing libraries compiled with different compilers. To understand it better, you should provide versions of:

  1. Jupyter Python, for example, the following code can give you the exact Python version inside Jupyter notebook: 
    import sys
print("Python version")
print (sys.version)
print("Version info.")
print (sys.version_info)
  2. libraries and python which are used to compile Hermes... for example, to get these you can attach CMakeCache.txt found in your build directory.

From those, I think, we'll see where the inconsistency comes from...

dariograsso commented 3 years ago

Hello Andrei,

thanks for the prompt answer ! Yes, that must be the problem indeed.

  1. The version I get using the commands you suggest is : Python version 3.9.5 (default, May 4 2021, 03:36:27) [Clang 12.0.0 (clang-1200.0.32.29)] Version info. sys.version_info(major=3, minor=9, micro=5, releaselevel='final', serial=0)
  2. In the CMakeCache.txt I guess the relevant lines are the following: //Python version to use for compiling modules PYBIND11_PYTHON_VERSION:STRING=2.7 //Path to a file. PYTHON_INCLUDE_DIR:PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/usr/include/python2.7 //No help, variable specified on the command line. PYTHON_INCLUDE_PATH:UNINITIALIZED=/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/include //Path to a library. PYTHON_LIBRARY:FILEPATH=/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib
adundovi commented 3 years ago

Ciao Dario,

it looks that the issue is in mismatched versions of Python, first, Python 3.9.5 is used, while later Python 2.7 is found during cmake. Everywhere should be 3.9 (or 3.9.5).

You should try specifying paths to python libraries and includes directly instead:

export PYTHON_BREW_PATH=$(brew --cellar python)/$(brew info --json python | jq -r '.[0].installed[0].version');

cmake .. \
  -DPYTHON_EXECUTABLE=$PYTHON_BREW_PATH/bin/python3 \
  -DPYTHON_LIBRARY=$PYTHON_BREW_PATH/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7.dylib \
  -DPYTHON_INCLUDE_PATH=$PYTHON_BREW_PATH/Frameworks/Python.framework/Versions/3.7/include \
  -DENABLE_TESTING=On

Or changing these paths manually with ccmake within the build folder.

dariograsso commented 3 years ago

Hi Andrei,

I specified the correct paths to cmake and it is using v 3.9.5 now:

In the Jupiter notebook, I checked that v3.9 is used using sys.info : This is what I get: Python version 3.9.5 (default, May 4 2021, 03:36:27) [Clang 12.0.0 (clang-1200.0.32.29)]

Jupyter is reading the correct library installed with make install. However, the problem persists:

ImportError: dlopen(/usr/local/lib/python3.9/site-packages/pyhermes.so, 2): Symbol not found: __PyThreadState_Current Referenced from: /usr/local/lib/python3.9/site-packages/pyhermes.so Expected in: flat namespace in /usr/local/lib/python3.9/site-packages/pyhermes.so

dariograsso commented 3 years ago

To add more info: in CMakeCache.txt I find:

//Force new FindPython PYBIND11_FINDPYTHON:BOOL=OFF

//Install pybind11 header files? PYBIND11_INSTALL:BOOL=OFF

//Disable search for Python PYBIND11_NOPYTHON:BOOL=OFF

//Python version to use for compiling modules PYBIND11_PYTHON_VERSION:STRING=2.7

//Build pybind11 test suite? PYBIND11_TEST:BOOL=OFF

//Path to a program. PYTHON_EXECUTABLE:FILEPATH=/usr/local/Cellar/python@3.9/3.9.5/bin/python3

//Path to a file. PYTHON_H_FOUND:FILEPATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/System/Library/Frameworks/Python.framework/Headers/Python.h

//Path to a file. PYTHON_INCLUDE_DIR:PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/usr/include/python2.7

//No help, variable specified on the command line. PYTHON_INCLUDE_PATH:UNINITIALIZED=/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/include

//Path to a library. PYTHON_LIBRARY:FILEPATH=/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib

//Path to a library. PYTHON_LIBRARY_DEBUG:FILEPATH=PYTHON_LIBRARY_DEBUG-NOTFOUND

It seems to be using python3.9 correctly.

However, I am not sure about this line: PYBIND11_PYTHON_VERSION:STRING=2.7

is this correct?

dariograsso commented 3 years ago

Solved!

I had to specify to cmake

cmake .. \ -DPYTHON_EXECUTABLE=$PYTHON_BREW_PATH/bin/python3 \ -DPYTHON_LIBRARY=$PYTHON_BREW_PATH/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib \ -DPYTHON_INCLUDE_DIR=$PYTHON_BREW_PATH/Frameworks/Python.framework/Versions/3.9/include/python3.9 \ -DENABLE_TESTING=On

so PYTHON_INCLUDE_DIR instead of PYTHON_INCLUDE_PATH

adundovi commented 3 years ago

OK. Closing. Thanks Dario.