Open abaleato opened 5 years ago
Can you try adding -fPIC
to CCFLAGS_NO_C and CLFLAGS in the config.auto
file and recompiling?
Thank you very much for your fast response.
I just tried your recommendation, and although the error message is different, we're still not quite there.
To be clear, what I did was: 1) delete my libsharp directory, clone it again. 2) follow the instructions in the readme, making sure to add -fPIC to CCFLAGS_NO_C and CLFLAGS in config/config.auto before running make
. 3) define LIBSHARP=/global/homes/a/ab2368/Software/libsharp/auto/
, then export LIBSHARP
. 4) Finally, cd python
and then python setup.py install
, after which I obtain the following error message:
`(delensing_modfsht) ab2368@cori09:~/Software/libsharp/python> python setup.py install Compiling libsharp/libsharp.pyx because it changed. Compiling libsharp/libsharp_mpi.pyx because it changed. [1/2] Cythonizing libsharp/libsharp.pyx /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /global/u1/a/ab2368/Software/libsharp/python/libsharp/libsharp.pxd tree = Parsing.p_module(s, pxd, full_module_name) [2/2] Cythonizing libsharp/libsharp_mpi.pyx /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /global/u1/a/ab2368/Software/libsharp/python/libsharp/libsharp_mpi.pyx tree = Parsing.p_module(s, pxd, full_module_name) running install running bdist_egg running egg_info creating libsharp.egg-info writing requirements to libsharp.egg-info/requires.txt writing libsharp.egg-info/PKG-INFO writing top-level names to libsharp.egg-info/top_level.txt writing dependency_links to libsharp.egg-info/dependency_links.txt writing manifest file 'libsharp.egg-info/SOURCES.txt' reading manifest file 'libsharp.egg-info/SOURCES.txt' writing manifest file 'libsharp.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/libsharp copying libsharp/init.py -> build/lib.linux-x86_64-2.7/libsharp creating build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_smoothing_noise_pol_mpi.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_legendre_table.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_sht.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/init.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_legendre.py -> build/lib.linux-x86_64-2.7/libsharp/tests running build_ext building 'libsharp.libsharp' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/libsharp gcc -pthread -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/global/homes/a/ab2368/Software/libsharp/auto/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/include/python2.7 -c libsharp/libsharp.c -o build/temp.linux-x86_64-2.7/libsharp/libsharp.o In file included from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:0, from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12, from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from libsharp/libsharp.c:630: /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
^~~
gcc -pthread -shared -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -L/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -Wl,-rpath=/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-2.7/libsharp/libsharp.o -L/global/homes/a/ab2368/Software/libsharp/auto/lib -L/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -lsharp -lfftpack -lc_utils -lpython2.7 -o build/lib.linux-x86_64-2.7/libsharp/libsharp.so -fopenmp
building 'libsharp.libsharp_mpi' extension
gcc -pthread -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/global/homes/a/ab2368/Software/libsharp/auto/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/include/python2.7 -c libsharp/libsharp_mpi.c -o build/temp.linux-x86_64-2.7/libsharp/libsharp_mpi.o
libsharp/libsharp_mpi.c:619:10: fatal error: mpi.h: No such file or directory
^~~~~~~
compilation terminated. error: command 'gcc' failed with exit status 1 `
even recompiling with ./configure --disable-mpi --enable-pic
does not do it... I shall keep trying.
Thanks again for your help!
Sorry, I fear I'm not familiar enough with the Python<->C ecosystem to provide any useful input. @dagss, any insight?
Thank you very much @mreineck !
Hi everyone,
I have the exact same problem as mentioned by @abaleato: when I run python setup.py install
to install the python wrapper, I get the following error:
`Compiling libsharp/libsharp.pyx because it changed. Compiling libsharp/libsharp_mpi.pyx because it changed. [1/2] Cythonizing libsharp/libsharp.pyx /home/gducrocq/anaconda3/lib/python3.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/gducrocq/libsharp/python/libsharp/libsharp.pxd tree = Parsing.p_module(s, pxd, full_module_name) [2/2] Cythonizing libsharp/libsharp_mpi.pyx /home/gducrocq/anaconda3/lib/python3.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/gducrocq/libsharp/python/libsharp/libsharp_mpi.pyx tree = Parsing.p_module(s, pxd, full_module_name) running install running bdist_egg running egg_info writing libsharp.egg-info/PKG-INFO writing dependency_links to libsharp.egg-info/dependency_links.txt writing requirements to libsharp.egg-info/requires.txt writing top-level names to libsharp.egg-info/top_level.txt reading manifest file 'libsharp.egg-info/SOURCES.txt' writing manifest file 'libsharp.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py running build_ext building 'libsharp.libsharp' extension gcc -pthread -B /home/gducrocq/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Ilibsharp -I/home/gducrocq/libsharp/auto/include -I/home/gducrocq/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/gducrocq/anaconda3/include/python3.7m -c libsharp/libsharp.c -o build/temp.linux-x86_64-3.7/libsharp/libsharp.o In file included from /home/gducrocq/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:0, from /home/gducrocq/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12, from /home/gducrocq/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from libsharp/libsharp.c:631: /home/gducrocq/anaconda3/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
^ gcc -pthread -shared -B /home/gducrocq/anaconda3/compiler_compat -L/home/gducrocq/anaconda3/lib -Wl,-rpath=/home/gducrocq/anaconda3/lib-Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/libsharp/libsharp.o -L/home/gducrocq/libsharp/auto/lib -lsharp -lfftpack-lc_utils -o build/lib.linux-x86_64-3.7/libsharp/libsharp.cpython-37m-x86_64-linux-gnu.so -fopenmp building 'libsharp.libsharp_mpi' extension gcc -pthread -B /home/gducrocq/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/gducrocq/libsharp/auto/include -I/home/gducrocq/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/gducrocq/anaconda3/include/python3.7m -c libsharp/libsharp_mpi.c -o build/temp.linux-x86_64-3.7/libsharp/libsharp_mpi.o libsharp/libsharp_mpi.c:619:17: fatal error: mpi.h: No such file or directory compilation terminated. error: command 'gcc' failed with exit status 1`
Did you find a way to handle that error ?
Thank you, Gabriel.
I finally found a temporary way around this:
After adding -fPIC to CCFLAGS_NO_C and CLFLAGS in the config.auto file and recompiling, I removed the libsharp_mpi
extension from the setup.py
file.
Then I ran python setup.py install
and everything went fine. But when I tried:
import libsharp
I got the following error:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/gducrocq/libsharp/python/libsharp/__init__.py", line 1, in <module> from .libsharp import * ModuleNotFoundError: No module named 'libsharp.libsharp'
It seems that's because the folder of the module in the site-packages
of my environment was named libsharp-0.1-py3.7-linux-x86_64.egg
. Renaming this folder as libsharp
made the trick, and now I can import it and use it.
I guess this comes at the cost of efficiency but at least I can test and debugg my code. Hope this helps :)
Hi all,
I would like to get the code to run on NERSC cori as a first step towards being able to use pixell.
So far, I have been able to successfully follow the instructions provided in the README file, culminating in the creation of a
libsharp/auto
folder with the installation. However, tryingimport sharp
from a python interpreter fails at this stage. To try to remedy this, I have triedpython setup.py install
from thelibsharp/python
directory, obtaining the following output and error message:`(delensing_modfsht) ab2368@cori06:~/Software/libsharp/python> python setup.py install Compiling libsharp/libsharp.pyx because it changed. Compiling libsharp/libsharp_mpi.pyx because it changed. [1/2] Cythonizing libsharp/libsharp.pyx /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /global/u1/a/ab2368/Software/libsharp/python/libsharp/libsharp.pxd tree = Parsing.p_module(s, pxd, full_module_name) [2/2] Cythonizing libsharp/libsharp_mpi.pyx /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /global/u1/a/ab2368/Software/libsharp/python/libsharp/libsharp_mpi.pyx tree = Parsing.p_module(s, pxd, full_module_name) running install running bdist_egg running egg_info creating libsharp.egg-info writing requirements to libsharp.egg-info/requires.txt writing libsharp.egg-info/PKG-INFO writing top-level names to libsharp.egg-info/top_level.txt writing dependency_links to libsharp.egg-info/dependency_links.txt writing manifest file 'libsharp.egg-info/SOURCES.txt' reading manifest file 'libsharp.egg-info/SOURCES.txt' writing manifest file 'libsharp.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/libsharp copying libsharp/init.py -> build/lib.linux-x86_64-2.7/libsharp creating build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_smoothing_noise_pol_mpi.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_legendre_table.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_sht.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/init.py -> build/lib.linux-x86_64-2.7/libsharp/tests copying libsharp/tests/test_legendre.py -> build/lib.linux-x86_64-2.7/libsharp/tests running build_ext building 'libsharp.libsharp' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/libsharp gcc -pthread -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/global/homes/a/ab2368/Software/libsharp/auto/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include -I/global/homes/a/ab2368/.conda/envs/delensing_modfsht/include/python2.7 -c libsharp/libsharp.c -o build/temp.linux-x86_64-2.7/libsharp/libsharp.o In file included from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:0, from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12, from /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4, from libsharp/libsharp.c:630: /global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
warning "Using deprecated NumPy API, disable it with " \
^
~~ gcc -pthread -shared -B /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat -L/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -Wl,-rpath=/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-2.7/libsharp/libsharp.o -L/global/homes/a/ab2368/Software/libsharp/auto/lib -L/global/homes/a/ab2368/.conda/envs/delensing_modfsht/lib -lsharp -lfftpack -lc_utils -lpython2.7 -o build/lib.linux-x86_64-2.7/libsharp/libsharp.so -fopenmp /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp.o): relocation R_X86_64_32 against.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_geomhelpers.o): relocation R_X86_64_32 against
.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_core.o): relocation R_X86_64_32 against.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_legendre.o): relocation R_X86_64_32S against
.rodata' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_legendre_roots.o): relocation R_X86_64_32 against.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_legendre_table.o): relocation R_X86_64_32 against
.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libsharp.a(sharp_ylmgen_c.o): relocation R_X86_64_32 against.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libfftpack.a(fftpack.o): relocation R_X86_64_32S against
.rodata' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: /global/homes/a/ab2368/Software/libsharp/auto/lib/libc_utils.a(c_utils.o): relocation R_X86_64_32 against.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /global/homes/a/ab2368/.conda/envs/delensing_modfsht/compiler_compat/ld: final link failed: nonrepresentable section on output collect2: error: ld returned 1 exit status error: command 'gcc' failed with exit status 1
I don't know if this is relevant information, but I made sure to be using gnu compilers before I started by entering
module unload PrgEnv-intel module load PrgEnv-gnu
Any ideas on how to solve this would be much appreciated.
Thank you very much,
Anton