For some strange reason, the python3.8 CI/CD for linux stopped working. The error message was the following:
ImportError while loading conftest '/home/vsts/work/1/s/pycraf/conftest.py'.
pycraf/__init__.py:65: in <module>
from . import pathprof
pycraf/pathprof/__init__.py:11: in <module>
from .cyprop import *
E ImportError: /home/vsts/work/1/s/pycraf/pathprof/cyprop.cpython-38-x86_64-linux-gnu.so: undefined symbol: omp_set_num_threads
This only happens on Python 3.8, despite most dependency package versions being identical to other Python versions.
A first hint towards the issue was found by switching on the pip debug mode:
Running command Getting requirements to build wheel
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: warning: librt.so.1, needed by /usr/share/miniconda/envs/pycraf-env/lib/libgomp.so, not found (try using -rpath or -rpath-link)
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: warning: libdl.so.2, needed by /usr/share/miniconda/envs/pycraf-env/lib/libgomp.so, not found (try using -rpath or -rpath-link)
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: /usr/share/miniconda/envs/pycraf-env/lib/libgomp.so: undefined reference to `dlopen@GLIBC_2.2.5'
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: /usr/share/miniconda/envs/pycraf-env/lib/libgomp.so: undefined reference to `dlerror@GLIBC_2.2.5'
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: /usr/share/miniconda/envs/pycraf-env/lib/libgomp.so: undefined reference to `dlclose@GLIBC_2.2.5'
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: /usr/share/miniconda/envs/pycraf-env/lib/libgomp.so: undefined reference to `dlsym@GLIBC_2.2.5'
collect2: error: ld returned 1 exit status
Cannot compile Cython/C/C++ extension with OpenMP, reverting to non-parallel code
On a side note: this also showed that pip was downloading several packages again, despite being present in the conda env already. This seems to be the desired behavior nowadays, with PEP518, which creates the build environment in an isolated manner (ignoring existing packages). That means that all build dependencies specified in pyproject.toml are installed from PyPI again. With the switch --no-build-isolation this can be turned off (which makes sense on Azure).
The above problem was obviously a problem with GLIBC. Installing sysroot_linux-64 helps, but leads to another problem:
Running command Preparing metadata (pyproject.toml)
/usr/share/miniconda/envs/pycraf-env/compiler_compat/ld: /usr/share/miniconda/envs/pycraf-env/bin/../x86_64-conda-linux-gnu/sysroot/lib64/librt.so.1: undefined reference to `__vdso_clock_gettime@GLIBC_PRIVATE'
collect2: error: ld returned 1 exit status
Cannot compile Cython/C/C++ extension with OpenMP, reverting to non-parallel code
A very blunt solution is, to just delete the librt.so file... The compiler will throw a warning, but it seems to work.
For some strange reason, the python3.8 CI/CD for linux stopped working. The error message was the following:
This only happens on Python 3.8, despite most dependency package versions being identical to other Python versions.
A first hint towards the issue was found by switching on the pip debug mode:
On a side note: this also showed that pip was downloading several packages again, despite being present in the conda env already. This seems to be the desired behavior nowadays, with PEP518, which creates the build environment in an isolated manner (ignoring existing packages). That means that all build dependencies specified in
pyproject.tomlare installed from PyPI again. With the switch--no-build-isolationthis can be turned off (which makes sense on Azure).The above problem was obviously a problem with GLIBC. Installing
sysroot_linux-64helps, but leads to another problem:A very blunt solution is, to just delete the
librt.sofile... The compiler will throw a warning, but it seems to work.