Closed traversaro closed 11 months ago
Originally reported in https://github.com/ami-iit/hippopt/issues/8#issuecomment-1715891086 .
I reproduced the failure in CI in https://github.com/conda-forge/casadi-feedstock/pull/90 and https://github.com/conda-forge/casadi-feedstock/pull/91 (with and without re-render)
2023-09-12T15:21:27.8690455Z + python ipopt_nl.py
2023-09-12T15:21:27.8691351Z /home/conda/feedstock_root/build_artifacts/casadi_1694531457704/test_tmp/run_test.sh: line 7: 2965 Segmentation fault (core dumped) python ipopt_nl.py
2023-09-12T15:21:28.3353626Z Tests failed for casadi-3.6.3-py310h528c296_1.conda - moving package to /home/conda/feedstock_root/build_artifacts/broken
On C++, there is no segfault, the segfault is only on Python. I was able to obtain a really simple reproducer:
python -c "import ctypes; ctypes._dlopen('libcasadi_nlpsol_ipopt.so', 9)"
or even simpler:
python -c "import ctypes; ctypes._dlopen('libcasadi_nlpsol_ipopt.so', 8)"
The flags are:
RTLD_NOW: 2
RTLD_LAZY: 1
RTLD_GLOBAL: 256
RTLD_LOCAL: 0
RTLD_DEEPBIND: 8
So this seems some issue of Python with RTLD_DEEPBIND and ipopt linked with spral.
Exploring via ldd which library is segfaulting when loaded via RTLD_DEEPBIND
, it turns out that it is libgomp
, see:
python -c "import ctypes; ctypes._dlopen('/home/traversaro/miniforge3/envs/hippoptdev/lib/./libgomp.so.1', 8)"
None of the libraries linked by libgomp
have this problem, so the issue is probably due to libgomp itself. The issue was introduced in https://github.com/conda-forge/ipopt-feedstock/pull/94 as since that PR ipopt links spral that links libgomp.
Indeed, the backtrace indicate a failure inside a libgomp function:
(gdb) bt
#0 initialize_env () at ../../../libgomp/env.c:2062
#1 0x00007ffff7fc947e in call_init (l=<optimized out>, argc=argc@entry=3, argv=argv@entry=0x7fffffffc1f8, env=env@entry=0x7fffffffc218)
at ./elf/dl-init.c:70
#2 0x00007ffff7fc9568 in call_init (env=0x7fffffffc218, argv=0x7fffffffc1f8, argc=3, l=<optimized out>) at ./elf/dl-init.c:33
#3 _dl_init (main_map=0x555555b8e620, argc=3, argv=0x7fffffffc1f8, env=0x7fffffffc218) at ./elf/dl-init.c:117
#4 0x00007ffff7e09c85 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>)
at ./elf/dl-error-skeleton.c:182
#5 0x00007ffff7fd0ff6 in dl_open_worker (a=0x7fffffffb910) at ./elf/dl-open.c:808
From https://conda-forge.org/feedstock-outputs/ I get that libgomp is produced by https://github.com/conda-forge/ctng-compilers-feedstock , and as there is no specific patch for that file (see https://github.com/conda-forge/ctng-compilers-feedstock/tree/845b9c62143c4c2b2af8e35736e64aa29d8c3e05/recipe/patches ), so the failure is in https://github.com/gcc-mirror/gcc/blob/releases/gcc-13.2.0/libgomp/env.c#L2062 .
The issue happens with libgomp 13.2.0 and 13.1.0, but not with libgomp 12.* .
@SDafarra a simple workaround is to install libgomp (if you want to use spral enabled in the ipopt 3.14.12_1 build).
The crash is fixed, but a problem remains related to the use of ipopt+spral with CasADi Python bindings: https://github.com/conda-forge/casadi-feedstock/issues/93 .
TL;DR: A workaround is to pin
libgomp
to 12, i.e.mamba install libgomp=12
.Solution to issue cannot be found in the documentation.
Issue
Installed packages
Environment info