search

conda-forge / casadi-feedstock

A conda-smithy repository for casadi.
BSD 3-Clause "New" or "Revised" License
8 stars 12 forks source link

Cannot load shared library 'libcasadi_nlpsol_fatrop.so' #121

Closed FabioBergonti closed 2 months ago

FabioBergonti commented 2 months ago

Solution to issue cannot be found in the documentation.

Issue

I'm testing fatrop with this code https://github.com/jgillis/fatrop_demo/blob/master/after.py, but I'm encountering the following error:

CasADi - 2024-09-03 14:56:10 WARNING(".../casadi/core/casadi_os.cpp:205: Assertion "handle!=nullptr" failed:
PluginInterface::load_plugin: Cannot load shared library 'libcasadi_nlpsol_fatrop.so':
   (
    Searched directories: 1. casadipath from GlobalOptions
                          2. CASADIPATH env var
                          3. PATH env var (Windows)
                          4. LD_LIBRARY_PATH env var (Linux)
                          5. DYLD_LIBRARY_PATH env var (osx)
    A library may be 'not found' even if the file exists:
          * library is not compatible (different compiler/bitness)
          * the dependencies are not found
   )
  Tried '/home/fbergonti/miniforge3/envs/test_casadi/lib/python3.12/site-packages/casadi' :
    Error code: /home/fbergonti/miniforge3/envs/test_casadi/lib/python3.12/site-packages/casadi/libcasadi_nlpsol_fatrop.so: cannot open shared object file: No such file or directory
  Tried '' :
    Error code: /home/fbergonti/miniforge3/envs/test_casadi/lib/python3.12/site-packages/casadi/../../../././libblasfeo.so: undefined symbol: kernel_dpack_buffer_ft
  Tried '.' :
    Error code: ./libcasadi_nlpsol_fatrop.so: cannot open shared object file: No such file or directory") [.../casadi/core/plugin_interface.hpp:142]
Traceback (most recent call last):
  File "/home/fbergonti/fatrop_demo/after.py", line 119, in <module>
    sol = opti.solve()
          ^^^^^^^^^^^^
  File "/home/fbergonti/miniforge3/envs/test_casadi/lib/python3.12/site-packages/casadi/casadi.py", line 48339, in solve
    return _casadi.Opti_solve(self, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Error in Opti::solve [OptiNode] at .../casadi/core/optistack.cpp:165:
.../casadi/core/plugin_interface.hpp:292: Plugin 'fatrop' is not found.

If I run the same code with CasADi 3.6.6 installed via pip, it works correctly.

Installed packages

# packages in environment at /home/fbergonti/miniforge3/envs/test_casadi:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_x86_64-microarch-level   4                     2_icelake    conda-forge
ampl-mp                   3.1.0             h2cc385e_1006    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
ca-certificates           2024.8.30            hbcca054_0    conda-forge
casadi                    3.6.6           py312h0385bb2_0    conda-forge
eigen                     3.4.0                h00ab1b0_0    conda-forge
icu                       75.1                 he02047a_0    conda-forge
ipopt                     3.14.16              h3696c94_4    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_7    conda-forge
libblas                   3.9.0           23_linux64_openblas    conda-forge
libblasfeo                0.1.3              had105d5_302    conda-forge
libcblas                  3.9.0           23_linux64_openblas    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libexpat                  2.6.2                h59595ed_0    conda-forge
libfatrop                 0.0.3                h5888daf_2    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.1.0               h77fa898_1    conda-forge
libgcc-ng                 14.1.0               h69a702a_1    conda-forge
libgfortran               14.1.0               h69a702a_1    conda-forge
libgfortran-ng            14.1.0               h69a702a_1    conda-forge
libgfortran5              14.1.0               hc5f4f2c_1    conda-forge
libgomp                   14.1.0               h77fa898_1    conda-forge
libhwloc                  2.11.1          default_hecaa2ac_1000    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
liblapack                 3.9.0           23_linux64_openblas    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.27          pthreads_hac2b453_1    conda-forge
libosqp                   0.6.3                h59595ed_0    conda-forge
libqdldl                  0.1.5                h27087fc_1    conda-forge
libscotch                 7.0.4                h2fe6a88_5    conda-forge
libspral                  2024.05.08           h831f25b_3    conda-forge
libsqlite                 3.46.1               hadc24fc_0    conda-forge
libstdcxx                 14.1.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.1.0               h4852527_1    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
metis                     5.1.0             h59595ed_1007    conda-forge
mumps-include             5.7.2                ha770c72_0    conda-forge
mumps-seq                 5.7.2                h6e8dedb_0    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
numpy                     2.1.0           py312h58c1407_1    conda-forge
openssl                   3.3.1                hb9d3cd8_3    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
proxsuite                 0.6.7           py312h68727a3_0    conda-forge
python                    3.12.5          h2ad013b_0_cpython    conda-forge
python_abi                3.12                    5_cp312    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scipy                     1.14.1          py312h7d485d2_0    conda-forge
setuptools                73.0.1             pyhd8ed1ab_0    conda-forge
simde                     0.8.2                h84d6215_0    conda-forge
tinyxml2                  10.0.0               h59595ed_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tzdata                    2024a                h8827d51_1    conda-forge
unixodbc                  2.3.12               h661eb56_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.3.1                h4ab18f5_1    conda-forge

Environment info

active environment : test_casadi
    active env location : /home/fbergonti/miniforge3/envs/test_casadi
            shell level : 1
       user config file : /home/fbergonti/.condarc
 populated config files : /home/fbergonti/miniforge3/.condarc
                          /home/fbergonti/.condarc
          conda version : 24.5.0
    conda-build version : not installed
         python version : 3.10.14.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=icelake
                          __conda=24.5.0=0
                          __cuda=12.4=0
                          __glibc=2.35=0
                          __linux=5.15.153.1=0
                          __unix=0=0
       base environment : /home/fbergonti/miniforge3  (writable)
      conda av data dir : /home/fbergonti/miniforge3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/fbergonti/miniforge3/pkgs
                          /home/fbergonti/.conda/pkgs
       envs directories : /home/fbergonti/miniforge3/envs
                          /home/fbergonti/.conda/envs
               platform : linux-64
             user-agent : conda/24.5.0 requests/2.31.0 CPython/3.10.14 Linux/5.15.153.1-microsoft-standard-WSL2 ubuntu/22.04.3 glibc/2.35 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
traversaro commented 2 months ago

The key error is:

    Error code: /home/fbergonti/miniforge3/envs/test_casadi/lib/python3.12/site-packages/casadi/../../../././libblasfeo.so: undefined symbol: kernel_dpack_buffer_ft

Indeed, there are some symbols missing in blasfeo, see:

(casadi366) traversaro@IITBMP014LW012:~/fatrop_demo$ nm ${CONDA_PREFIX}/lib/libblasfeo.so | grep "U kernel"
                 U kernel_dgetr_tn_4_lib
                 U kernel_dgetr_tn_4_vs_lib
                 U kernel_dgetr_tn_8_lib
                 U kernel_dpack_buffer_fn
                 U kernel_dpack_buffer_ft
                 U kernel_dpack_buffer_ln
                 U kernel_dpack_buffer_lt
                 U kernel_dpack_buffer_ut
traversaro commented 2 months ago

I fixed locally by backporting, but the example still fails with:

(casadi366) traversaro@IITBMP014LW012:~/fatrop_demo$ python after.py
 it   obj            cv        du        lg(mu) reg  alpha_du  alpha_pr  ls
  0   2.1000000e+02  9.70e-01  9.85e-01   2.0   -.-  0.00e+00  0.00e+00  0x
  1   2.3645160e+02  4.63e-01  7.85e+00  -0.1   -.-  1.22e-01  1.70e-01  2h
  2   2.2329015e+02  3.77e-01  8.55e+00  -0.1   -.-  7.95e-01  1.78e-01  1h
  3   2.3412732e+02  1.03e-01  3.23e+00  -0.1   -.-  6.06e-01  1.00e+00  1h
  4   2.2272949e+02  4.34e-02  1.18e+00  -0.8   -.-  1.00e+00  7.83e-01  1h
  5   2.1727447e+02  7.26e-03  1.85e-01  -1.5   -.-  1.00e+00  1.00e+00  1h
  6   2.1665629e+02  1.93e-03  1.82e-02  -2.2   -.-  1.00e+00  1.00e+00  1h
  7   2.1655221e+02  4.43e-05  1.11e-03  -3.4   -.-  1.00e+00  9.99e-01  1h
  8   2.1654383e+02  4.03e-07  6.74e-06  -5.0   -.-  1.00e+00  1.00e+00  1h
  9   2.1654363e+02  1.16e-10  1.76e-09  -7.6   -.-  1.00e+00  1.00e+00  1h
 10   2.1654363e+02  7.11e-15  1.21e-14  -9.0   -.-  1.00e+00  1.00e+00  1h
---- stats ----
compute_sd:     0.000365 s
duinf:          3.1e-05 s
initialization: 0.000472 s  count: 10
time_FE :       0.004185 s
    eval hess:  0.002379 s  count: 10
    eval jac:   0.001274 s  count: 12
    eval cv:    0.000423 s  count: 14
    eval grad:  4.2e-05 s  count: 12
    eval obj:   6.7e-05 s  count: 35
rest  :       0.000648 s
-----
time_w/o_FE : 0.001516 s
time_FE :       0.004185 s
time_total : 0.005701 s  iterations: 10
found solution
      solver  :   t_proc      (avg)   t_wall      (avg)    n_eval
       nlp_f  |  39.00us (  1.11us)  34.41us (983.14ns)        35
       nlp_g  | 372.00us ( 26.57us) 368.98us ( 26.36us)        14
  nlp_grad_f  |  16.00us (  1.33us)  17.53us (  1.46us)        12
  nlp_hess_l  |   2.29ms (228.60us)   2.29ms (229.08us)        10
   nlp_jac_g  |   1.14ms ( 94.75us)   1.14ms ( 94.79us)        12
       total  |   5.98ms (  5.98ms)  14.58ms ( 14.58ms)         1
Traceback (most recent call last):
  File "/home/traversaro/fatrop_demo/after.py", line 119, in <module>
    sol = opti.solve()
          ^^^^^^^^^^^^
  File "/home/traversaro/miniforge3/envs/casadi366/lib/python3.12/site-packages/casadi/casadi.py", line 48339, in solve
    return _casadi.Opti_solve(self, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Error in Opti::solve [OptiNode] at .../casadi/core/optistack.cpp:165:
.../casadi/core/generic_type.cpp:438: Assertion "is_string()" failed:
type mismatch

Probably we should include the python after.py example in the feedstock to check if fatrop works.

traversaro commented 2 months ago

Interestingly, I also get the `Assertion "is_string()" failed" error with casadi installed via pip, @FabioBergonti can you report the conda list of the environment were that is working?

FabioBergonti commented 2 months ago

Interestingly, I also get the `Assertion "is_string()" failed" error with casadi installed via pip, @FabioBergonti can you report the conda list of the environment were that is working?

It is failing also on my laptop. The example seems to have a bug as pointed out in this issue. I resolved it with this workaround.

traversaro commented 2 months ago

Ah great, that means by fix works, I will open a PR in blasfeo-feedstock.

traversaro commented 2 months ago

Ah great, that means by fix works, I will open a PR in blasfeo-feedstock.

Done in https://github.com/conda-forge/blasfeo-feedstock/pull/9 .

traversaro commented 2 months ago

Fixed in https://github.com/conda-forge/blasfeo-feedstock/pull/9 .