JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.45k stars 186 forks source link

libstdc++.so.6: version `GLIBCXX_3.4.30' not found #999

Open torrance opened 1 year ago

torrance commented 1 year ago

I'm encountering an issue where:

  1. A Conda package is built against libc 3.4.30 (in this case, python-casacore).
  2. Conda ships with libc 3.4.30.
  3. Running ldd libcasa_casa.so shows that casa is correctly linked to the Conda version of libc.
  4. If I enter the Julia-managed conda (~/.julia/3/bin/python) and import casacore.tables it works fine.

BUT, when I pyimport("casacore.tables") I get the following error:

<class 'ImportError'>
ImportError("/home/torrance/julia-1.7.3/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/torrance/.julia/conda/3/lib/python3.9/site-packages/casacore/tables/../../../../libcasa_tables.so.7)")

It looks like to me that when run in the context of Julia, the pyimport statement wants to preferentially link against the Julia supplied libc, and not the Conda supplied libc. Moreover, the Julia supplied libc is at version 3.4.29.

Is this choice to link against the Julia libc a deliberate choice? If so, how to we mitigate this kind of issue happening each time Conda and Julia libc versions aren't aligned?

joegilkes commented 1 year ago

I've been having the same issue, my workaround for now has been to set the LD_PRELOAD environment variable whenever I start a Julia REPL/script, ie.

LD_PRELOAD=/path/to/my_conda_env/lib/libstdc++.so.6 julia

This is a bit inconvenient since I have to always run Julia through a separate bash script whenever I want to start it for this project, and I have to do the same thing if I want to run a Julia Jupyter notebook. The correct behaviour could be set as default within a project if there was a way to save project-specific permanent environment variables (e.g. running julia --project=xyz would automatically have the correct LD_PRELOAD set for that project), but I'm not aware of a way to do this at the moment.

Naikless commented 1 year ago

Same issue for me with some scipy.optimize modules imported via PyCall from the Julia-managed conda environment.

While the above solution of preloading the more recent library works as a local fix, it is no solution if these imports are part of a published package. You can't expect every user to check for this. Since conda should already provide a consistent environment in terms of binary libs, the choice to link conda packages to Julia binaries does not seem helpful to me and I would also like to know if there are technical reasons for this behavior.

EDIT: ...but given that this isn't the first time this has happened it seems like there is not much that can be done about it. I suppose you just can't have the same library loaded twice.

stevengj commented 1 year ago

Welcome to dependency hell: the operating system does not let you load two versions of the libstdc++ library simultaneously.

As far as I know, the the only solutions here are (1) recompile Julia to use the same C++ compiler as the Python library, or (2) recompile Scipy to use the same C++ compiler as Julia, or (3) use the LD_PRELOAD workaround.