Open sethaxen opened 2 years ago
This is the underlying error:
ImportError("/home/sethaxen/.julia/juliaup/julia-1.7.3+0~x64/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/sethaxen/.julia/conda/3/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so)")
i.e. it is an incompatibility with the C++ standard library — Julia links one version, pypocketfft
wants another.
Not much we can do about this in PyCall. You'll need either a different version of Julia or a different version of pypocketfft, built with compatible versions of libstdc++
.
Ah okay, so this has the same solution as https://github.com/JuliaPy/PyCall.jl/issues/722#issuecomment-575982875, namely
$ LD_PRELOAD=$HOME/.julia/conda/3/lib/libstdc++.so julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.3 (2022-05-06)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using PyCall
julia> pyimport("scipy.fft")
PyObject <module 'scipy.fft' from '/home/sethaxen/.julia/conda/3/lib/python3.9/site-packages/scipy/fft/__init__.py'>
Is this the best advice I can give to users of my package? It seems that this incompatibility of Julia's libstdc++ with some Python package's can happen at any time. In this case, a new Julia version has even been released (v1.8.0-rc1) since scipy v1.8.1 was released, and has the same error.
@sethaxen Thank you for your workaround, this fixes (for now) my setup (both with Julia 1.7 and 1.8), I had a conda package "kneed" that dependend on scipy.fft, with the above error. I can confirm that the PRELOAD trick works.
I can see a difference between MS Windows and Linux, on Windows, I do not observe an issue.
Here my code snippet that fails:
using PyCall
py"""
from scipy import signal
"""
and here the error output:
ERROR: PyError ($(Expr(:escape, :(ccall(#= /home/stefan/.julia/packages/PyCall/ygXW2/src/pyeval.jl:38 =# @pysym(:PyEval_EvalCode), PyPtr, (PyPtr, PyPtr, PyPtr), o, globals, locals))))) <class 'ModuleNotFoundError'>
ModuleNotFoundError("No module named 'scipy'")
File "/home/stefan/.julia/packages/PyCall/ygXW2/src/pyeval.jl", line 1, in <module>
const Py_single_input = 256 # from Python.h
Stacktrace:
[1] pyerr_check
@ ~/.julia/packages/PyCall/ygXW2/src/exception.jl:62 [inlined]
[2] pyerr_check
@ ~/.julia/packages/PyCall/ygXW2/src/exception.jl:66 [inlined]
[3] _handle_error(msg::String)
@ PyCall ~/.julia/packages/PyCall/ygXW2/src/exception.jl:83
[4] macro expansion
@ ~/.julia/packages/PyCall/ygXW2/src/exception.jl:97 [inlined]
[5] #117
@ ~/.julia/packages/PyCall/ygXW2/src/pyeval.jl:38 [inlined]
[6] disable_sigint
@ ./c.jl:458 [inlined]
[7] pyeval_(s::String, globals::PyDict{String, PyObject, true}, locals::PyDict{String, PyObject, true}, input_type::Int64, fname::String)
@ PyCall ~/.julia/packages/PyCall/ygXW2/src/pyeval.jl:37
[8] top-level scope
@ ~/.julia/packages/PyCall/ygXW2/src/pyeval.jl:230
and if I issue the last script the output is:
include("/home/stefan/.julia/packages/PyCall/ygXW2/src/pyeval.jl")
ERROR: LoadError: UndefVarError: @pycheckn not defined
in expression starting at /home/stefan/.julia/packages/PyCall/ygXW2/src/pyeval.jl:10
@StefanPofahl
This works for me
using PyCall; p=pyimport("scipy.signal")
Note that to get around the fft GLIBC issue I use:
using PyCall
using Conda
Conda.add("gcc=12.1.0"; channel="conda-forge")
Conda.add("scikit-image")
# Pin this version, to avoid clashes with libgcc.34
Conda.add("scipy=1.8.0")
@StefanPofahl for completeness, in my packages where I use PyCall/scipy, I have ./deps/build.jl
with
#./deps/build.jl
using Pkg;
using Logging;
@info "Initiating build"
ENV["PYTHON"] = ""
Pkg.add("Conda")
Pkg.add("PyCall")
Pkg.build("PyCall")
Pkg.build("Conda")
using PyCall
using Conda
## Add the two packages we need
Conda.add("gcc=12.1.0"; channel="conda-forge")
Conda.add("scikit-image")
# Pin this version, to avoid clashes with libgcc.34
Conda.add("scipy=1.8.0")
Note that this is failing due to an unrelated issue with Conda on MacOs (I'm told the latest Mac updates fix that), other than that this works for me across systems.
Hi bencardoen this worked fine, thanks :-) Regards, Stefan
glad it was of use :-) @StefanPofahl the build.jl approach also (for me) means that the pyimport is precompiled, so when it's actually called, e.g. in a tight loop, there's no latency on first call. Though in general, using PackageCompiler.jl allows for even faster precompilation in my experience.
Hi @bencardoen , I will have a look on PackageCompiler.jl. Thanks!
If we install scipy v1.8.1 in the Julia-specific conda environment and call
pyimport("scipy.fft")
, we get the following error:If we manually launch the Julia-specific conda's Python executable and call
import scipy.fft
, it loads with no problem. If we instead install scipy v1.8.0, thenpyimport("scipy.fft")
also runs with no problem.