JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.45k stars 186 forks source link

Wrong libpython configuration after changing python dir on Linux #993

Open sairus7 opened 2 years ago

sairus7 commented 2 years ago

I want to make a portable app with PyCall included, so I move conda into a common folder (this folder I would later copy to another machine when deploying compiled app binaries):

sudo cp -rf ~/.julia/conda /opt/conda

Then I rebuild PyCall and make sure all conda and python-related paths (there are much of them!) point to the right place:

julia -e 'import Pkg; 
ENV["CONDA_JL_HOME"] = "/opt/conda/3"; 
ENV["PYTHON"] = "/opt/conda/3/bin/python3.9"; 
Pkg.add("PyCall"); 
Pkg.build("PyCall");

using PyCall
@info(
           "Conda.jl and PyCall.jl configuration after build, BUT wrong libpython!",
           PyCall.Conda.ROOTENV,
           PyCall.Conda.MINICONDA_VERSION,
           PyCall.Conda.deps_file,
           PyCall.Conda.PREFIX,
           PyCall.Conda.BINDIR,
           PyCall.Conda.LIBDIR,
           PyCall.Conda.SCRIPTDIR,
           PyCall.Conda.PYTHONDIR,
           PyCall.Conda.conda,
           PyCall.Conda.CONDARC,
           PyCall.PYTHONHOME,
           PyCall.libpython
       )
'

Which gives me all paths set right but PyCall.libpython, which prevents me from running my compiled app on another machine: (also, I'm not sure about colons in PyCall.PYTHONHOME)

┌ Info: Conda.jl and PyCall.jl configuration after build, BUT wrong libpython!
│   PyCall.Conda.ROOTENV = "/opt/conda/3"
│   PyCall.Conda.MINICONDA_VERSION = "3"
│   PyCall.Conda.deps_file = "/home/root2/.julia/packages/Conda/x2UxR/src/../deps/deps.jl"
│   PyCall.Conda.PREFIX = "/opt/conda/3"
│   PyCall.Conda.BINDIR = "/opt/conda/3/bin"
│   PyCall.Conda.LIBDIR = "/opt/conda/3/lib"
│   PyCall.Conda.SCRIPTDIR = "/opt/conda/3/bin"
│   PyCall.Conda.PYTHONDIR = "/opt/conda/3/bin"
│   PyCall.Conda.conda = "/opt/conda/3/bin/conda"
│   PyCall.Conda.CONDARC = "/opt/conda/3/condarc-julia.yml"
│   PyCall.PYTHONHOME = "/opt/conda/3:/opt/conda/3"
└   PyCall.libpython = "/home/root2/.julia/conda/3/lib/libpython3.9.so.1.0"

Contents of deps.jl:

$ cat /home/root2/.julia/packages/Conda/x2UxR/src/../deps/deps.jl
const ROOTENV = "/opt/conda/3"
const MINICONDA_VERSION = "3"
const USE_MINIFORGE = true
const CONDA_EXE = "/opt/conda/3/bin/conda"
YuliyaCl commented 2 years ago

Additional thoughts about the issue The error occurred during running the compiled app on the another machine and links to the line :

libpy_handle = libpython === nothing ? C_NULL :
    Libdl.dlopen(libpython, Libdl.RTLD_LAZY|Libdl.RTLD_DEEPBIND|Libdl.RTLD_GLOBAL)

Also it seems that PyCall.libpython value defines here:

    append_truthy(lib_dirs, sysconfig.get_config_var('LIBPL'))
    append_truthy(lib_dirs, sysconfig.get_config_var('srcdir'))
    append_truthy(lib_dirs, sysconfig.get_config_var("LIBDIR"))

But I have no idea how to change it to the correct path.

PS. On Windows OS everything works correctly using the same approach.