search

JuliaPy / pyjulia

python interface to julia
MIT License
884 stars 101 forks source link

Can't Run pyjulia in Ubuntu 20.04 #384

Open pmaldona opened 4 years ago

pmaldona commented 4 years ago

Hello, firts thanks for the efort to bing this on. I have follow the installation instruction of the page (https://pyjulia.readthedocs.io/en/latest/installation.html) but whe i test the program I get the fowwing error:

Python 3.8.2 (default, Apr 27 2020, 15:53:34) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. '>>>' from julia import Base Traceback (most recent call last): File "", line 1, in File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 655, in _load_unlocked File "", line 618, in _load_backward_compatible File "/home/pmaldona/.local/lib/python3.8/site-packages/julia/core.py", line 247, in load_module elif self.julia.isafunction(juliapath): File "/home/pmaldona/.local/lib/python3.8/site-packages/julia/core.py", line 238, in julia self.class.julia = julia = Julia() File "/home/pmaldona/.local/lib/python3.8/site-packages/julia/core.py", line 467, in init jlinfo = JuliaInfo.load(runtime) File "/home/pmaldona/.local/lib/python3.8/site-packages/julia/juliainfo.py", line 87, in load raise subprocess.CalledProcessError( subprocess.CalledProcessError: Command '['julia', '-e', '...']' returned non-zero exit status 1.

Thanks in advance for the help.

Greetings

samirop commented 4 years ago

+1

rideckard commented 4 years ago

+1

This is probably a bad recommendation!!, but the problem seems to be with the dynamic linking to libjulia.so. So for a workaround sudo ln /usr/lib/x86_64-linux-gnu/libjulia.so.1 /usr/lib/x86_64-linux-gnu/libjulia.so sudo ldconfig

Then it works for me. At least the example

from julia.api import Julia jl = Julia(compiled_modules=False) from julia import Base Base.sind(90)

Maybe somebody can work out a more solid solution

akabla commented 4 years ago

+1 I'm having the same problem. I'm also running Ubuntu 20.04.

casv2 commented 4 years ago

+1 having the same problem

tkf commented 4 years ago

The output of

$ python3
>>> import julia
>>> julia.install()

would be useful for debugging this.

cycomanic commented 4 years ago

I have the same error on opensuse tumbleweed. Below is the output from julia.install()

Python 3.8.3 (default, May 17 2020, 14:48:56) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import julia
>>> julia.install()
[ Info: Julia version info
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-suse-linux)
  uname: Linux 5.7.1-1-default #1 SMP Wed Jun 10 11:53:46 UTC 2020 (6a549f6) x86_64 x86_64
  CPU: AMD Ryzen 5 3600 6-Core Processor: 
                 speed         user         nice          sys         idle          irq
       #1-12  4015 MHz     212773 s        146 s      37812 s   14028861 s          0 s

  Memory: 31.297752380371094 GB (23734.51171875 MB free)
  Uptime: 11943.0 sec
  Load Avg:  0.54150390625  0.63623046875  0.7646484375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, znver1)
Environment:
  AF_PATH = /home/jschrod/Downloads/Scientific/OpenCL/arrayfire-3
  HOME = /home/jschrod
  MANPATH = /home/jschrod/.local/share/man:/usr/share/man:/usr/local/share/man
  OMF_PATH = /home/jschrod/.local/share/omf
  PATH = /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/jschrod/.local/bin:/home/jschrod/.emacs.d/bin:/home/jschrod/.local/bin:/home/jschrod/.emacs.d/bin
  TERM = xterm-kitty
[ Info: Julia executable: /usr/bin/julia
[ Info: Trying to import PyCall...
┌ Error: `import PyCall` failed
│   exception =
│    ArgumentError: Package PyCall not found in current path:
│    - Run `import Pkg; Pkg.add("PyCall")` to install the PyCall package.
│    
│    Stacktrace:
│     [1] require(::Module, ::Symbol) at ./loading.jl:887
│     [2] top-level scope at /home/jschrod/.local/lib/python3.8/site-packages/julia/install.jl:32
│     [3] include at ./boot.jl:328 [inlined]
│     [4] include_relative(::Module, ::String) at ./loading.jl:1105
│     [5] include(::Module, ::String) at ./Base.jl:31
│     [6] exec_options(::Base.JLOptions) at ./client.jl:287
│     [7] _start() at ./client.jl:460
└ @ Main ~/.local/lib/python3.8/site-packages/julia/install.jl:34
[ Info: Installing PyCall...
   Cloning default registries into `~/.julia`
   Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: LoadError: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Stacktrace:
 [1] pkgerror(::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:113
 [2] #clone#4(::String, ::Base.Iterators.Pairs{Symbol,LibGit2.CachedCredentials,Tuple{Symbol},NamedTuple{(:credentials,),Tuple{LibGit2.CachedCredentials}}}, ::typeof(Pkg.GitTools.clone), ::String, ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/GitTools.jl:131
 [3] #clone at ./none:0 [inlined]
 [4] (::Pkg.Types.var"#124#126"{Pkg.Types.RegistrySpec,String})(::LibGit2.CachedCredentials) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1029
 [5] shred!(::Pkg.Types.var"#124#126"{Pkg.Types.RegistrySpec,String}, ::LibGit2.CachedCredentials) at ./secretbuffer.jl:184
 [6] clone_or_cp_registries(::Pkg.Types.Context, ::Array{Pkg.Types.RegistrySpec,1}, ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1028
 [7] clone_or_cp_registries at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1013 [inlined] (repeats 2 times)
 [8] clone_default_registries() at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:963
 [9] find_registered!(::Pkg.Types.EnvCache, ::Array{String,1}, ::Array{Base.UUID,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1244
 [10] registry_resolve!(::Pkg.Types.EnvCache, ::Array{Pkg.Types.PackageSpec,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:880
 [11] #add#25(::Bool, ::Pkg.BinaryPlatforms.Linux, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:95
 [12] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:72
 [13] #add#24 at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:69 [inlined]
 [14] add at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:69 [inlined]
 [15] #add#21 at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:67 [inlined]
 [16] add at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:67 [inlined]
 [17] #add#20(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:66
 [18] add(::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:66
 [19] top-level scope at /home/jschrod/.local/lib/python3.8/site-packages/julia/install.jl:102
 [20] include at ./boot.jl:328 [inlined]
 [21] include_relative(::Module, ::String) at ./loading.jl:1105
 [22] include(::Module, ::String) at ./Base.jl:31
 [23] exec_options(::Base.JLOptions) at ./client.jl:287
 [24] _start() at ./client.jl:460
in expression starting at /home/jschrod/.local/lib/python3.8/site-packages/julia/install.jl:57
caused by [exception 1]
GitError(Code:ERROR, Class:Invalid, invalid version 0 on git_proxy_options)
Stacktrace:
 [1] macro expansion at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/LibGit2/src/error.jl:101 [inlined]
 [2] clone(::String, ::String, ::LibGit2.CloneOptions) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/LibGit2/src/repository.jl:459
 [3] #clone#131(::String, ::Bool, ::Ptr{Nothing}, ::LibGit2.CachedCredentials, ::Dict{Symbol,Tuple{Ptr{Nothing},Any}}, ::typeof(LibGit2.clone), ::String, ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/LibGit2/src/LibGit2.jl:580
 [4] #clone at ./none:0 [inlined]
 [5] #clone#4(::String, ::Base.Iterators.Pairs{Symbol,LibGit2.CachedCredentials,Tuple{Symbol},NamedTuple{(:credentials,),Tuple{LibGit2.CachedCredentials}}}, ::typeof(Pkg.GitTools.clone), ::String, ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/GitTools.jl:123
 [6] #clone at ./none:0 [inlined]
 [7] (::Pkg.Types.var"#124#126"{Pkg.Types.RegistrySpec,String})(::LibGit2.CachedCredentials) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1029
 [8] shred!(::Pkg.Types.var"#124#126"{Pkg.Types.RegistrySpec,String}, ::LibGit2.CachedCredentials) at ./secretbuffer.jl:184
 [9] clone_or_cp_registries(::Pkg.Types.Context, ::Array{Pkg.Types.RegistrySpec,1}, ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1028
 [10] clone_or_cp_registries at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1013 [inlined] (repeats 2 times)
 [11] clone_default_registries() at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:963
 [12] find_registered!(::Pkg.Types.EnvCache, ::Array{String,1}, ::Array{Base.UUID,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1244
 [13] registry_resolve!(::Pkg.Types.EnvCache, ::Array{Pkg.Types.PackageSpec,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:880
 [14] #add#25(::Bool, ::Pkg.BinaryPlatforms.Linux, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:95
 [15] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:72
 [16] #add#24 at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:69 [inlined]
 [17] add at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:69 [inlined]
 [18] #add#21 at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:67 [inlined]
 [19] add at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:67 [inlined]
 [20] #add#20(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:66
 [21] add(::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:66
 [22] top-level scope at /home/jschrod/.local/lib/python3.8/site-packages/julia/install.jl:102
 [23] include at ./boot.jl:328 [inlined]
 [24] include_relative(::Module, ::String) at ./loading.jl:1105
 [25] include(::Module, ::String) at ./Base.jl:31
 [26] exec_options(::Base.JLOptions) at ./client.jl:287
 [27] _start() at ./client.jl:460
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jschrod/.local/lib/python3.8/site-packages/julia/tools.py", line 102, in install
    raise PyCallInstallError("Installing", output)
julia.tools.PyCallInstallError: Installing PyCall failed.

** Important information from Julia may be printed before Python's Traceback **

Some useful information may also be stored in the build log file
`~/.julia/packages/PyCall/*/deps/build.log`.

So looks like it fails to install pycall. I get the same error trying to install in the julia REPL.

Some further digging shows that this seems to be an error with julia/or packaging of julia linking to the wrong libgit. See the julia bug here: https://github.com/JuliaLang/julia/issues/33111#issuecomment-540539851 which also provides a workaround (clone manually into ~/.julia/registries/General)

tkf commented 4 years ago

If you have Julia 1.4 or above, it looks like you can set the environment variable export JULIA_PKG_SERVER=pkg.julialang.org (in your system shell) to work around this issue https://github.com/JuliaLang/julia/issues/33111#issuecomment-612638509

cycomanic commented 4 years ago

I just fixed by manually cloning the General registry and now I can install PyCall, however I still get the same error:

[ins] In [1]: from julia.api import Julia 
         ...: jl = Julia(compiled_modules=False) 
         ...: from julia import Base 
         ...: Base.sind(90)                                                                                                                                   
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
<ipython-input-1-135f1876e7df> in <module>
      1 from julia.api import Julia
----> 2 jl = Julia(compiled_modules=False)
      3 from julia import Base
      4 Base.sind(90)

~/.local/lib/python3.8/site-packages/julia/core.py in __init__(self, init_julia, jl_init_path, runtime, jl_runtime_path, debug, **julia_options)
    465             self.api = get_libjulia()
    466         elif init_julia:
--> 467             jlinfo = JuliaInfo.load(runtime)
    468             if jlinfo.version_info < (0, 7):
    469                 raise RuntimeError("PyJulia does not support Julia < 0.7 anymore")

~/.local/lib/python3.8/site-packages/julia/juliainfo.py in load(cls, julia, **popen_kwargs)
     85                 )
     86             else:
---> 87                 raise subprocess.CalledProcessError(
     88                     retcode, [julia, "-e", "..."], stdout, stderr
     89                 )

CalledProcessError: Command '['julia', '-e', '...']' returned non-zero exit status 1.

Output of julia.install()

[ins] In [1]: import julia                                                                                                                                    

[ins] In [2]: julia.install()                                                                                                                                 
[ Info: Julia version info
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Linux (x86_64-suse-linux)
  uname: Linux 5.7.1-1-default #1 SMP Wed Jun 10 11:53:46 UTC 2020 (6a549f6) x86_64 x86_64
  CPU: AMD Ryzen 5 3600 6-Core Processor: 
                 speed         user         nice          sys         idle          irq
       #1-12  2198 MHz     245649 s        146 s      44275 s   16642154 s          0 s

  Memory: 31.297752380371094 GB (22802.3359375 MB free)
  Uptime: 14158.0 sec
  Load Avg:  0.1298828125  0.35595703125  0.453125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, znver1)
Environment:
  AF_PATH = /home/jschrod/Downloads/Scientific/OpenCL/arrayfire-3
  HOME = /home/jschrod
  MANPATH = /home/jschrod/.local/share/man:/usr/share/man:/usr/local/share/man
  OMF_PATH = /home/jschrod/.local/share/omf
  PATH = /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/jschrod/.local/bin:/home/jschrod/.emacs.d/bin:/home/jschrod/.local/bin:/home/jschrod/.emacs.d/bin
  TERM = xterm-kitty
[ Info: Julia executable: /usr/bin/julia
[ Info: Trying to import PyCall...
┌ Info: PyCall is already installed and compatible with Python executable.
│ 
│ PyCall:
│     python: python3
│     libpython: /usr/lib64/libpython3.8.so.1.0
│ Python:
│     python: /usr/bin/python3
└     libpython: /usr/lib64/libpython3.8.so.1.0

so something else is going on. Might be a linking error as said above.

tkf commented 4 years ago

The output of julia --startup-file=no ~/.local/lib/python3.8/site-packages/julia/juliainfo.jl might be helpful.

cycomanic commented 4 years ago

I think I found the issue. It seems under opensuse at least the simlink from libjulia.so to libjulia.so.1 was not created. which caused the failure in dlpath. Creating a simlink manually fixes the issue. So this does not look like a pyjulia issue but more like a packaging issue (interesting that it seems to happen on opensuse and ubuntu). For the record below is the output of julia --startup-file=no ~/.local/lib/python3.8/site-packages/julia/juliainfo.jl

1.3.1
1
3
1
/usr/bin
ERROR: LoadError: could not load library "libjulia"
libjulia.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:109
 [2] dlopen at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)
 [3] dlpath(::String) at /home/abuild/rpmbuild/BUILD/julia-1.3.1/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:232
 [4] top-level scope at /home/jschrod/.local/lib/python3.8/site-packages/julia/juliainfo.jl:14
 [5] include at ./boot.jl:328 [inlined]
 [6] include_relative(::Module, ::String) at ./loading.jl:1105
 [7] include(::Module, ::String) at ./Base.jl:31
 [8] exec_options(::Base.JLOptions) at ./client.jl:287
 [9] _start() at ./client.jl:460
in expression starting at /home/jschrod/.local/lib/python3.8/site-packages/julia/juliainfo.jl:14
tkf commented 4 years ago

So this does not look like a pyjulia issue but more like a packaging issue

Well, PyJulia should at least throw a better error!

(I wonder why CalledProcessError does not show stderr on failure. Anyway, it's probably better to change it to a PyJulia-specific exception type.)