Initialization of Julia libraries fails in RStudio

[jeremiedb]

Calling a Julia library fails on first 2 attempts in RStudio, with different error message at each attempt, before successfully working at 3rd attempt. Using R 4.03 and latest RStudio: Version 1.3.1093. The error doesn't occur with all Julia libraries. For example, DataFrames works fine, but Flux gets an error on first call and needs a second call to julia_library before loading successfully.

R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"

> JuliaCall::julia_setup()
Julia version 1.5.0 at location /home/jeremie/julia-1.5.0/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
> JuliaCall::julia_library("EvoTrees")
Error: Error happens in Julia.
InitError: Multiple LLVM libraries loaded by Julia.
Please file an issue and attach the output of `Libdl.dllist()`.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] __init__() at /home/jeremie/.julia/packages/LLVM/T8ZBA/src/LLVM.jl:70
 [3] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:697
 [4] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:782
 [5] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:712
 [6] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:771
 [7] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:712
 [8] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:771
 [9] _require(::Base.PkgId) at ./loading.jl:1007
 [10] require(::Base.PkgId) at ./loading.jl:928
 [11] require(::Module, ::Symbol) at ./loading.jl:923
 [12] eval(::Module, ::Any) at ./boot.jl:331
 [13] eval_string(::String) at /home/j
> JuliaCall::julia_library("EvoTrees")
Error: Error happens in Julia.
InitError: UndefRefError: access to undefined reference
Stacktrace:
 [1] getproperty at ./Base.jl:33 [inlined]
 [2] getindex at ./refvalue.jl:32 [inlined]
 [3] (::LLVM.API.var"#3493#cache_fptr!#1011")() at /home/jeremie/.julia/packages/LLVM/T8ZBA/src/util.jl:98
 [4] macro expansion at /home/jeremie/.julia/packages/LLVM/T8ZBA/src/util.jl:106 [inlined]
 [5] LLVMParseCommandLineOptions(::Int64, ::Array{String,1}, ::Ptr{Nothing}) at /home/jeremie/.julia/packages/LLVM/T8ZBA/lib/libLLVM_h.jl:4128
 [6] clopts(::String) at /home/jeremie/.julia/packages/LLVM/T8ZBA/src/support.jl:5
 [7] __init__() at /home/jeremie/.julia/packages/CUDA/dZvbp/src/initialization.jl:68
 [8] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:697
 [9] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:782
 [10] _tryrequire_from_serialized(::Base.PkgId, ::UInt64, ::String) at ./loading.jl:712
 [11] _require_search_from_serialized(::Base.PkgId, ::Strin
> JuliaCall::julia_library("EvoTrees")

It however go fine if making the same directly in R console:

> library(JuliaCall)
> julia_setup()
Julia version 1.5.0 at location /home/jeremie/julia-1.5.0/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
> library(EvoTrees)
> 

[Non-Contradiction]

Thanks for the feedback!

I just do a quick search and find that CUDA.jl is a dependency for both EvoTrees.jl and Flux.jl, which depends on LLVM.jl. And the error seems to be from https://github.com/maleadt/LLVM.jl/blob/b0780fa710219b1ca2e0a418d5dc82fa6dfafe71/src/LLVM.jl#L72-L76 So I think the cause of the problem is that during the initialization of LLVM.jl, it finds multiple LLVM library, so the initialization process is interrupted. The second and the third try will try to continue the initiation process, but since LLVM.jl is not initialized properly, some weird things will happen.

So I guess the problem is that RStudio (or something else) somehow loads another LLVM library, which is the cause of the confliction. Since the error seems to be related to LLVM.jl and CUDA.jl, functionality without CUDA may still work.

To investigate the problem deeper,

1. Are things running in a clean R session of RStudio? Or are there anything else that could be in conflict with CUDA or LLVM?
2. What is the result of

   julia_library("Libdl")
   julia_command("Libdl.dllist()")

   as suggested by the first error message?

3. Maybe it is also helpful to open an issue at https://github.com/maleadt/LLVM.jl and link the issue to this one.

[jeremiedb]

Thanks for quick feedback! I think you nailed it with about the multiple LLVM lbraries.

From the R console, only one LLVM library found:

dllist <- julia_eval("Libdl.dllist()")
grep("LLVM", dllist, value=T)
[1] "/home/jeremie/julia-1.5.0/bin/../lib/julia/libLLVM-9jl.so"

While the same call through Rstudio returns 2:

[1] "/usr/lib/x86_64-linux-gnu/libLLVM-10.so.1"                 "/home/jeremie/julia-1.5.0/bin/../lib/julia/libLLVM-9jl.so"

These 2 libraries are returned on a Ubuntu machine. On Windows (also with a CUDA.jl install), the error doesn't occur and only 1 LLVM library is found from RStudio.

Do you think that having this 2nd LLVM library being linked by RStudio in Ubuntu (but not in windows) could be an issue on the RStudio side? Otherwise, I'd assume that it should be to the LLVM.jl library to handle the case where 2 LLVMs are found.

[Non-Contradiction]

Solved in maleadt/LLVM.jl#209