search

JuliaInterop / JuliaCall

Embed Julia in R
https://non-contradiction.github.io/JuliaCall/index.html
Other
267 stars 36 forks source link

julia_setup (still) failing on Ubuntu 16.04 (Julia 1.1.1) #110

Closed bbolker closed 5 years ago

bbolker commented 5 years ago

Attempting to follow troubleshooting instructions via LD_LIBRARY_PATH and/or R_LD_LIBRARY_PATH, but no luck so far. Any sanity checks/diagnostic tips/help welcome ...

tl;dr so far I either get a segfault from R (if doing nothing or setting LD_LIBRARY_PATH) or R fails to find its own libraries (if setting R_LD_LIBRARY_PATH)

  1. vanilla (don't set any env variables): segfault

    $ export JLIB=/usr/local/lib/julia-1.1.1/lib/julia/
$ Rscript -e "library(JuliaCall); julia_setup()"
Julia version 1.1.1 at location /usr/local/lib/julia-1.1.1/bin will be used.
Error in dyn.load(.julia$dll_file) : 
unable to load shared object '/usr/local/lib/julia-1.1.1/bin/../lib/libjulia.so.1':
/usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /usr/local/lib/julia-1.1.1/bin/../lib/julia/libLLVM-6.0.so)

*** caught segfault ***
address 0x38, cause 'memory not mapped'

  2. setting LD_LIBRARY_PATH: segfault again

$ echo $LD_LIBRARY_PATH
[empty]
$ export LD_LIBRARY_PATH=$JLIB
$ echo $LD_LIBRARY_PATH
/usr/local/lib/julia-1.1.1/lib/julia/
$ Rscript -e "library(JuliaCall); julia_setup()"
[segfault as above]

  1. set R_LD_LIBRARY_PATH: R can't start

    $ echo $R_LD_LIBRARY_PATH
[empty]
$ export R_LD_LIBRARY_PATH=$JLIB
$ echo $R_LD_LIBRARY_PATH 
/usr/local/lib/julia-1.1.1/lib/julia/
$ Rscript -e "library(JuliaCall); julia_setup()"
/usr/local/lib/R/bin/exec/R: error while loading shared libraries: libRblas.so: cannot open shared object file: No such file or directory

  2. set R_LD_LIBRARY_PATH differently: segfault

    $ export R_LD_LIBRARY_PATH=$JLIB:/usr/local/lib/R/lib/
$ Rscript -e "library(JuliaCall); julia_setup()"
[segfault again]

Julia (1.1.1) appears to work fine when started separately.

R Under development (unstable) (2019-06-28 r76752)

lsb_release -a
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.6 LTS
Release:    16.04
Codename:   xenial
Non-Contradiction commented 5 years ago

Oh, sorry. I forgot to mention that R's library itself also has to be in the R_LD_LIBRARY_PATH like this: export R_LD_LIBRARY_PATH="$(R RHOME)/lib:$JLIB" The travis CI setting can be found at: https://github.com/Non-Contradiction/JuliaCall/blob/master/.travis.yml

And I will update in the README to make things more clear if the solution works.

bbolker commented 5 years ago

Got farther but still segfaulted later on (maybe I'm running out of memory? This is inside a VirtualBox ...) Any ideas?

Julia version 1.1.1 at location /usr/local/lib/julia-1.1.1/bin will be used.
Loading setup script for JuliaCall...

signal (11): Segmentation fault
in expression starting at /usr/local/lib/R/site-library/JuliaCall/julia/setup.jl:72
GetNewPage at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/memory.c:913
Rf_allocVector3 at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/memory.c:2515
Rf_allocVector at /home/bolker/R/r-build/src/main/../../../r-devel/src/include/Rinlinedfuns.h:531 [inlined]
Rf_ScalarInteger at /home/bolker/R/r-build/src/main/../../../r-devel/src/include/Rinlinedfuns.h:938
__init__ at /home/bolker/.julia/packages/RCall/iojZI/src/setup.jl:162
unknown function (ip: 0x7f1a9dc02171)
jl_fptr_trampoline at /buildworker/worker/package_linux64/build/src/gf.c:1842
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2197
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1571 [inlined]
jl_module_run_initializer at /buildworker/worker/package_linux64/build/src/toplevel.c:72
jl_init_restored_modules at /buildworker/worker/package_linux64/build/src/dump.c:2418
_include_from_serialized at ./loading.jl:633
_require_from_serialized at ./loading.jl:684
_require at ./loading.jl:967
require at ./loading.jl:858
require at ./loading.jl:853
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2197
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1571 [inlined]
call_require at /buildworker/worker/package_linux64/build/src/toplevel.c:397 [inlined]
eval_import_path at /buildworker/worker/package_linux64/build/src/toplevel.c:432
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:630
jl_eval_module_expr at /buildworker/worker/package_linux64/build/src/toplevel.c:179
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:614
jl_parse_eval_all at /buildworker/worker/package_linux64/build/src/ast.c:883
jl_load at /buildworker/worker/package_linux64/build/src/toplevel.c:826
include at ./boot.jl:326 [inlined]
include_relative at ./loading.jl:1038
include at ./sysimg.jl:29
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2197
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:323
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:411
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:362 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:773
jl_interpret_toplevel_thunk_callback at /buildworker/worker/package_linux64/build/src/interpreter.c:885
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7f1aa7bcf43f)
unknown function (ip: (nil))
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:894
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:764
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:793
jl_eval_string at /buildworker/worker/package_linux64/build/src/jlapi.c:94
juliacall_cmd at /tmp/RtmpH5iMAM/R.INSTALL206772867a5b/JuliaCall/src/JuliaCall.cpp:31
_JuliaCall_juliacall_cmd at /tmp/RtmpH5iMAM/R.INSTALL206772867a5b/JuliaCall/src/RcppExports.cpp:26
do_dotcall at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/dotcode.c:1252
bcEval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:6775
Rf_eval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:620
R_execClosure at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:1780
Rf_applyClosure at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:1706
bcEval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:6743
Rf_eval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:620
R_execClosure at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:1780
Rf_applyClosure at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:1706
bcEval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:6743
Rf_eval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:620
R_execClosure at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:1780
Rf_applyClosure at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:1706
Rf_eval at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/eval.c:743
Rf_ReplIteration at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/main.c:260
R_ReplConsole at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/main.c:310
run_Rmainloop at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/main.c:1108
main at /home/bolker/R/r-build/src/main/../../../r-devel/src/main/Rmain.c:29
__libc_start_main at /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
_start at /usr/local/lib/R/bin/exec/R (unknown line)
Allocations: 4402303 (Pool: 4401880; Big: 423); GC: 6
Segmentation fault (core dumped)
Non-Contradiction commented 5 years ago

It seems the segfault is from RCall.jl, which is a Julia package dependency for JuliaCall. Maybe RCall.jl is broken for R development.

Could you please try: using RCall; R"1+1"; in Julia to see whether RCall itself works?

bbolker commented 5 years ago

indeed, in a fresh shell/fresh Julia session:

julia> using RCall
ERROR: InitError: error compiling __init__: error compiling initEmbeddedR: could not load library "/usr/local/lib/R/lib/libR.so"
libRblas.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:633
 [2] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:713
 [3] _require(::Base.PkgId) at ./loading.jl:937
 [4] require(::Base.PkgId) at ./loading.jl:858
 [5] require(::Module, ::Symbol) at ./loading.jl:853
during initialization of module RCall

I'll try making sure my R-devel is completely up to date ... where/to whom should I report this if it persists?

Now trying to re-build RCall, descending into some kind of configuration hell ...

$ export R_HOME=/usr/local/lib/R
$ julia
               _  | | |_| | | | (_| |  |  Version 1.1.1 (2019-05-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
julia> using Pkg
julia> Pkg.build("RCall")
  Building Conda → `~/.julia/packages/Conda/kLXeC/deps/build.log`
  Building RCall → `~/.julia/packages/RCall/iojZI/deps/build.log`
┌ Error: Error building `RCall`: 
│ ERROR: LoadError: R cannot be found. Set the "R_HOME" environment variable to re-run Pkg.build("RCall").
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] top-level scope at /home/bolker/.julia/packages/RCall/iojZI/deps/build.jl:49
│  [3] include at ./boot.jl:326 [inlined]
│  [4] include_relative(::Module, ::String) at ./loading.jl:1038
│  [5] include(::Module, ::String) at ./sysimg.jl:29
│  [6] include(::String) at ./client.jl:403
│  [7] top-level scope at none:0
│ in expression starting at /home/bolker/.julia/packages/RCall/iojZI/deps/build.jl:10
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Pkg/src/Operations.jl:1075

??? also tried ENV["R_HOME"]="/usr/local/lib/R" within Julia, same results ...

PS I know this isn't your problem any more, if you can suggest where I can post this that would be great ... (sadly, this kind of experience is pretty typical of my infrequent attempts to get up to speed with Julia ...)

Non-Contradiction commented 5 years ago

The RCall error seems moer simmilar to this: https://github.com/JuliaInterop/RCall.jl/issues/277, so the surface problem is that Julia (and RCall.jl) cannot find R's library just as R cannot find Julia's library.

Does things like export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:R RHOME/lib" work before using RCall?

But I'm afraid that after solving this problem, there will be some deeper problem for RCall with R dev. And if the deeper problem does exist, you can post it on RCall's issue track.

bbolker commented 5 years ago

nope, get the same "R cannot be found" error. I'll continue this on the JuliaInterop/RCall issues list. Thanks!