ERROR: LoadError: RCall not properly installed. Please run Pkg.build("RCall")

[samd1993]

Hi Neal,

Unfortunately I have run into another issue that I cannot find a solution for. I am trying to run the simulation but am receiving this error about RCall not being installed. I've double checked that R works on the system I am using, and I've tried installing RCall which hasn't helped. Below is the output:

samd1993@DESKTOP-V6PBDGC:~/mimix$ julia scripts/sim-mcmc.jl \
>     --data simulation-study/configs/data.yml \
>     --hyper simulation-study/configs/hyper.yml \
>     --monitor simulation-study/configs/monitor.yml \
>     --inits simulation-study/configs/inits.yml \
>     --seed 123 \
>     --factors 20 \
>     simulation-results
ERROR: LoadError: RCall not properly installed. Please run Pkg.build("RCall")
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] top-level scope at /home/samd1993/.julia/packages/RCall/Qzssx/src/RCall.jl:36
 [3] include(::Module, ::String) at ./Base.jl:377
 [4] top-level scope at none:2
 [5] eval at ./boot.jl:331 [inlined]
 [6] eval(::Expr) at ./client.jl:449
 [7] top-level scope at ./none:3
in expression starting at /home/samd1993/.julia/packages/RCall/Qzssx/src/RCall.jl:33
ERROR: LoadError: Failed to precompile RCall [6f49c342-dc21-5d91-9882-a32aef131414] to /home/samd1993/.julia/compiled/v1.4/RCall/8GFyb_iiRov.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /home/samd1993/mimix/MicrobiomeMixedModels.jl/src/MicrobiomeMixedModels.jl:7
ERROR: LoadError: Failed to precompile MicrobiomeMixedModels [0930ccc7-16db-55b2-bbc5-5e6d0ae52009] to /home/samd1993/.julia/compiled/v1.4/MicrobiomeMixedModels/hXJHx_iiRov.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] exec_options(::Base.JLOptions) at ./client.jl:288
 [8] _start() at ./client.jl:484
in expression starting at /home/samd1993/mimix/scripts/sim-mcmc.jl:5

Any ideas? I've also asked the people at RCall's github as well. Cheers, Sam

[samd1993]

I also tried running this on my labs server to see if a linux friendly system would help and got this error when running everything:

julia (base) [sdegregori@surfingmantis mimix]$ julia scripts/sim-mcmc.jl \
>     --data simulation-study/configs/data.yml \
>     --hyper simulation-study/configs/hyper.yml \
>     --monitor simulation-study/configs/monitor.yml \
>     --inits simulation-study/configs/inits.yml \
>     --seed 123 \
>     --factors 20 \
>     simulation-results
┌ Warning: `@add_arg_table` is deprecated, use `@add_arg_table!` instead
└ @ ArgParse :-1
ERROR: LoadError: StackOverflowError:
Stacktrace:
 [1] MvNormal(::Array{Float64,2}) at /data/home/sdegregori/.julia/packages/Mamba/PkMTm/src/distributions/constructors.jl:29 (repeats 
79984 times)
in expression starting at /data/home/sdegregori/mimix/scripts/sim-mcmc.jl:164

[nsgrantham]

Hi Samuel,

There are a few things going on here:

1. I have no idea how well this code runs on Windows, or if it runs at all. I've only run it on macOS and Ubuntu. Furthermore, the code was written to run with Julia 1.0. The latest version of Julia is 1.4, and I'm hopeful everything still works, but it may be best to downgrade to 1.0.

2. For your first error (on your Windows machine), did you try to run Pkg.build("RCall")?

3. For your second error, I can't quite tell where the error is coming from. Warning:@add_arg_tableis deprecated, use@add_arg_table!instead is just a warning, not an error, but the StackOverflowError isn't descriptive enough for me to diagnose what went wrong.

4. Instead of running the simulation study, which depends on R, can you try running the analysis script, which doesn't depend on R, and see if that works? It's the following command:

   mkdir nutnet-analysis-results
   julia scripts/fit-mcmc.jl \
   --hyper nutnet-analysis/configs/hyper.yml \
   --monitor nutnet-analysis/configs/monitor-mimix.yml \
   --inits nutnet-analysis/configs/inits.yml \
   --factors 20 \
   nutnet-analysis/reduced-data \
   nutnet-analysis-results

[samd1993]

Hi Neal,

I'll try the older version of Julia and report back. I was able to install RCall with add RCall but not with Pkg.build.

I tried just the script that you gave me and I still get errors unfortunately. Is there anything else I can do to give you more precise info on these errors? Here is the output:

(base) [sdegregori@surfingmantis mimix]$ julia scripts/fit-mcmc.jl \
>     --hyper nutnet-analysis/configs/hyper.yml \
>     --monitor nutnet-analysis/configs/monitor-mimix.yml \
>     --inits nutnet-analysis/configs/inits.yml \
>     --factors 20 \
>     nutnet-analysis/reduced-data \
>     nutnet-analysis-results
┌ Warning: `@add_arg_table` is deprecated, use `@add_arg_table!` instead
└ @ ArgParse :-1
Reading X.csv
┌ Warning: `CSV.read(input; kw...)` is deprecated in favor of `DataFrame!(CSV.File(input; kw...))`
└ @ CSV ~/.julia/packages/CSV/OM6FO/src/CSV.jl:40
Reading Y.csv
┌ Warning: `CSV.read(input; kw...)` is deprecated in favor of `DataFrame!(CSV.File(input; kw...))`
└ @ CSV ~/.julia/packages/CSV/OM6FO/src/CSV.jl:40
Reading Z.csv
┌ Warning: `CSV.read(input; kw...)` is deprecated in favor of `DataFrame!(CSV.File(input; kw...))`
└ @ CSV ~/.julia/packages/CSV/OM6FO/src/CSV.jl:40
Beginning MCMC sampling
MCMC Simulation of 500 Iterations x 1 Chain...

Chain 1:   1% [0:13:40 of 0:13:49 remaining]
Chain 1:  10% [0:01:34 of 0:01:45 remaining]
Chain 1:  20% [0:00:51 of 0:01:04 remaining]
Chain 1:  30% [0:00:36 of 0:00:52 remaining]
Chain 1:  40% [0:00:27 of 0:00:45 remaining]
Chain 1:  50% [0:00:21 of 0:00:41 remaining]
Chain 1:  60% [0:00:15 of 0:00:38 remaining]
Chain 1:  70% [0:00:11 of 0:00:36 remaining]
Chain 1:  80% [0:00:07 of 0:00:35 remaining]
Chain 1:  90% [0:00:03 of 0:00:34 remaining]
Chain 1: 100% [0:00:00 of 0:00:33 remaining]

ERROR: LoadError: UndefVarError: q not defined
Stacktrace:
 [1] get_post(::ModelChains, ::Dict{Symbol,Any}, ::Symbol) at /data/home/sdegregori/mimix/MicrobiomeMixedModels.jl/src/utils.jl:50
 [2] top-level scope at /data/home/sdegregori/mimix/scripts/fit-mcmc.jl:162
in expression starting at /data/home/sdegregori/mimix/scripts/fit-mcmc.jl:106

[samd1993]

Hi Neal, I was not able to install an older version of Julia on the linux server unfortunately, but I did do it on my computer and get a similar error as before.

julia scripts/fit-mcmc.jl     --hyper nutnet-analysis/configs/hyper.yml     --monitor nutnet-analysis/configs/monitor-mimix.yml     --inits nutnet-analysis/configs/inits.yml     --factors 20     nutnet-analysis/reduced-data     nutnet-analysis-results
WARNING: Method definition (::Type{Distributions.MvNormal{T, Cov, Mean} where Mean<:(AbstractArray{T, 1} where T) where Cov<:(PDMats.AbstractPDMat{T} where T<:Real) where T<:Real})(AbstractArray{T<:Real, 1}, AbstractArray{U<:Real, 2}) where {T<:Real, U<:Real} in module Distributions at /home/sam/.julia/packages/Distributions/RAeyY/src/multivariate/mvnormal.jl:206 overwritten in module Mamba at /home/sam/.julia/packages/Mamba/Jotzr/src/distributions/constructors.jl:19.
WARNING: Method definition (::Type{Distributions.MvNormal{T, Cov, Mean} where Mean<:(AbstractArray{T, 1} where T) where Cov<:(PDMats.AbstractPDMat{T} where T<:Real) where T<:Real})(AbstractArray{T<:Real, 1}, AbstractArray{U<:Real, 1}) where {T<:Real, U<:Real} in module Distributions at /home/sam/.julia/packages/Distributions/RAeyY/src/multivariate/mvnormal.jl:212 overwritten in module Mamba at /home/sam/.julia/packages/Mamba/Jotzr/src/distributions/constructors.jl:22.
┌ Warning: `@add_arg_table` is deprecated, use `@add_arg_table!` instead
└ @ ArgParse :-1
Reading X.csv
┌ Warning: `CSV.read(input; kw...)` is deprecated in favor of `DataFrame!(CSV.File(input; kw...))`
└ @ CSV ~/.julia/packages/CSV/OM6FO/src/CSV.jl:40
Reading Y.csv
┌ Warning: `CSV.read(input; kw...)` is deprecated in favor of `DataFrame!(CSV.File(input; kw...))`
└ @ CSV ~/.julia/packages/CSV/OM6FO/src/CSV.jl:40
Reading Z.csv
┌ Warning: `CSV.read(input; kw...)` is deprecated in favor of `DataFrame!(CSV.File(input; kw...))`
└ @ CSV ~/.julia/packages/CSV/OM6FO/src/CSV.jl:40
Beginning MCMC sampling
ERROR: LoadError: StackOverflowError:
Stacktrace:
 [1] MvNormal(::Array{Float64,1}, ::Array{Float64,1}) at /home/sam/.julia/packages/Mamba/Jotzr/src/distributions/constructors.jl:22 (repeats 80000 times)
in expression starting at /home/sam/mimix/scripts/fit-mcmc.jl:106

[samd1993]

Hi Neal,

Just wanted to follow up on this. Cheers, Sam

[nsgrantham]

Glad to hear you were able to install RCall successfully.

I made some changes, please pull the latest version of this repository and try again. You may need to re-git clone and redo the installation instructions.

Also, you will need to upgrade to Mamba 0.12.4. You can do so by opening Julia, typing ], and then develop Mamba.

Try the commands again and report back any errors. Everything ran smoothly on my machine, so I'm hopeful it works for you too!

[samd1993]

It works! Thank you for the fix. Here is the output of the results just to confirm. I do not understand them completely but will try it on my own data

(base) samd1993@DESKTOP-V6PBDGC:~/mimix/simulation-results$ cat global-test.tsv
reject_global_null      dense   block_var       error_var       form
true    0.1     1.0     1.0     grouped
(base) samd1993@DESKTOP-V6PBDGC:~/mimix/simulation-results$ head local-estimates.tsv
mamba_name      mamba_node      value   mean    2.5%    25.0%   50.0%   75.0%   97.5%
β[1, 1] β       0.0     -0.010511752010771204   -0.3608680708057918     -0.024891957463448522   -3.053356821034312e-5  0.012955115758628653                                               0.2560719760995312
β[2, 1] β       0.0     -0.009209593331574829   -0.35323138212134286    -0.027794453460294407   -0.00037779682608627427                                                                   0.014963461989942424     0.21481375173417358
β[3, 1] β       0.0     -0.018259946078397073   -0.33297503558168123    -0.012957267127328021   -4.428577886278265e-6  0.007444876969222012                                               0.1304986804750615
β[4, 1] β       0.0     -0.03449270404380294    -0.3845886820545747     -0.024083087520670596   -0.0001551819013557428 0.0025286759160129584                                              0.12758727531922914
β[5, 1] β       0.0     -0.03394286566131153    -0.3030900055894073     -0.04332108644739287    -0.0016560668461119794 0.0015452183704638993                                              0.10548093795437968
β[6, 1] β       0.0     0.0018874392781088021   -0.226699750461055      -0.016542586337679042   -3.542355877585865e-11 0.022777289340046643                                               0.2334851132171152
β[7, 1] β       0.0     0.014449525178303634    -0.19895459704012747    -0.0032748893920792636  5.5114789979424515e-5  0.024592245399020607                                               0.2532283105165487
β[8, 1] β       0.0     -0.02672424239715789    -0.3452687599613158     -0.024790914733152026   -8.739499623338722e-6  0.003376134323487847                                               0.1464081030249542
β[9, 1] β       0.0     -0.14975538323627025    -0.7246320789154984     -0.25969582425871124    -0.025313673587776093  2.4280011041447577e-5                                              0.06316562775135519
(base) samd1993@DESKTOP-V6PBDGC:~/mimix/simulation-results$

One question however: In checking the sample data you've provided there are no row names or column headers. Is this intended? I simply downloaded them as CSV files and opened with excel so maybe I am missing something here.

[nsgrantham]

Lookin' good! If you aren't too familiar with Bayesian models, I highly recommend Statistical Rethinking by Richard McElreath.

Yes, the CSV files do not include row names or column headers so they can easily be read in as numeric matrices. I stored column headers separately in nutnet-analysis/full-data/tax.csv.

I'm going to close this issue. Please open a new issue if you run into other problems.

Happy MIMIXing!