julia chunk errors when using knitr

Bug description

Having julia chunks leads to error when there is an R chunk too; not in all cases, but e.g. when using Plots but also many other packages. The julia chunks work when executed in julia cli, and also when the R chunk is removed (but then the Python section is ignored). When removing the Plots part, doing only the DatFrame and CSV part, the julia chunk does not throw an error.

Steps to reproduce

# Introduction

```{r}
1

using DataFrames
using CSV
path = joinpath(pkgdir(DataFrames), "docs", "src", "assets", "german.csv");
german_ref = CSV.read(path, DataFrame)

using Plots
plot(x->x)

a = 3


### Expected behavior

It should create the plot and inline it.

### Actual behavior

An error occurs; the following output is generated:

$ quarto render index.qmd

processing file: index.qmd 1/7
2/7 [unnamed-chunk-1] 3/7
4/7 [unnamed-chunk-2] Julia version 1.9.4 at location /home/edzer/julia-1.9.4/bin will be used. Loading setup script for JuliaCall... Finish loading setup script for JuliaCall.

Quitting from lines 8-15 [unnamed-chunk-2] (index.qmd) Error: ! Error happens in Julia. InitError: could not load library "/home/edzer/.julia/artifacts/b6ebc4def1211ec4043d7c055f450d59f56747cc/lib/libgobject-2.0.so" /home/edzer/.julia/artifacts/b6ebc4def1211ec4043d7c055f450d59f56747cc/lib/libgobject-2.0.so: undefined symbol: g_bookmark_file_copy Stacktrace: [1] dlopen(s::String, flags::UInt32; throw_error::Bool) @ Base.Libc.Libdl ./libdl.jl:117 [2] dlopen(s::String, flags::UInt32) @ Base.Libc.Libdl ./libdl.jl:116 [3] macro expansion @ ~/.julia/packages/JLLWrappers/pG9bm/src/products/library_generators.jl:63 [inlined] [4] init() @ Glib_jll ~/.julia/packages/Glib_jll/MZy06/src/wrappers/x86_64-linux-gnu.jl:36 [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String) @ Base ./loading.jl:1115 [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any}) @ Base ./loading.jl:1061 [7] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::String, sourcepath::String, depmods::Vector{Any}) @ Base ./loading.jl:1391 [8] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128) @ Base ./loading.jl:1494 [9] _require(pkg::Base.PkgId, env::String) @ Base ./loading.jl:1783 [10] _require_prelocked(uuidkey::Base.PkgId, env::String) @ Base ./loading.jl:1660 [11] macro expansion @ ./loading.jl:1648 [inlined] [12] macro expansion @ ./lock.jl:267 [inlined] [13] require(into::Module, mod::Symbol) @ Base ./loading.jl:1611 [14] top-level scope @ none:3 [15] eval(m::Module, e::Any) @ Core ./boot.jl:370 [16] top-level scope @ ~/R/x86_64-pc-linux-gnu-library/4.3/JuliaCall/julia/RmdStd.jl:15 [17] eval @ ./boot.jl:370 [inlined] [18] eval_string(x::String) @ Main.JuliaCall ~/R/x86_64-pc-linux-gnu-library/4.3/JuliaCall/julia/setup.jl:203 [19] docall(call1::Ptr{Nothing}) @ Main.JuliaCall ~/R/x86_64-pc-linux-gnu-library/4.3/JuliaCall/julia/setup.jl:176 during initialization of module Glib_jll Backtrace:

global .main()
execute(...)
rmarkdown::render(...)
knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
knitr:::process_file(text, output) ...
1. JuliaCall:::stdout_capture_command(buffer)
2. base::tryCatch(...)
3. base (local) tryCatchList(expr, classes, parentenv, handlers)
4. base (local) tryCatchOne(...)
5. value[3L] Execution halted Julia exit.

Your environment

Ubuntu 22.04, nvim

Quarto check output

Quarto 1.4.514
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.9: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.514
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/bin
      Version: 2021

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.11.5 (Conda)
      Path: /home/edzer/miniconda3/bin/python
      Jupyter: 5.5.0
      Kernels: julia-1.9, python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.3.2
      Path: /usr/lib/R
      LibPaths:
        - /home/edzer/R/x86_64-pc-linux-gnu-library/4.3
        - /usr/local/lib/R/site-library
        - /usr/lib/R/site-library
        - /usr/lib/R/library
      knitr: 1.45
      rmarkdown: 2.24

[✓] Checking Knitr engine render......OK

Hi. Thanks for the report.

Some context here on what is happening.

When using a R chunk inside the document, Quarto computation engine binding will be set to engine: knitr. This means that all the non R chunk will be run through their knitr support.

python chunk will be run with reticulate and julia chunk will be run with JuliaCall (https://github.com/Non-Contradiction/JuliaCall)

The traceback you shared shows that as the error is thrown from JuliaCall

 17. JuliaCall:::stdout_capture_command(buffer)

So first thing would be to test that your Julia environment can be called from R using JuliaCall directly (outside of any knitr engine support). What knitr is doing is calling JuliaCall directly with no specific processing

> knitr:::eng_julia
function (options) 
{
    JuliaCall::eng_juliacall(options)
}

So if possibly I would check that you can run your Julia code from R using JuliaCall in R console - I think there could be a configuration problem at this step somehow.

The JuliaCall website should help : https://non-contradiction.github.io/JuliaCall/index.html#installation

Unless you did check that already, and this is something else...

Though I believe you would get the same error outside of Quarto directly with knitr (knitr::knit()) or within R Markdown.

Hope it helps.

Thanks @cderv, very helpful! Using a hint from https://github.com/Non-Contradiction/JuliaCall/issues/192 I can get the following to work: the julia script (cli)

using Plots
plot(x->x)

The R script:

library(JuliaCall)
julia_setup()
julia_library("Plots")
a = 1:10
b = rnorm(10)
julia_call("plot", a, b)

but only when called with R_LD_LIBRARY_PATH set (see https://github.com/Non-Contradiction/JuliaCall/issues/192):

R_LD_LIBRARY_PATH="~/julia-1.9.4/lib/julia/" R -q --vanilla < tst.R

which gives as output:

> library(JuliaCall)
> julia_setup()
Julia version 1.9.4 at location /home/edzer/julia-1.9.4/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
> julia_library("Plots")
> a = 1:10
> b = rnorm(10)
> julia_call("plot", a, b)
Julia Object of type Plots.Plot{Plots.GRBackend}.
Plot{Plots.GRBackend() n=1}> 
Julia exit.

but not the following qmd:

```{r}
1

using Plots
plot(x->x^2)

when compiled with
```bash
R_LD_LIBRARY_PATH="~/julia-1.9.4/lib/julia/" quarto render tst.qmd

it gives me the following output

``` processing file: tst.qmd 1/4 [unnamed-chunk-1] 2/4 3/4 [unnamed-chunk-2] Julia version 1.9.4 at location /home/edzer/julia-1.9.4/bin will be used. Loading setup script for JuliaCall... Finish loading setup script for JuliaCall. Quitting from lines 6-8 [unnamed-chunk-2] (tst.qmd) Error: ! Error happens in Julia. InitError: could not load library "/home/edzer/.julia/artifacts/b6ebc4def1211ec4043d7c055f450d59f56747cc/lib/libgobject-2.0.so" /home/edzer/.julia/artifacts/b6ebc4def1211ec4043d7c055f450d59f56747cc/lib/libgobject-2.0.so: undefined symbol: g_bookmark_file_copy Stacktrace: [1] dlopen(s::String, flags::UInt32; throw_error::Bool) @ Base.Libc.Libdl ./libdl.jl:117 [2] dlopen(s::String, flags::UInt32) @ Base.Libc.Libdl ./libdl.jl:116 [3] macro expansion @ ~/.julia/packages/JLLWrappers/pG9bm/src/products/library_generators.jl:63 [inlined] [4] __init__() @ Glib_jll ~/.julia/packages/Glib_jll/MZy06/src/wrappers/x86_64-linux-gnu.jl:36 [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String) @ Base ./loading.jl:1115 [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any}) @ Base ./loading.jl:1061 [7] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::String, sourcepath::String, depmods::Vector{Any}) @ Base ./loading.jl:1391 [8] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128) @ Base ./loading.jl:1494 [9] _require(pkg::Base.PkgId, env::String) @ Base ./loading.jl:1783 [10] _require_prelocked(uuidkey::Base.PkgId, env::String) @ Base ./loading.jl:1660 [11] macro expansion @ ./loading.jl:1648 [inlined] [12] macro expansion @ ./lock.jl:267 [inlined] [13] require(into::Module, mod::Symbol) @ Base ./loading.jl:1611 [14] top-level scope @ none:3 [15] eval(m::Module, e::Any) @ Core ./boot.jl:370 [16] top-level scope @ ~/R/x86_64-pc-linux-gnu-library/4.3/JuliaCall/julia/RmdStd.jl:15 [17] eval @ ./boot.jl:370 [inlined] [18] eval_string(x::String) @ Main.JuliaCall ~/R/x86_64-pc-linux-gnu-library/4.3/JuliaCall/julia/setup.jl:203 [19] docall(call1::Ptr{Nothing}) @ Main.JuliaCall ~/R/x86_64-pc-linux-gnu-library/4.3/JuliaCall/julia/setup.jl:176 during initialization of module Glib_jll Backtrace: 1. global .main() 2. execute(...) 3. rmarkdown::render(...) 4. knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet) 5. knitr:::process_file(text, output) ... 17. JuliaCall:::stdout_capture_command(buffer) 18. base::tryCatch(...) 19. base (local) tryCatchList(expr, classes, parentenv, handlers) 20. base (local) tryCatchOne(...) 21. value[[3L]](cond) Execution halted Julia exit. ```

which points to the same problem as running

R -q --vanilla < tst.R

i.e. without setting R_LD_LIBRARY_PATH, which outputs

``` > library(JuliaCall) > julia_setup() Julia version 1.9.4 at location /home/edzer/julia-1.9.4/bin will be used. Loading setup script for JuliaCall... Finish loading setup script for JuliaCall. > julia_library("Plots") Error: Error happens in Julia. InitError: could not load library "/home/edzer/.julia/artifacts/b6ebc4def1211ec4043d7c055f450d59f56747cc/lib/libgobject-2.0.so" /home/edzer/.julia/artifacts/b6ebc4def1211ec4043d7c055f450d59f56747cc/lib/libgobject-2.0.so: undefined symbol: g_bookmark_file_copy Stacktrace: [1] dlopen(s::String, flags::UInt32; throw_error::Bool) @ Base.Libc.Libdl ./libdl.jl:117 [2] dlopen(s::String, flags::UInt32) @ Base.Libc.Libdl ./libdl.jl:116 [3] macro expansion @ ~/.julia/packages/JLLWrappers/pG9bm/src/products/library_generators.jl:63 [inlined] [4] __init__() @ Glib_jll ~/.julia/packages/Glib_jll/MZy06/src/wrappers/x86_64-linux-gnu.jl:36 [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String) @ Base ./loading.jl:1115 [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any}) @ Base ./loading.jl:1061 [7] _tryrequire_from_serialized(modkey::Base.PkgId, path::Strin Execution halted Julia exit. ```

Note that the problem comes from using Plots, not from the second line. Are there b.t.w. any examples of using quarto documents that combine R and Julia, and handle Julia plots?

I managed to get some environment on Ubuntu 22.04 running to try reproduce the Julia error. I believe this is all some conflicts in library .so version with newest Plots.jl and the fact this is call from R.

First doing something like

R_LD_LIBRARY_PATH="~/julia-1.9.4/lib/julia/" R

Does not work on my side, because changing R_LD_LIBRARY_PATH which is R default will hide the libR.so and I get this error.

/opt/R/4.3.2/lib/R/bin/exec/R: error while loading shared libraries: libR.so: cannot open shared object file: No such file or directory

This makes sense to me considering that this is default value set by R itself according to doc (https://cran.r-project.org/doc/manuals/r-release/R-admin.html#External-software-1)

he secondary mechanism is to consult the environment variable LD_LIBRARY_PATH. The R script controls that variable, and sets it to the concatenation of R_LD_LIBRARY_PATH, R_JAVA_LD_LIBRARY_PATH and the environment value of LD_LIBRARY_PATH. The first two have defaults which are normally set when R is installed (but can be overridden in the environment) so LD_LIBRARY_PATH is the best choice for a user to set.

So I am surprised this has worked for you.

However, R_LD_LIBRARY_PATH is set first, so I could modify the other value to add if I do

export LD_LIBRARY_PATH=/home/cderv/.julia/juliaup/julia-1.9.4+0.x64.linux.gnu/lib/julia/

However, I will still get the error on Julia code from R because I believe the wrong version is looked for in wrong place.

This is to me the heart of the issue

libgobject-2.0.so: undefined symbol: g_bookmark_file_copy

and this seems to be available since fairly recently in Glib 2.76 according to https://docs.gtk.org/glib/method.BookmarkFile.copy.html. So maybe this could be fixed by installing / compiling Plots.jl against an older version maybe or something like this (not enough Julia expert for now).

Anyhow, if I continue trying to set Julia libs fist, I can do inverse top put Julia first

export R_LD_LIBRARY_PATH="/home/cderv/.julia/juliaup/julia-1.9.4+0.x64.linux.gnu/lib/julia/:$(R RHOME)/lib:/opt/local/lib"

Then I can make the R code work. But Quarto rendering still has a problem.

I believe there is then an issue inside Quarto rendering because Quarto calls R through Deno, which call Julia through R and somehow the LD paths got mixed up again at some point.

I would say this is something in how R calls Julia with not setting the Julia Libs first for resolution probably, and some conflict in dynamic library linking.

I'll keep digging next week, but just wanted to share my feedback on what I tried.

quarto-dev / quarto-cli