R/Python Chunks?

[versipellis]

Does Weave support R/Python code chunks? Is there a best practice for using R/Python if you're not explicitly using RCall/PyCall?

[dmbates]

For R I think that it would be possible to incorporate RCall support more tightly in Weave. An advantage of RCall is that it provides many conversion methods for R objects to Julia. At present RCall provides IJulia integration for plots. In a Jupyter notebook including an RCall chunk that generates a plot results in the plot being displayed. (i.e. it writes a .png or .svg file which is then included automatically in the notebook.) I think a similar approach could be used in Weave.

Looking at https://github.com/JuliaInterop/RCall.jl/blob/master/src/ijulia.jl it seems that very little of that code is IJulia-specific. I would be willing to work on adapting RCall and Weave to cooperate if someone (@mpastell ?) could point me in the right direction to adapt the Weave code.

Another possible point of integration between RCall and Weave is to automatically convert chunk enclosed in "r" and "" as

R"""
<block contents>
"""

[Non-Contradiction]

Just my 2 cents, I work on JuliaCall, which embeds julia in R and provides the functionality for RMarkdown document to have julia chunks. This can be done because knitr (the RMarkdown document package) provides ways for integrating language chunks other than R. I think a similar way can be incorporated into Weave quite easily utilizing julia's multiple dispatching. When I browse the code of Weave, I think the run_code is the main function to weave all thing together. https://github.com/JunoLab/Weave.jl/blob/b01b18ea2f824febe7742821dfc1829740769a5b/src/run.jl#L198 It should be easily extensible by allowing the run_code function to have a language parameter. So individual packages like RCall and PyCall can have their own run_code method for their specific languages like R and Python. So Weave could have things like:

function run_code(lang::Lang(:julia), chunk::CodeChunk, report::Report, SandBox::Module)

And RCall and PyCall could have functions like

function run_code(lang::Lang(:R), chunk::CodeChunk, report::Report, SandBox::Module)

function run_code(lang::Lang(:python), chunk::CodeChunk, report::Report, SandBox::Module)

Or maybe instead of changing run_code to have language arguments, we could allow parse_input and capture_output to have their first arguments as languages. Maybe this is a better design? After going through the code again, extending run_code function with an additional language augument seems to be the best way currently. Will a PR for this be accepted?

[dmbates]

Bump. Any reactions from JunoLab members or @mpastell to this suggestion?

[pfitzseb]

The approach outlined by @Non-Contradiction makes sense to me -- it would probably make sense to run_code(lang::Lang(:python) etc in a @require block in Weave.jl instead of in e.g. PyCall though.

In a Jupyter notebook including an RCall chunk that generates a plot results in the plot being displayed. (i.e. it writes a .png or .svg file which is then included automatically in the notebook.) I think a similar approach could be used in Weave.

That would be great to have. It seems like it should be relatively to do the relevant setup and post-execution tasks here with minimal refactoring in RCall.

[dmbates]

@Non-Contradiction Would you have time to collaborate on adding this functionality? I'll start but I would probably benefit from your looking at what I do and offering suggestions.

[Non-Contradiction]

@dmbates Yes, I am willing to collaborate for this functionality.

[dmbates]

@Non-Contradiction Can you clarify what you mean by an expression like Lang(:R) as an argument type? To me it seems that you have in mind something like

julia> struct Lang
          lang::Symbol
       end

julia> Lang(:julia)
Lang(:julia)

julia> typeof(ans)
Lang

but you can't dispatch on the value, only the type.

[dmbates]

Perhaps it would be better to use an abstract Lang type and have specific languages inherit from the abstract type? I have some suggestions in a document doc/src/overall_structure.md in https://github.com/dmbates/Weave.jl Could you take a look @Non-Contradiction (and perhaps also @pfitzseb) to see if this is what you would have had in mind?

[Non-Contradiction]

Sorry I messed up the notation. The thing originally in my mind was something like this:

julia> struct Lang{T} end

julia> Lang{:R}
Lang{:R}

julia> f(::Lang{T}) where T = T
f (generic function with 1 method)

julia> f(Lang{:R}())
:R

[Non-Contradiction]

After more consideration, I think the abstract type way as in the overall_structure.md is more flexible and extensible and should be the way to go. For example, the run_code function does not seem to have any arguments for options currently, but if we want to allow some options for R/Python/other languages, it is easy to have more fields for the specific language type.

[dmbates]

I have a version that runs some examples in the RCall branch at https://github.com/dmbates/Weave.jl This part is just an attempt to get the .jmd input to function as before but with a lang field that is an AbstractLanguage in the CodeChunk struct. The CodeChunk struct is parameterized by the language, which is almost what @Non-Contradiction was suggesting. I also named the capture groups in the codestart regular expression for MarkupInput just to make it easier to figure out what is where.

Changing the codestart regular expression for the markdown files will necessitate changing the code for noweb files, which don't have a language specification. I haven't done that yet so some of the tests will error.

Before devoting more effort to this, can I get opinions on whether this approach seems reasonable? The next step will be to write run_code and collect_results methods for CodeChunk{RLang} types.

[ym-han]

Just bumping this thread --- would love to see support for RCall.jl

[TheCedarPrince]

Bump - any news on this @dmbates or @Non-Contradiction ?

[dmbates]

No progress from me as I haven't been using Weave.jl recently.

[TheCedarPrince]

Gotcha @dmbates - do you happen to still have your fork around? I would be happy to tinker with trying to get RCall-based blocks working within Weave .

[TheCedarPrince]

@pfitzseb - I would really like to get this working for Weave.jl Is there any interest in still having this in the Weave ecosystem? My first priority would be to work with RCall.jl to get things working.

[pfitzseb]

Sure. I don't really have an opinion on the best implementation for this, but would be happy to discuss if you open a PR that sketches out the functionality.