Extend completion with source by recursive parsing

[renkun-ken]

It is tricky to provide completion without attaching to the R session being used by user. One simplified approach that I believe can serve most of the purposes is parse the currently editing document and find top-level source functions, and parse those files recursively.

For example, the following code is from several R scripts:

main.R:

library(data.table)
library(glmnet)
source("my_plot.R")
source("data/utils.R")

my_plot.R:

library(ggplot2)

data/utils.R:

library(DBI)
requireNamespace("RMariaDB")

When main.R is being edited, those library expressions are captured. If those source expressions are captured and my_plot.R and data/utils.R are analyzed, where library and source expressions in these files are also captured, the completion will be much better and no less accurate, especially when #19 is implemented.

Also those top-level library(), require(), loadNamespace(), requireNamespace() calls can be used to provide diagnostics when required package is not installed.

[randy3k]

Right now, we search for library and require in the current document only and show docs from those packages required. We need to extended it to source and support R package.

We also have to be cautious about infinitely loop while the files are loaded recursively.

[renkun-ken]

Agreed.

A set of files associated with the document can be cached to avoid the infinite loop of source, i.e., each source will update the set of source files before processing the file, so that if a source file is already in the set, it would be ignored.

[randy3k]

A related but not entirely the same issue

tidyverse uses a complicated way to load its dependencies rather than just using the Depends field. It would be hard for us to determine what packages need to recursive loaded.

One solution is to open a new R session via callr, load the library (in this case tidyverse) and see what packages are being attached.

callr::r(function(){library(tidyverse); search()})

[renkun-ken]

Seems like a smart approach that avoids most of the complexities.

[syu-id]

I'm using the jupyterlab-lsp extension on JupyterLab, and completion for packages introduced by library(tidyverse) are not provided. Support for tidyverse would be great.

[randy3k]

It’s has already been done. However, the package dependencies are only resolved when the file is open and saved.

[syu-id]

It’s has already been done. However, the package dependencies are only resolved when the file is open and saved.

Hi, Thanks for the reply! I tried with Vim and completion for tidyverse packages works nicely. So my previous issue might just be specific to JupyterLab.

[randy3k]

It may be related to #15 and #27.

[syu-id]

Thank you. I have filed an issue at the jupyterlab-lsp repo and linked your suggestions: https://github.com/krassowski/jupyterlab-lsp/issues/95

[MilesMcBain]

This would not work that well if the thing being sourced is lot a literal string. In my own workflow I frequently have something that does a thing like lapply(list.files(...), source, ...).

[siegfried]

Hi, I got a no visible global function definition for 'dseconds' if I write library(tidyverse). It can be removed by adding library(lubridate). Is it related to this issue?