error in prepcommand (interactive testing before docker)

[FMKerckhof]

Summary of my (general) question: does renv need to be in the renv.lock file for jetpack to work? If so: should this be taken care of by jetpack::migrate?

The reason for this question is outlined below, I have not able to resolve it so far. I apologize for not really providing a code reprex, but I think I describe the process in enough detail to reproduce.

---

Goal of the issue

For a legacy deploy I am running R3.6.2 in WSL1 Ubuntu 18.04 - I am trying to better understand the interplay between jetpack and it's renv backend - the ultimate aim is to build a docker image as outlined in the README of jetpack here, running init.R in the DOCKERFILE will take care of the installation of jetpack and dependents. First, I want to be able to interactively tweak a few more packages and versions.

Set-up

Hence, for interactive work, R 3.6.2 was installed from Rstudio-provided binaries, and I only installed remotes and jetpack (0.5.2) as additional packages. Jetpack will automatically install renv (0.15.4) upon install, since it is a dependency. Hence the 'system' R library has access to both jetpack and renv.

The project used to be under jetpack/packrat control (older versions of jetpack used packrat) and I have runjetpack::migrate and deleted the packrat.lock file. Therefore, the project folder has a DESCRIPTION and renv.lock file inside. The package renv itself is not part of the renv.lock file, since it should be installed when installing jetpack through init.R in the DOCKERFILE and is system-available.

Workflow and problems

When I clone from github and start R, jetpack will ask me if I want to create a virtual environment, which I agreed to.

Now, I want to run jetpack::install(deployment=TRUE) in order to restore my renv. First, I ran jetpack::check() which showed me that renv is missing.

Missing packages: renv
Run 'jetpack::install()' to install them

It appears that in the virtual environment, renv was not present. However, running renv::status() does list the package as installed:

The following package(s) are installed but not recorded in the lockfile:
       _
  renv   [0.15.4]

Use `renv::snapshot()` to add these packages to your lockfile.

running renv::depencies will list renv as a dependency because there is a lockfile present - getMissing will therefore always throw a diff if renv is not in the lockfile (even if it is available).

Hence I proceeded with taking jetpack's advice and using jetpack::install() -> I assumed that this would add renv to the virtual (sandbox) environment - which results in a very odd error:

Error: "" does not exist
Traceback (most recent calls last):
19: jetpack::install()
18: sandbox({
        if (deployment) {
            status <- getStatus()
            missing <- getMissing(status)
            if (length(missing) > 0) {
                stop(paste("Missing packages:", paste(missing, collapse = ", ")))
            }
            verboseRenv(suppressWarnings(renv::restore(prompt = FALSE)))
            showStatus(status)
        }
        else {
            installHelper(show_status = TRUE)
        }
        success("Pack complete!")
    })
17: configureRenv(code)
16: eval(code)
15: installHelper(show_status = TRUE)
14: verboseRenv(suppressWarnings(renv::restore(project = dir, prompt = FALSE,
        clean = TRUE)))
13: eval(code)
12: suppressWarnings(renv::restore(project = dir, prompt = FALSE,
        clean = TRUE))
11: withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))
10: renv::restore(project = dir, prompt = FALSE, clean = TRUE)
 9: renv_restore_run_actions(project, diff, current, lockfile, rebuild)
 8: retrieve(packages)
 7: sprintf("renv (%s)", renv_package_version("renv"))
 6: base::sprintf(fmt, ...)
 5: renv_package_version("renv")
 4: renv_package_description_field(package, "Version")
 3: renv_description_read(path)
 2: stopf(fmt, renv_path_pretty(path))
 1: stop(sprintf(fmt, ...), call. = call.)

So clearly, something is off with the normal way of doing things. If the "missingness" of renv is the problem, let's just record it into the lockfile?

renv::record("renv@0.15.4")
* Updated 1 record in "~/.local/share/renvs/src-2572/renv.lock".
Warning messages:
1: In read.dcf(file.path(p, "DESCRIPTION"), c("Package", "Version")) :
  cannot open compressed file '/home/fpkerckh/.cache/R/renv/library/src-2572-fd232139/R-3.6/x86_64-pc-linux-gnu/renv/DESCRIPTION', probable reason 'No such file or directory'
2: In file(con, "r") :
  file("") only supports open = "w+" and open = "w+b": using the former

Clearly that tries to update the renv.lock in the virtual environment, but not the original one in the project directory (which remains untouched). jetpack::check() keeps on telling me there is no renv.

Since I was stumped at this stage I decided to quit R, remove the project and clone it again. From then on, it appears that I could not run a single jetpack command anymore because of errors in the

Error in prepCommand() :
  renv must be loaded to run this. Restart your R session to continue.

Loading renv (library(renv)) does not help quitting R doesn't help either, starting R with --no-restore (or vanilla) neither.

My sessionInfo() output (after running library("renv")) is:

R version 3.6.2 (2019-12-12)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] renv_0.15.4

loaded via a namespace (and not attached):
[1] compiler_3.6.2  R6_2.5.1        rprojroot_2.0.3 cli_3.2.0
[5] tools_3.6.2     jetpack_0.5.2   remotes_2.4.2   docopt_0.7.1
[9] desc_1.4.1

[FMKerckhof]

I tried to run jetpack check from outside of R (i.e. via the CLI) - which also resulted the message that renv is missing. Adding it via the CLI jetpack add renv@0.15.4 failed with the error "" does not exist

[ankane]

Hey @FMKerckhof, Jetpack should add renv to renv.lock when calling migrate() (relevant code), but if that's not happening for some reason, you can try adding it manually:

    "renv": {
      "Package": "renv",
      "Version": "0.15.4",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "c1078316e1d4f70275fc1ea60c0bc431",
      "Requirements": []
    },

[FMKerckhof]

thank you - I don't know why migrate didn't add, looking at your code, but manually adding the entry allowed me to resolve the issue.

[FMKerckhof]

A question on portability @ankane : after jetpack::install(deployment=TRUE), when I want to add packages (or change to specific versions of them or add remote packages) jetpack::add() will update the DESCRIPTION (and renv.lock?), but only in the virtual environment, is that correct? - therefore they will not be picked up by git. What is the advised way to include these changes under version control?

[ankane]

jetpack::add() will update DESCRIPTION and renv.lock in the project directory, which should be checked into version control. Also, fwiw, deployment=TRUE is meant to be used for deploying code to a production environment rather than an environment where you're adding packages.