misconfiguration of {renv} and renv version causes bootstrap problems

zkamvar commented 1 year ago

I am trying to set up this repo so that others can use it, but unfortunately, it is borked in some way that I am not sure how to get out of.

Here are the issues:

[x] dplyr is needed in the lockfile (it was inadvertantly removed in 91087190a113ef0906eec2973409880c97090632, but because I only had this running on my machine, the package was still in my local {renv} library)
[x] There is some miscommunication with renv::hydrate(), renv::restore() and renv::snapshot() that I cannot seem to pin down.

zkamvar commented 1 year ago

Indeed the second problem is appearing in brand new lessons: https://github.com/zkamvar/urban-succotash/actions/runs/4493152698/jobs/7903987389

Output of sandpaper::manage_deps() with renv 0.17.2

``` ℹ Consent to use package cache provided * "~/.local/share/renv/" has been created. → Searching for and installing available dependencies Finding R package dependencies ... Done! * Discovering package dependencies ... Done! The following packages were discovered: # ~/work/_temp/Library =============== - R6 2.5.1 - base64enc 0.1-3 - bslib 0.4.2 - cachem 1.0.7 - cli 3.6.0 - digest 0.6.31 - ellipsis 0.3.2 - evaluate 0.20 - fastmap 1.1.1 - fs 1.6.1 - glue 1.6.2 - highr 0.10 - htmltools 0.5.4 - jquerylib 0.1.4 - jsonlite 1.8.4 - knitr 1.42 - lifecycle 1.0.3 - magrittr 2.0.3 - memoise 2.0.1 - mime 0.12 - rappdirs 0.3.3 - rlang 1.1.0 - rmarkdown 2.20 - sass 0.4.5 - stringi 1.7.12 - stringr 1.5.0 - tinytex 0.44 - vctrs 0.6.1 - xfun 0.37 - yaml 2.3.7 They will be linked into the project library. * Linking packages into the project library ... The following packages are used in this project, but not available locally: reprex renv will attempt to download and install these packages. Done! * Hydrated 30 packages in 0.28 seconds. * Resolving missing dependencies ... Retrieving 'https://cran.rstudio.com/src/contrib/reprex_2.0.2.tar.gz' ... OK [downloaded 1 Mb in 0.17 seconds] Installing reprex [2.0.2] ... OK [built from source in 2.2 seconds] Moving reprex [2.0.2] into the cache ... OK [moved to cache in 22 milliseconds] → Restoring any dependency versions The following package(s) will be updated: # CRAN =============================== - vctrs [0.6.1 -> 0.6.0] - renv [* -> 0.17.2] * Querying repositories for available source packages ... Done! Retrieving 'https://cran.rstudio.com/src/contrib/Archive/vctrs/vctrs_0.6.0.tar.gz' ... OK [downloaded 941.6 Kb in 0.32 seconds] Retrieving 'https://cran.rstudio.com/src/contrib/renv_0.17.2.tar.gz' ... OK [downloaded 1 Mb in 0.18 seconds] Installing vctrs [0.6.0] ... OK [built from source in 19 seconds] Moving vctrs [0.6.0] into the cache ... OK [moved to cache in 6.7 milliseconds] Installing renv [0.17.2] ... OK [built from source in 9 seconds] Moving renv [0.17.2] into the cache ... OK [moved to cache in 7.3 milliseconds] → Recording changes in lockfile The following required packages are not installed: callr [required by reprex] clipr [required by reprex] rstudioapi [required by reprex] withr [required by reprex] Consider reinstalling these packages before snapshotting the lockfile. Error: ! in callr subprocess. Caused by error in `renv::snapshot(project = path, lockfile = renv_lock, prompt = FALSE)`: ! aborting snapshot due to pre-flight validation failure --- Standard output: Finding R package dependencies ... Done! * Discovering package dependencies ... Done! * Linking packages into the project library ... Done! * Hydrated 30 packages in 0.28 seconds. * Resolving missing dependencies ... Retrieving 'https://cran.rstudio.com/src/contrib/reprex_2.0.2.tar.gz' ... OK [downloaded 1 Mb in 0.17 seconds] Installing reprex [2.0.2] ... OK [built from source in 2.2 seconds] Moving reprex [2.0.2] into the cache ... OK [moved to cache in 22 milliseconds] * Querying repositories for available source packages ... Done! Retrieving 'https://cran.rstudio.com/src/contrib/Archive/vctrs/vctrs_0.6.0.tar.gz' ... OK [downloaded 941.6 Kb in 0.32 seconds] Retrieving 'https://cran.rstudio.com/src/contrib/renv_0.17.2.tar.gz' ... OK [downloaded 1 Mb in 0.18 seconds] Installing vctrs [0.6.0] ... OK [built from source in 19 seconds] Moving vctrs [0.6.0] into the cache ... OK [moved to cache in 6.7 milliseconds] Installing renv [0.17.2] ... OK [built from source in 9 seconds] Moving renv [0.17.2] into the cache ... OK [moved to cache in 7.3 milliseconds] --- Standard error: → Searching for and installing available dependencies The following packages were discovered: # ~/work/_temp/Library =============== - R6 2.5.1 - base64enc 0.1-3 - bslib 0.4.2 - cachem 1.0.7 - cli 3.6.0 - digest 0.6.31 - ellipsis 0.3.2 - evaluate 0.20 - fastmap 1.1.1 - fs 1.6.1 - glue 1.6.2 - highr 0.10 - htmltools 0.5.4 - jquerylib 0.1.4 - jsonlite 1.8.4 - knitr 1.42 - lifecycle 1.0.3 - magrittr 2.0.3 - memoise 2.0.1 - mime 0.12 - rappdirs 0.3.3 - rlang 1.1.0 - rmarkdown 2.20 - sass 0.4.5 - stringi 1.7.12 - stringr 1.5.0 - tinytex 0.44 - vctrs 0.6.1 - xfun 0.37 - yaml 2.3.7 They will be linked into the project library. The following packages are used in this project, but not available locally: reprex renv will attempt to download and install these packages. → Restoring any dependency versions The following package(s) will be updated: # CRAN =============================== - vctrs [0.6.1 -> 0.6.0] - renv [* -> 0.17.2] → Recording changes in lockfile The following required packages are not installed: callr [required by reprex] clipr [required by reprex] rstudioapi [required by reprex] withr [required by reprex] Consider reinstalling these packages before snapshotting the lockfile. --- Backtrace: 1. sandpaper::manage_deps(path = wd, quiet = FALSE) 2. callr::r(func = callr_manage_deps, args = args, show = !quiet, … 3. callr:::get_result(output = out, options) 4. callr:::throw(callr_remote_error(remerr, output), parent = fix_msg(remerr[[3]])) --- Subprocess backtrace: 1. renv::snapshot(project = path, lockfile = renv_lock, prompt = FALSE) 2. base::stop("aborting snapshot due to pre-flight validation failure") 3. | base::.handleSimpleError(function (e) … 4. global h(simpleError(msg, call)) Execution halted ```

I get a strong feeling that it has something to do with a pre-provisioned package cache. I was able to go in and fix this problem by deleting the local project folder and restoring from there and I will recreate the steps here to see what the hell is going on.

zkamvar commented 1 year ago

And I've just run into a fun permissions kerfuffle that's taking my time away from this: https://github.com/carpentries/sandpaper/pull/421

zkamvar commented 1 year ago

As I suspected: it does have something to do with how renv works in 0.17.2 as I can revert to 0.16.0 and have success in https://github.com/zkamvar/urban-succotash/actions/runs/4493152698/jobs/7908951296

output of sandpaper::manage_deps() for renv 0.16.0

``` Installing package into ‘/home/runner/work/_temp/Library’ (as ‘lib’ is unspecified) inferring 'repos = NULL' from 'pkgs' trying URL 'https://cran.r-project.org/src/contrib/Archive/renv/renv_0.16.0.tar.gz' Content type 'application/x-gzip' length 1004572 bytes (981 KB) ================================================== downloaded 981 KB * installing *source* package ‘renv’ ... ** package ‘renv’ successfully unpacked and MD5 sums checked ** using staged installation ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices *** copying figures ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (renv) ℹ Consent to use package cache provided → Searching for and installing available dependencies Finding R package dependencies ... Done! * Discovering package dependencies ... Done! * Copying packages into the cache ... Done! * Resolving missing dependencies ... Retrieving 'https://cran.rstudio.com/src/contrib/reprex_2.0.2.tar.gz' ... OK [downloaded 1 Mb in 0.1 secs] Retrieving 'https://cran.rstudio.com/src/contrib/callr_3.7.3.tar.gz' ... OK [downloaded 95.5 Kb in 0.1 secs] Retrieving 'https://cran.rstudio.com/src/contrib/processx_3.8.0.tar.gz' ... OK [downloaded 158.7 Kb in 0.1 secs] Retrieving 'https://cran.rstudio.com/src/contrib/ps_1.7.3.tar.gz' ... OK [downloaded 126.1 Kb in 0.1 secs] Retrieving 'https://cran.rstudio.com/src/contrib/clipr_0.8.0.tar.gz' ... OK [downloaded 21.4 Kb in 0.1 secs] Retrieving 'https://cran.rstudio.com/src/contrib/rstudioapi_0.14.tar.gz' ... OK [downloaded 112.7 Kb in 0.1 secs] Retrieving 'https://cran.rstudio.com/src/contrib/withr_2.5.0.tar.gz' ... OK [downloaded 99.7 Kb in 0.1 secs] Installing ps [1.7.3] ... OK [built from source] Moving ps [1.7.3] into the cache ... OK [moved to cache in 7.4 milliseconds] Installing processx [3.8.0] ... OK [built from source] Moving processx [3.8.0] into the cache ... OK [moved to cache in 7.7 milliseconds] Installing callr [3.7.3] ... OK [built from source] Moving callr [3.7.3] into the cache ... OK [moved to cache in 6.7 milliseconds] Installing clipr [0.8.0] ... OK [built from source] Moving clipr [0.8.0] into the cache ... OK [moved to cache in 6.7 milliseconds] Installing rstudioapi [0.14] ... OK [built from source] Moving rstudioapi [0.14] into the cache ... OK [moved to cache in 6.9 milliseconds] Installing withr [2.5.0] ... OK [built from source] Moving withr [2.5.0] into the cache ... OK [moved to cache in 6.7 milliseconds] Installing reprex [2.0.2] ... OK [built from source] Moving reprex [2.0.2] into the cache ... OK [moved to cache in 7.1 milliseconds] → Restoring any dependency versions The following package(s) will be updated: # CRAN =============================== - vctrs [0.6.1 -> 0.6.0] - renv [* -> 0.17.2] Installing vctrs [0.6.0] ... OK [linked cache] Installing renv [0.17.2] ... OK [linked cache] The following package(s) have been updated: renv [installed version 0.17.2 != loaded version 0.16.0] Consider restarting the R session and loading the newly-installed packages. → Recording changes in lockfile The following package(s) will be updated in the lockfile: # CRAN =============================== - callr [* -> 3.7.3] - clipr [* -> 0.8.0] - processx [* -> 3.8.0] - ps [* -> 1.7.3] - reprex [* -> 2.0.2] - rstudioapi [* -> 0.14] - withr [* -> 2.5.0] * Lockfile written to '~/work/urban-succotash/urban-succotash/renv/profiles/lesson-requirements/renv.lock'. ```

zkamvar commented 1 year ago

The first point was addressed in https://github.com/carpentries/lesson-transition/commit/a46fefb9fbeb994ce93c657f6a710cd5e98e76ce

zkamvar commented 1 year ago

The second point was addressed in https://github.com/carpentries/lesson-transition/commit/dbd0ea9358c1a02dadaac29e66942e810d7e150b with the most recent update of sandpaper

zkamvar commented 1 year ago

I believe the bootstrap issues are over

carpentries / lesson-transition

misconfiguration of {renv} and renv version causes bootstrap problems #18