RevolutionAnalytics / checkpoint

Install R packages from snapshots on checkpoint-server
164 stars 38 forks source link

error: 'arguments imply differing number of rows' when sub-dependencies being installed from local MRAN #283

Closed mungojam closed 4 years ago

mungojam commented 4 years ago

When running

checkpoint(
          librarySnapshot,
          project = project,
          R.version = R.version,
          checkpointLocation = checkpointLocation,verbose = TRUE
        )

we get this error:

Error in data.frame(timestamp = Sys.time(), snapshotDate = snapshotDate, : arguments imply differing number of rows: 1, 18, 0

It is happening in this bit of code:

https://github.com/RevolutionAnalytics/checkpoint/blob/233634f25764cec5a8e872d60e693f8045db88f0/R/checkpoint_log.R#L16-L23

And I believe it happens because the install.packages message is in a format that the interpreter isn't expecting.

Here is what the message ends up looking like:

Browse[4]> log [1] "also installing the dependencies ‘backports’, ‘ellipsis’, ‘digest’, ‘zeallot’, ‘utf8’, ‘vctrs’, ‘fansi’, ‘pillar’, ‘purrr’, ‘glue’, ‘pkgconfig’, ‘Rcpp’, ‘rlang’, ‘tibble’, ‘tidyselect’, ‘BH’, ‘plogr’" [2] ""
[3] "installing the source packages ‘pillar’, ‘pkgconfig’"
[4] ""

This leads to the byte count being a zero length character string:

Browse[5]> extract_bytes(log) character(0)

which then fails when combined with the other columns:

Browse[5]> data.frame(timestamp = Sys.time(), snapshotDate = snapshotDate, + pkg = c(extract_pkgs(log), pkg), bytes = extract_bytes(log), + stringsAsFactors = FALSE, check.names = FALSE) Error in data.frame(timestamp = Sys.time(), snapshotDate = snapshotDate, : arguments imply differing number of rows: 1, 18, 0

The other columns are as follows:

Browse[5]> snapshotDate [1] "2019-11-01"

Browse[5]> c(extract_pkgs(log), pkg) [1] "backports" "ellipsis" "digest" "zeallot" "utf8"
[6] "vctrs" "fansi" "pillar" "purrr" "glue"
[11] "pkgconfig" "Rcpp" "rlang" "tibble" "tidyselect" [16] "BH" "plogr" "dplyr"

It looks like the code tries to detect something similar but maybe just needs an extra check for the case where there are both binary and source packages:

https://github.com/RevolutionAnalytics/checkpoint/blob/233634f25764cec5a8e872d60e693f8045db88f0/R/checkpoint_log.R#L14

hongooi73 commented 4 years ago

A lot of these issues stem from the shortcomings of install.packages, specifically, there is no way other than screen-scraping to detect the outcome of an install. I'm hoping to switch the backend to using the pkgdepends package, which will hopefully be a much more robust and flexible solution.

This may take a while though, pkgdepends is not yet on CRAN and I'm also pressed for spare cycles at the moment.

280

hongooi73 commented 4 years ago

Unfortunately the current pkgcache/pkgdepends framework doesn't support local CRAN. Hopefully this will be addressed before pkgdepends leaves beta.

hongooi73 commented 4 years ago

@mungojam can you give the v1.0 checkpoint a go? Unfortunately it doesn't support a local MRAN yet, but you can try it with the regular MRAN site.

mungojam commented 4 years ago

Thanks for fixing, I don't really use R these days so I'd struggle to recreate. My old team also have to use an internal Cran so I imagine they'd need support for that first anyway