search

rstudio / packrat

Packrat is a dependency management system for R
http://rstudio.github.io/packrat/
401 stars 89 forks source link

packrat::restore deletes sources then complains it can't find them #560

Open torbjorn opened 5 years ago

torbjorn commented 5 years ago

I'm experiencing odd behavior.

I using packrat to control the production R environment, but deploying to production and calling restore does strange things. At first go it removes sources of the package I want to restore. It then complains it can't find the sources.

I do the following:

  1. packrat::install_local("ga.data") # on development system
  2. rsync the packrat dir using --delete, over to the production environment's packrat dir
  3. start R in production environment's packrat dir
  4. packrat::restore() # on production system
  5. go back to repeat the rsync
  6. restore again, this time it works.

Odd. In my setup this is reproducing every time.

Here is the text output from a terminal as I go through the steps:

    ## After rsync of development system to the production system, sources
    ##   of a particular package, ga.data, looks like this:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ ll packrat/src/ga.data/
    total 6492
    -rw-rw-r--+ 1 tl tl 1070060 May  9 10:14 ga.data_1.3.7-10.tar.gz
    -rw-r--r--+ 1 tl tl 1070072 May  9 10:14 ga.data_1.3.7-24.tar.gz
    -rw-r--r--+ 1 tl tl 1070288 May  9 10:14 ga.data_1.3.7-30.tar.gz
    -rw-r--r--+ 1 tl tl 1131170 May  9 10:14 ga.data_1.3.8-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131817 May  9 10:14 ga.data_1.3.9-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131821 May 24 13:39 ga.data_1.3.9-5.tar.gz

    ## entering R to restore - it complains it can't find 1.3.9_5

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ R -q
    Packrat mode on. Using library in directory:
    - "/var/www/profileplotter/perl/R_Home/packrat/lib"
    > packrat::restore()
    Replacing ga.data (upgrade 1.3.9-3 to 1.3.9-5) ... Error in installPkg(pkgRecord, project, repos, lib) : 
      Failed to install ga.data (1.3.9-5): sources missing at /var/www/profileplotter/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-5.tar.gz
    In addition: Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    > packrat::restore()
    Installing ga.data (1.3.9-5) ... 
    Error in installPkg(pkgRecord, project, repos, lib) : 
      Failed to install ga.data (1.3.9-5): sources missing at /var/www/profileplotter/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-5.tar.gz
    In addition: Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    > 
    Save workspace image? [y/n/c]:

    ## And indeed, now that particular source (and a bunch of other sources is gone)

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ ll packrat/src/ga.data/
    total 1108
    -rw-rw-r--+ 1 tl tl 1129979 May 24 13:42 ga.data_1.3.9-1.tar.gz

    ## I repeat rsync the packrat dir from development system, now the sources are back again:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ ll packrat/src/ga.data/
    total 6492
    -rw-rw-r--+ 1 tl tl 1070060 May 24 13:42 ga.data_1.3.7-10.tar.gz
    -rw-r--r--+ 1 tl tl 1070072 May 24 13:42 ga.data_1.3.7-24.tar.gz
    -rw-r--r--+ 1 tl tl 1070288 May 24 13:42 ga.data_1.3.7-30.tar.gz
    -rw-r--r--+ 1 tl tl 1131170 May 24 13:42 ga.data_1.3.8-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131817 May 24 13:42 ga.data_1.3.9-3.tar.gz
    -rw-r--r--+ 1 tl tl 1131821 May 24 13:42 ga.data_1.3.9-5.tar.gz

    ## And this time restore works:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$ R -q
    Packrat mode on. Using library in directory:
    - "/var/www/profileplotter/perl/R_Home/packrat/lib"
    p> packrat::restore()
    Installing ga.data (1.3.9-5) ... 
            OK (built source)
    Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    > 
    Save workspace image? [y/n/c]:

    tl@gamap(/var/www/profileplotter/perl/R_Home)$
kevinushey commented 5 years ago

To the best of my knowledge, Packrat won't ever try to automatically remove any downloaded source packages from the src folder. Of course, I could be wrong.

Is there any chance you could distill this into a small, reproducible example? E.g. using a snippet of code that I could run on my own machine to reproduce the problem.

torbjorn commented 5 years ago

I tried creating a minimal setup with only a locally installed package. That did not reproduce this behaviour.

I did notice however that after a couple of iterations, my locally installed package has multiple source tar balls on the dev system under packrat/src, while on the prod system, after restore, there is only the one current tar ball. Even if I rsync'ed them all over.

Could that be at play above? The mechanism which removes sources not used or needed?

torbjorn commented 5 years ago

I had another stab at this one, this time running the ::restore with strace -e trace=unlink, below is what unfolded. As you can see, right under Replacing ga.data (upgrade 1.3.9-5 to 1.3.9-9) ... it goes on to delete all the sources, including the one it should upgrade to. Once I run rsync again it does it just fine. Is it picky on file mtimes? Could there be slight time sync differences between two systems that causes this?

    strace -e unlink R -e 'packrat::restore()'
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28943, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28944, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28947, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    unlink("/tmp/tmpfnBdNQq")               = 0

    R version 3.5.1 (2018-07-02) -- "Feather Spray"
    Copyright (C) 2018 The R Foundation for Statistical Computing
    Platform: x86_64-redhat-linux-gnu (64-bit)

    R is free software and comes with ABSOLUTELY NO WARRANTY.
    You are welcome to redistribute it under certain conditions.
    Type 'license()' or 'licence()' for distribution details.

      Natural language support but running in an English locale

    R is a collaborative project with many contributors.
    Type 'contributors()' for more information and
    'citation()' on how to cite R or R packages in publications.

    Type 'demo()' for some demos, 'help()' for on-line help, or
    'help.start()' for an HTML browser interface to help.
    Type 'q()' to quit R.

    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28951, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    > packrat::restore()
    unlink("/tmp/RtmplM6VjB/file710e4deaa62b") = 0
    unlink("/tmp/RtmplM6VjB/file710e1807a70a") = 0
    unlink("/tmp/RtmplM6VjB/file710e434ebd67") = 0
    unlink("/tmp/RtmplM6VjB/file710e7bcf92b5") = 0
    unlink("/tmp/RtmplM6VjB/file710e4bdebb10") = 0
    unlink("/tmp/RtmplM6VjB/file710e40ff47a") = 0
    unlink("/tmp/RtmplM6VjB/file710e2a75150e") = 0
    unlink("/tmp/RtmplM6VjB/file710e74ce4d51") = 0
    unlink("/tmp/RtmplM6VjB/file710e1aee383a") = 0
    unlink("/tmp/RtmplM6VjB/file710e4a30f515") = 0
    unlink("/tmp/RtmplM6VjB/file710e5f217cdf") = 0
    unlink("/tmp/RtmplM6VjB/file710e27bc877c") = 0
    unlink("/tmp/RtmplM6VjB/file710e1b108ab5") = 0
    unlink("/tmp/RtmplM6VjB/file710e6aa25025") = 0
    unlink("/tmp/RtmplM6VjB/file710e566db79b") = 0
    unlink("/tmp/RtmplM6VjB/file710e31a5eb87") = 0
    unlink("/tmp/RtmplM6VjB/file710e779f773e") = 0
    unlink("/tmp/RtmplM6VjB/file710e348d3edb") = 0
    unlink("/tmp/RtmplM6VjB/file710e3468d6e") = 0
    unlink("/tmp/RtmplM6VjB/file710e504a6836") = 0
    unlink("/tmp/RtmplM6VjB/file710e7125aa80") = 0
    unlink("/tmp/RtmplM6VjB/file710e2990a8ec") = 0
    unlink("/tmp/RtmplM6VjB/file710e3eb30f20") = 0
    unlink("/tmp/RtmplM6VjB/file710e39cd47f4") = 0
    unlink("/tmp/RtmplM6VjB/file710e15359ca4") = 0
    unlink("/tmp/RtmplM6VjB/file710e4fbe396f") = 0
    unlink("/tmp/RtmplM6VjB/file710e7ab1ac9e") = 0
    unlink("/tmp/RtmplM6VjB/file710e33651309") = 0
    unlink("/tmp/RtmplM6VjB/file710e33357736") = 0
    unlink("/tmp/RtmplM6VjB/file710e2d85800c") = 0
    unlink("/tmp/RtmplM6VjB/file710e19dff764") = 0
    unlink("/tmp/RtmplM6VjB/file710e1201d61") = 0
    unlink("/tmp/RtmplM6VjB/file710e458d2716") = 0
    unlink("/tmp/RtmplM6VjB/file710e5d2eb4cb") = 0
    unlink("/tmp/RtmplM6VjB/file710e7cefb016") = 0
    unlink("/tmp/RtmplM6VjB/file710e116be227") = 0
    unlink("/tmp/RtmplM6VjB/file710e613ea945") = 0
    unlink("/tmp/RtmplM6VjB/file710e2764c525") = 0
    unlink("/tmp/RtmplM6VjB/file710e63a2f78") = 0
    unlink("/tmp/RtmplM6VjB/file710e7c2ce17f") = 0
    unlink("/tmp/RtmplM6VjB/file710e7195ba3a") = 0
    unlink("/tmp/RtmplM6VjB/file710e655bac57") = 0
    unlink("/tmp/RtmplM6VjB/file710e23e968fc") = 0
    unlink("/tmp/RtmplM6VjB/file710eca644f0") = 0
    unlink("/tmp/RtmplM6VjB/file710e4ffdfc7c") = 0
    unlink("/tmp/RtmplM6VjB/file710e7a572097") = 0
    unlink("/tmp/RtmplM6VjB/file710e3e4c3077") = 0
    unlink("/tmp/RtmplM6VjB/file710e479d73bb") = 0
    unlink("/tmp/RtmplM6VjB/file710e2ee45f72") = 0
    unlink("/tmp/RtmplM6VjB/file710e4192bde5") = 0
    unlink("/tmp/RtmplM6VjB/file710e17e7dbf1") = 0
    unlink("/tmp/RtmplM6VjB/file710e200a09f2") = 0
    unlink("/tmp/RtmplM6VjB/file710e6b2366d1") = 0
    unlink("/tmp/RtmplM6VjB/file710e569aeb11") = 0
    unlink("/tmp/RtmplM6VjB/file710e59d751e7") = 0
    unlink("/tmp/RtmplM6VjB/file710e590376") = 0
    unlink("/tmp/RtmplM6VjB/file710e26592480") = 0
    unlink("/tmp/RtmplM6VjB/file710e5488fe85") = 0
    unlink("/tmp/RtmplM6VjB/file710e33be167f") = 0
    unlink("/tmp/RtmplM6VjB/file710e598e9bb6") = 0
    unlink("/tmp/RtmplM6VjB/file710e20e7e92") = 0
    unlink("/tmp/RtmplM6VjB/file710e4d9e0de3") = 0
    unlink("/tmp/RtmplM6VjB/file710e5aaeb917") = 0
    unlink("/tmp/RtmplM6VjB/file710e479ba5a8") = 0
    unlink("/tmp/RtmplM6VjB/file710e2accc2ae") = 0
    unlink("/tmp/RtmplM6VjB/file710e579e692e") = 0
    unlink("/tmp/RtmplM6VjB/file710e590787cf") = 0
    unlink("/tmp/RtmplM6VjB/file710ec0b6bf3") = 0
    unlink("/tmp/RtmplM6VjB/file710e7f032e53") = 0
    unlink("/tmp/RtmplM6VjB/file710e5f41b747") = 0
    unlink("/tmp/RtmplM6VjB/file710e8384d73") = 0
    unlink("/tmp/RtmplM6VjB/file710e7098e88d") = 0
    unlink("/tmp/RtmplM6VjB/file710e449d639e") = 0
    unlink("/tmp/RtmplM6VjB/file710e2c21b66f") = 0
    unlink("/tmp/RtmplM6VjB/file710e7d3f2d7d") = 0
    unlink("/tmp/RtmplM6VjB/file710e149b601b") = 0
    unlink("/tmp/RtmplM6VjB/file710e2678d706") = 0
    unlink("/tmp/RtmplM6VjB/file710e3b8b5df5") = 0
    unlink("/tmp/RtmplM6VjB/file710e5c38d3d6") = 0
    unlink("/tmp/RtmplM6VjB/file710e555d3678") = 0
    unlink("/tmp/RtmplM6VjB/file710e7d1e1bda") = 0
    unlink("/tmp/RtmplM6VjB/file710e7420afc7") = 0
    unlink("/tmp/RtmplM6VjB/file710e7567406b") = 0
    unlink("/tmp/RtmplM6VjB/file710e684182ac") = 0
    unlink("/tmp/RtmplM6VjB/file710e4abb9ad9") = 0
    unlink("/tmp/RtmplM6VjB/file710e4f3e9252") = 0
    unlink("/tmp/RtmplM6VjB/file710e689a8622") = 0
    unlink("/tmp/RtmplM6VjB/file710e7114bf59") = 0
    unlink("/tmp/RtmplM6VjB/file710e23c790d7") = 0
    unlink("/tmp/RtmplM6VjB/file710e1c589ca1") = 0
    unlink("/tmp/RtmplM6VjB/file710e4aa35b10") = 0
    unlink("/tmp/RtmplM6VjB/file710e25d60f69") = 0
    unlink("/tmp/RtmplM6VjB/file710e69f6aa85") = 0
    unlink("/tmp/RtmplM6VjB/file710e25521427") = 0
    unlink("/tmp/RtmplM6VjB/file710e6d71b512") = 0
    unlink("/tmp/RtmplM6VjB/file710e14c36d33") = 0
    unlink("/tmp/RtmplM6VjB/file710e7cf07d55") = 0
    unlink("/tmp/RtmplM6VjB/file710e46793ce1") = 0
    unlink("/tmp/RtmplM6VjB/file710e20ced927") = 0
    unlink("/tmp/RtmplM6VjB/file710e7bf3aba8") = 0
    unlink("/tmp/RtmplM6VjB/file710e25baf429") = 0
    unlink("/tmp/RtmplM6VjB/file710e2907269a") = 0
    unlink("/tmp/RtmplM6VjB/file710e6c8c9436") = 0
    unlink("/tmp/RtmplM6VjB/file710e6a5857c7") = 0
    unlink("/tmp/RtmplM6VjB/file710e5528dd09") = 0
    unlink("/tmp/RtmplM6VjB/file710e69cbc1b3") = 0
    unlink("/tmp/RtmplM6VjB/file710e7ef3b7e2") = 0
    unlink("/tmp/RtmplM6VjB/file710e7ba1b40f") = 0
    unlink("/tmp/RtmplM6VjB/libloc_228_ccadf3d65a81c52b.rds") = -1 ENOENT (No such file or directory)
    Replacing ga.data (upgrade 1.3.9-5 to 1.3.9-9) ... unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-5.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.7-30.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.7-20.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-9.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.7-24.tar.gz") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/biobank_db.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/kitlot_mapping.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_probe_levels.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_lx200_probe_diff.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_probe_categories.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_model_pca.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_kitlot_map.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_customer_bacteria_names.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/data/bacteria_limits_table_revision2.RData") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/data/bacteria_limits_table_revision3.RData") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_kitlots.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_genus_annotations.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_bacteria_names.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_probe_set.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_batch_correction.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat/test_model_data.r") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/tests/testthat.R") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/ga.data.rdx") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/AnIndex") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/ga.data.rdb") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/aliases.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/help/paths.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/links.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/hsearch.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/package.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/nsInfo.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/Rd.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/Meta/features.rds") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/bacteria_db.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/probe_categories.json") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/INDEX") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/html/R.css") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/html/00Index.html") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/DESCRIPTION") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/NAMESPACE") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/ga.data.rdx") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/sysdata.rdx") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/ga.data") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/ga.data.rdb") = 0
    unlink("/var/www/ga-biobank/perl/R_Home/packrat/lib/x86_64-redhat-linux-gnu/3.5.1/ga.data/R/sysdata.rdb") = 0
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28960, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    Error in installPkg(pkgRecord, project, repos, lib) : 
      Failed to install ga.data (1.3.9-9): sources missing at /var/www/ga-biobank/perl/R_Home/packrat/src/ga.data/ga.data_1.3.9-9.tar.gz
    Calls: <Anonymous> -> restoreImpl -> playActions -> installPkg
    In addition: Warning message:
    In packrat::restore() :
      The most recent snapshot was generated using R version 3.5.3
    Execution halted
    --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28971, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
    +++ exited with 1 +++
kevinushey commented 5 years ago

Thanks -- I think I understand what's going on now.

When a package is upgraded / downgraded, Packrat attempts to remove the old version of the package, and then install the new one. Part of that implies calling removePkgs(), e.g.

https://github.com/rstudio/packrat/blob/ffd810657948739d1e0d7c364d2b97dbe093633b/R/restore.R#L812

and that will also imply removing all cached sources for that package:

https://github.com/rstudio/packrat/blob/ffd810657948739d1e0d7c364d2b97dbe093633b/R/restore.R#L597-L601

It looks like this was intentional:

https://github.com/rstudio/packrat/issues/19

but in hind-sight it seems over-aggressive. I don't think we should remove the package sources during a restore; they should only be cleaned up manually (e.g. by the user or by a smart-enough clean() function)

torbjorn commented 5 years ago

Ok, then.

So is my chosen route for deploying my R based app wrong then?

My production system typically doesn't have access to a checked out dev. directory of package sources that my development system has, and from which it can install_local, and I'd be hesitant to provide it.

The other R-package-bundle approach is definitely an over kill. We make incremental updates to select few packages, recreating and reinstalling the entire bundle would be an insane waste of time for a small update. If I understand that approach right.

But why does it work the other time around? All I have to do, annoying as it is, is to have the restore() process remove the sources, complain loudly that they can't be found after that, then rsync them back, and run restore again, and this time its super happy.

The intended pattern for how this should work may or may not be optimal (goes without saying I think it isn't), but it ought to be consistent.

kevinushey commented 5 years ago

I don't think you're doing anything wrong. Packrat seems to have some unexpected / sub-optimal behavior here, and I think it should work without the extra orchestration that seems to be required in your case. I'll plan to fix this in Packrat directly.