search

rstudio / packrat

Packrat is a dependency management system for R
http://rstudio.github.io/packrat/
401 stars 89 forks source link

"Error in !packrat::opts$symlink.system.packages() : invalid argument type" #524

Open slhck opened 5 years ago

slhck commented 5 years ago

I have recently upgraded R from 3.5.1 to 3.5.2 with Homebrew, which means that existing package installs were no longer available.

I have an installation script for my project, which looks like this:

➜ cat requirements.R
#!/usr/bin/env Rscript
install.packages("packrat")
library(packrat)
packrat::restore()

When I run it:

➜ Rscript requirements.R
Installing package into ‘/Users/werner/Documents/Projects/test/data-analyze/rscript/packrat/lib/x86_64-apple-darwin17.7.0/3.5.2’
(as ‘lib’ is unspecified)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  123k  100  123k    0     0   225k      0 --:--:-- --:--:-- --:--:--  224k
* installing *source* package ‘packrat’ ...
** package ‘packrat’ successfully unpacked and MD5 sums checked
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (packrat)

The downloaded source packages are in
    ‘/private/var/folders/1s/9jbdrzg17wvcnr76p_ntdnbw0000gn/T/RtmprkwTTu/downloaded_packages’

The following packages were not installed by packrat and will be ignored:
            _
    packrat   0.4.9-2

If you would like to overwrite them, call restore again with
overwrite.dirty = TRUE.
Installing BH (1.66.0-1) ...
    OK (built source)
...
Installing ggmap (2.6.1) ...
Failed to extract archive:
- '/Users/werner/Documents/Projects/test/data-analyze/rscript/packrat/src/ggmap/ggmap_2.6.1.tar.gz' => '/var/folders/1s/9jbdrzg17wvcnr76p_ntdnbw0000gn/T//RtmprkwTTu'
Reason: incomplete block on file

Error in if (!file.exists(file.path(pkg_path, "DESCRIPTION"))) { :
  argument is of length zero
Calls: <Anonymous> ... installPkg -> <Anonymous> -> install_local_path -> lapply -> FUN
In addition: Warning message:
In packrat::restore() :
  The most recent snapshot was generated using R version 3.5.1
Execution halted

So, for some reason it halted. Now when I try to run it again, I get:

➜ Rscript requirements.R
Error in !packrat::opts$symlink.system.packages() : invalid argument type
Calls: source ... setPackratModeOn -> afterPackratModeOn -> symlinkSystemPackages
Execution halted

I would think that there's a bug somewhere that leads to the current project being broken due to an incomplete installation.

I am not sure what I'm supposed to do here … what are the steps necessary to revert properly?

Some debugging info:

➜ /usr/bin/env Rscript --version
R scripting front-end version 3.5.2 (2018-12-20)

➜ R

R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.7.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Error in !packrat::opts$symlink.system.packages() : invalid argument type
[Previously saved workspace restored]
slhck commented 5 years ago

Here's what I did to resolve the broken state:

Here's the log:

R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.7.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[Workspace loaded from ~/Documents/Projects/test/data-analyze/rscript/.RData]

Packrat is not installed in the local library -- attempting to bootstrap an installation...
> No source tarball of packrat available locally
> Using user-library packrat (0.4.9.3) to bootstrap this project
Packrat mode on. Using library in directory:
- "~/Documents/Projects/test/data-analyze/rscript/packrat/lib"
> library(tidyverse, warn.conflicts = FALSE)
Error in library(tidyverse, warn.conflicts = FALSE) : 
  there is no package called ‘tidyverse’
> install.packages("packrat")
Installing package into ‘/Users/werner/Documents/Projects/test/data-analyze/rscript/packrat/lib/x86_64-apple-darwin17.7.0/3.5.2’
(as ‘lib’ is unspecified)
trying URL 'https://ftp.gwdg.de/pub/misc/cran/src/contrib/packrat_0.5.0.tar.gz'
Content type 'application/octet-stream' length 126837 bytes (123 KB)
==================================================
downloaded 123 KB

* installing *source* package ‘packrat’ ...
** package ‘packrat’ successfully unpacked and MD5 sums checked
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (packrat)

The downloaded source packages are in
    ‘/private/var/folders/1s/9jbdrzg17wvcnr76p_ntdnbw0000gn/T/RtmpfH4F9X/downloaded_packages’
> library(packrat)
> chooseCRANmirror()
Secure CRAN mirrors 

 1: 0-Cloud [https]                   2: Algeria [https]                   3: Australia (Canberra) [https]      4: Australia (Melbourne 1) [https]
 5: Australia (Melbourne 2) [https]   6: Australia (Perth) [https]         7: Austria [https]                   8: Belgium (Ghent) [https]        
 9: Brazil (PR) [https]              10: Brazil (RJ) [https]              11: Brazil (SP 1) [https]            12: Brazil (SP 2) [https]          
13: Bulgaria [https]                 14: Chile 1 [https]                  15: Chile 2 [https]                  16: China (Hong Kong) [https]      
17: China (Guangzhou) [https]        18: China (Lanzhou) [https]          19: China (Shanghai 1) [https]       20: China (Shanghai 2) [https]     
21: Colombia (Cali) [https]          22: Czech Republic [https]           23: Denmark [https]                  24: East Asia [https]              
25: Ecuador (Cuenca) [https]         26: Ecuador (Quito) [https]          27: Estonia [https]                  28: France (Lyon 1) [https]        
29: France (Lyon 2) [https]          30: France (Marseille) [https]       31: France (Montpellier) [https]     32: France (Paris 2) [https]       
33: Germany (Erlangen) [https]       34: Germany (Göttingen) [https]      35: Germany (Münster) [https]        36: Greece [https]                 
37: Iceland [https]                  38: India [https]                    39: Indonesia (Jakarta) [https]      40: Ireland [https]                
41: Italy (Padua) [https]            42: Japan (Tokyo) [https]            43: Japan (Yonezawa) [https]         44: Korea (Busan) [https]          
45: Korea (Gyeongsan-si) [https]     46: Korea (Seoul 1) [https]          47: Korea (Ulsan) [https]            48: Malaysia [https]               
49: Mexico (Mexico City) [https]     50: Norway [https]                   51: Philippines [https]              52: Serbia [https]                 
53: Spain (A Coruña) [https]         54: Spain (Madrid) [https]           55: Sweden [https]                   56: Switzerland [https]            
57: Turkey (Denizli) [https]         58: Turkey (Mersin) [https]          59: UK (Bristol) [https]             60: UK (London 1) [https]          
61: USA (CA 1) [https]               62: USA (IA) [https]                 63: USA (KS) [https]                 64: USA (MI 1) [https]             
65: USA (NY) [https]                 66: USA (OR) [https]                 67: USA (TN) [https]                 68: USA (TX 1) [https]             
69: Uruguay [https]                  70: Vietnam [https]                  71: (other mirrors)                  

Selection: 33
> packrat::restore()

The following packages were not installed by packrat and will be ignored:
            _        
    packrat   0.4.9-2

If you would like to overwrite them, call restore again with
overwrite.dirty = TRUE.
Installing BH (1.66.0-1) ...
...

(It finishes eventually.)

My assumption is that there are some cleanup steps that aren't run when packrat aborts during installation, which means that it does not fall back to a state where packrat simply isn't available.

slhck commented 5 years ago

I could reproduce this on a completely new system with no previous R installation. I installed R 3.5 through apt via the CRAN repositories for Ubuntu cosmic, then:

R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Packrat is not installed in the local library -- attempting to bootstrap an installation...
> No source tarball of packrat available locally
> Installing packrat from CRAN
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
Warning in install.packages("packrat") :
  'lib = "/usr/local/lib/R/site-library"' is not writable
Would you like to use a personal library instead? (yes/No/cancel) yes
Would you like to create a personal library
‘~/R/x86_64-pc-linux-gnu-library/3.5’
to install packages into? (yes/No/cancel) yes
trying URL 'https://cloud.r-project.org/src/contrib/packrat_0.5.0.tar.gz'
Content type 'application/x-gzip' length 126837 bytes (123 KB)
==================================================
downloaded 123 KB

* installing *source* package ‘packrat’ ...
** package ‘packrat’ successfully unpacked and MD5 sums checked
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (packrat)

The downloaded source packages are in
    ‘/tmp/Rtmprd1cpg/downloaded_packages’
Error in !packrat::opts$symlink.system.packages() : invalid argument type

… wait, could this be related to https://github.com/rstudio/packrat/issues/496?

nacnudus commented 5 years ago

I get the same error.

slhck commented 5 years ago

Try:

rm -rf packrat/src packrat/lib*

Then open R:

install.packages("devtools")
devtools::install_github("rstudio/packrat")
library(packrat)
packrat::set_opts(symlink.system.packages = TRUE)
packrat::restore()

Or, if the above does not work, manually fix your packrat.opts:

echo "symlink.system.packages: TRUE" >> packrat/packrat.opts

This did the trick for me (hopefully).

testlnord commented 5 years ago

The trick here is that symlink.system.packages has no default value. There is no such variable in my packrat.opts file, so it's not set and packrat::opts$symlink.system.packages(FALSE) returns NULL. The NULL is an invalid argument for the ! function. And we get this error.

sambrightman commented 5 years ago

Why is this closed - isn't the absence of a default a bug in Packrat that needs fixing?

slhck commented 5 years ago

@sambrightman AFAICT there is one here: https://github.com/rstudio/packrat/blob/da40b6e553ce397216472c13d8a6bbb360682149/R/options.R#L47

sambrightman commented 5 years ago

See the comment from @testlnord above. Maybe rephrase the problem as "the failure to merge defaults for new options with the existing configuration file".

kevinushey commented 5 years ago

Is this not fixed in the development version? https://github.com/rstudio/packrat/commit/da40b6e553ce397216472c13d8a6bbb360682149

sambrightman commented 5 years ago

I'm only using released versions; that does indeed look like a fix.

nick-youngblut commented 5 years ago

I'm seemingly getting the same error as reported above when I did the following:

  1. created an Rstudio project that uses packrat, with the R install being Microsoft R Open v3.5.1
  2. installing some packages from the Microsoft R Open snapshot
  3. realizing that many of the packages in the repo are too old
  4. changing the repo via options(repos = c(CRAN = "https://cran.revolutionanalytics.com")) as suggested at https://mran.microsoft.com/faq
  5. trying to update the packages with the different repo

The specific error log is: 

[...]
Error in !packrat::opts$symlink.system.packages() : invalid argument type
Calls: source ... setPackratModeOn -> afterPackratModeOn -> symlinkSystemPackages
Execution halted
Warning in install.packages :
  installation of package ‘mgcv’ had non-zero exit status
Error in !packrat::opts$symlink.system.packages() : invalid argument type
Calls: source ... setPackratModeOn -> afterPackratModeOn -> symlinkSystemPackages
Execution halted
Warning in install.packages :
  installation of package ‘survival’ had non-zero exit status

The downloaded source packages are in
    ‘/tmp/Rtmp4Yxb8s/downloaded_packages’
Updating HTML index of packages in '.Library'
Warning in install.packages :
  cannot create file '/opt/microsoft/ropen/3.5.1/lib64/R/doc/html/packages.html', reason 'Permission denied'
Warning in install.packages :
  cannot update HTML package index

sessionInfo:

R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.1 LTS

Matrix products: default
BLAS: /opt/microsoft/ropen/3.5.1/lib64/R/lib/libRblas.so
LAPACK: /opt/microsoft/ropen/3.5.1/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RevoUtils_11.0.1     RevoUtilsMath_11.0.0

loaded via a namespace (and not attached):
[1] compiler_3.5.1 tools_3.5.1    packrat_0.5.0
alunap commented 4 years ago

very odd, I never had that issue until I started using Anaconda to start RStudio. Now, in RStudio

packrat::set_opts(symlink.system.packages = TRUE) does indeed fix it.