search

r-lib / remotes

Install R packages from GitHub, GitLab, Bitbucket, git, svn repositories, URLs
https://remotes.r-lib.org/
Other
331 stars 152 forks source link

Bioconductor Local installation misclassified source #702

Closed muschellij2 closed 10 months ago

muschellij2 commented 2 years ago

Tagging @jennybc for the devtools component of this.

Overall the issue occurs when you have a local installation of a package that has a biocViews tag, typically from devtools, that the source of the package is "misattributed". This is important for containerization and reproducible environment creation.

One very edge use case would be someone working on a local package that is intended for bioC, hence may have biocViews, but is not on there. Or a less edgy of an edge case is someone is working a local copy of a Bioc package, typically a local dev copy on their machine that is past the BioC version. The intended behavior would mark this as a local package, rather than a BioC one. If you use remotes::install_local appropriate markup is added using the add_metadata function from https://github.com/r-lib/remotes/blob/main/R/install-remote.R#L68

library(git2r)
library(desc)
library(sessioninfo)

Making a package

Here we just make a fake description file with biocViews in there. THis is local.

pack_dir = tempfile()
dir.create(pack_dir, showWarnings = FALSE, recursive = TRUE)

pkg = "tmp.delete"
description = desc::desc("!new")
description$set("Package", pkg)
description$set("License", "GPL-2")
description$set("Title", "A Dev Package")
description$set("Description", "A Dev Package to check.")
auth = description$get_authors()
auth = trimws(sub("\\[.*", "", auth))
auth
#> [1] "Jo Doe <jodoe@dom.ain>"
description$set("Maintainer", auth)
description$set("biocViews", "Infrastructure")
description$write(file.path(pack_dir, "DESCRIPTION"))
print(description)
#> Package: tmp.delete
#> Title: A Dev Package
#> Version: 1.0.0
#> Authors@R (parsed):
#>     * Jo Doe <jodoe@dom.ain> [aut, cre]
#> Maintainer: Jo Doe <jodoe@dom.ain>
#> Description: A Dev Package to check.
#> License: GPL-2
#> URL: {{ URL }}
#> BugReports: {{ BugReports }}
#> biocViews: Infrastructure
#> Encoding: UTF-8

Make a dependency

Here we are making a file with library(tmp.delete) so that renv::snapshot picks it up

tdir = tempfile()
dir.create(tdir, showWarnings = FALSE, recursive = TRUE)
tfile = tempfile(fileext = ".R", tmpdir = tdir)
writeLines(paste0("library(", pkg, ")"), tfile)

Using a Local Install

This is shows a local installation

Insalling using devtools::install

devtools::install(pack_dir)
#> * checking for file ‘/tmp/RtmpHxKoPP/file12edf44a302bb1/DESCRIPTION’ ... OK
#> * preparing ‘tmp.delete’:
#> * checking DESCRIPTION meta-information ... OK
#> * checking for LF line-endings in source and make files and shell scripts
#> * checking for empty or unneeded directories
#> * creating default NAMESPACE file
#> * building ‘tmp.delete_1.0.0.tar.gz’
#> 
#> Running /usr/local/lib/R/bin/R CMD INSTALL \
#>   /tmp/RtmpHxKoPP/tmp.delete_1.0.0.tar.gz --install-tests 
#> * installing to library ‘/usr/local/lib/R/site-library’
#> * installing *source* package ‘tmp.delete’ ...
#> ** using staged installation
#> ** help
#> No man pages found in package  ‘tmp.delete’ 
#> *** installing help indices
#> ** building package indices
#> ** testing if installed package can be loaded from temporary location
#> ** testing if installed package can be loaded from final location
#> ** testing if installed package keeps a record of temporary installation path
#> * DONE (tmp.delete)

Here we print out the description file that’s in the .libPaths() so taht we cna see if any markup was added (nothing other than Built).

print(sessioninfo:::pkg_desc(pkg, lib.loc = .libPaths()))
#> Package: tmp.delete
#> Title: A Dev Package
#> Version: 1.0.0
#> Authors@R: c(person(given = "Jo", family = "Doe", email =
#>         "jodoe@dom.ain", role = c("aut", "cre")))
#> Maintainer: Jo Doe <jodoe@dom.ain>
#> Description: A Dev Package to check.
#> License: GPL-2
#> URL: {{ URL }}
#> BugReports: {{ BugReports }}
#> Encoding: UTF-8
#> biocViews: Infrastructure
#> NeedsCompilation: no
#> Packaged: 2022-03-31 19:51:19 UTC; rstudio
#> Author: Jo Doe [aut, cre]
#> Built: R 4.1.1; ; 2022-03-31 19:51:20 UTC; unix
#> 
#> -- File: /usr/local/lib/R/site-library/tmp.delete/Meta/package.rds

We see BIoconductor is the source from package_info, but it does not exist there, so renv/session_info code that tries to use this for a container of packages will fail

sessioninfo::package_info(pkg)
#>  package    * version date (UTC) lib source
#>  tmp.delete   1.0.0   2022-03-31 [1] Bioconductor
#> 
#>  [1] /usr/local/lib/R/site-library
#>  [2] /usr/local/lib/R/library

Trying renv for the dependency

renv::snapshot(project = tdir, prompt = FALSE)
#> Error in vapply(bioc$Package, function(package) {: values must be length 1,
#>  but FUN(X[[1]]) result is length 0
readLines(file.path(tdir, "renv.lock"))
#> Warning in file(con, "r"): cannot open file '/tmp/RtmpHxKoPP/file12edf43a7aedea/
#> renv.lock': No such file or directory
#> Error in file(con, "r"): cannot open the connection

remove.packages(pkg)
#> Removing package from '/usr/local/lib/R/site-library'
#> (as 'lib' is unspecified)

Insalling using remotes::install_local

Different behavior in remotes as it adds the tags to the DESCRIPTION file

remotes::install_local(pack_dir)
#> * checking for file ‘/tmp/RtmpHxKoPP/file12edf45dc41e4a/file12edf44a302bb1/DESCRIPTION’ ... OK
#> * preparing ‘tmp.delete’:
#> * checking DESCRIPTION meta-information ... OK
#> * checking for LF line-endings in source and make files and shell scripts
#> * checking for empty or unneeded directories
#> * creating default NAMESPACE file
#> * building ‘tmp.delete_1.0.0.tar.gz’
#> Installing package into '/usr/local/lib/R/site-library'
#> (as 'lib' is unspecified)
print(sessioninfo:::pkg_desc(pkg, lib.loc = .libPaths()))
#> Package: tmp.delete
#> Title: A Dev Package
#> Version: 1.0.0
#> Authors@R: c(person(given = "Jo", family = "Doe", email =
#>         "jodoe@dom.ain", role = c("aut", "cre")))
#> Maintainer: Jo Doe <jodoe@dom.ain>
#> Description: A Dev Package to check.
#> License: GPL-2
#> URL: {{ URL }}
#> BugReports: {{ BugReports }}
#> Encoding: UTF-8
#> biocViews: Infrastructure
#> RemoteType: local
#> RemoteUrl: /tmp/RtmpHxKoPP/file12edf44a302bb1
#> NeedsCompilation: no
#> Packaged: 2022-03-31 19:51:23 UTC; rstudio
#> Author: Jo Doe [aut, cre]
#> Built: R 4.1.1; ; 2022-03-31 19:51:23 UTC; unix
#> 
#> -- File: /usr/local/lib/R/site-library/tmp.delete/Meta/package.rds
sessioninfo::package_info(pkg)
#>  package    * version date (UTC) lib source
#>  tmp.delete   1.0.0   2022-03-31 [1] local (/tmp/RtmpHxKoPP/file12edf44a302bb1)
#> 
#>  [1] /usr/local/lib/R/site-library
#>  [2] /usr/local/lib/R/library

Trying renv for the dependency

renv::snapshot(project = tdir, prompt = FALSE)
#> * Lockfile written to '/tmp/RtmpHxKoPP/file12edf43a7aedea/renv.lock'.
readLines(file.path(tdir, "renv.lock"))
#>  [1] "{"                                                                  
#>  [2] "  \"R\": {"                                                         
#>  [3] "    \"Version\": \"4.1.1\","                                        
#>  [4] "    \"Repositories\": ["                                            
#>  [5] "      {"                                                            
#>  [6] "        \"Name\": \"CRAN\","                                        
#>  [7] "        \"URL\": \"https://packagemanager.rstudio.com/cran/latest\""
#>  [8] "      }"                                                            
#>  [9] "    ]"                                                              
#> [10] "  },"                                                               
#> [11] "  \"Packages\": {"                                                  
#> [12] "    \"renv\": {"                                                    
#> [13] "      \"Package\": \"renv\","                                       
#> [14] "      \"Version\": \"0.15.4\","                                     
#> [15] "      \"Source\": \"Repository\","                                  
#> [16] "      \"Repository\": \"RSPM\","                                    
#> [17] "      \"Hash\": \"c1078316e1d4f70275fc1ea60c0bc431\","              
#> [18] "      \"Requirements\": []"                                         
#> [19] "    },"                                                             
#> [20] "    \"tmp.delete\": {"                                              
#> [21] "      \"Package\": \"tmp.delete\","                                 
#> [22] "      \"Version\": \"1.0.0\","                                      
#> [23] "      \"Source\": \"Local\","                                       
#> [24] "      \"RemoteType\": \"local\","                                   
#> [25] "      \"RemoteUrl\": \"/tmp/RtmpHxKoPP/file12edf44a302bb1\","       
#> [26] "      \"Hash\": \"b71230110be5afc08c78cab0bd366ae2\","              
#> [27] "      \"Requirements\": []"                                         
#> [28] "    }"                                                              
#> [29] "  }"                                                                
#> [30] "}"

remove.packages(pkg)
#> Removing package from '/usr/local/lib/R/site-library'
#> (as 'lib' is unspecified)

Created on 2022-03-31 by the reprex package (v2.0.1)

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.1.1 (2021-08-10) #> os Ubuntu 20.04.3 LTS #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Etc/UTC #> date 2022-03-31 #> pandoc 2.14.0.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> BiocManager 1.30.16 2021-06-15 [1] RSPM (R 4.1.0) #> brio 1.1.3 2021-11-30 [1] RSPM (R 4.1.0) #> cachem 1.0.6 2021-08-19 [1] RSPM (R 4.1.0) #> callr 3.7.0 2021-04-20 [1] RSPM (R 4.1.0) #> cli 3.2.0 2022-02-14 [1] RSPM (R 4.1.0) #> crayon 1.5.1 2022-03-26 [1] RSPM (R 4.1.0) #> desc * 1.4.0 2021-09-28 [1] RSPM (R 4.1.0) #> devtools 2.4.3 2021-11-30 [1] RSPM (R 4.1.0) #> digest 0.6.29 2021-12-01 [1] RSPM (R 4.1.0) #> ellipsis 0.3.2 2021-04-29 [1] RSPM (R 4.1.0) #> evaluate 0.15 2022-02-18 [1] RSPM (R 4.1.0) #> fansi 1.0.3 2022-03-24 [1] RSPM (R 4.1.0) #> fastmap 1.1.0 2021-01-25 [1] RSPM (R 4.1.0) #> fs 1.5.2 2021-12-08 [1] RSPM (R 4.1.0) #> git2r * 0.29.0 2021-11-22 [1] RSPM (R 4.1.0) #> glue 1.6.2 2022-02-24 [1] RSPM (R 4.1.0) #> highr 0.9 2021-04-16 [1] RSPM (R 4.1.0) #> htmltools 0.5.2 2021-08-25 [1] RSPM (R 4.1.0) #> knitr 1.37 2021-12-16 [1] RSPM (R 4.1.0) #> lifecycle 1.0.1 2021-09-24 [1] RSPM (R 4.1.0) #> magrittr 2.0.2 2022-01-26 [1] RSPM (R 4.1.0) #> memoise 2.0.1 2021-11-26 [1] RSPM (R 4.1.0) #> pillar 1.7.0 2022-02-01 [1] RSPM (R 4.1.0) #> pkgbuild 1.3.1 2021-12-20 [1] RSPM (R 4.1.0) #> pkgconfig 2.0.3 2019-09-22 [1] RSPM (R 4.1.0) #> pkgload 1.2.4 2021-11-30 [1] RSPM (R 4.1.0) #> prettyunits 1.1.1 2020-01-24 [1] RSPM (R 4.1.0) #> processx 3.5.2 2021-04-30 [1] RSPM (R 4.1.0) #> ps 1.6.0 2021-02-28 [1] RSPM (R 4.1.0) #> purrr 0.3.4 2020-04-17 [1] RSPM (R 4.1.0) #> R6 2.5.1 2021-08-19 [1] RSPM (R 4.1.0) #> remotes 2.4.2 2021-11-30 [1] RSPM (R 4.1.0) #> renv 0.15.4 2022-03-03 [1] RSPM (R 4.1.0) #> reprex 2.0.1 2021-08-05 [1] RSPM (R 4.1.0) #> rlang 1.0.2 2022-03-04 [1] RSPM (R 4.1.0) #> rmarkdown 2.11 2021-09-14 [1] RSPM (R 4.1.0) #> rprojroot 2.0.2 2020-11-15 [1] RSPM (R 4.1.0) #> rstudioapi 0.13 2020-11-12 [1] RSPM (R 4.1.0) #> sessioninfo * 1.2.2.9000 2022-03-04 [1] Github (r-lib/sessioninfo@f971f10) #> stringi 1.7.6 2021-11-29 [1] RSPM (R 4.1.0) #> stringr 1.4.0 2019-02-10 [1] RSPM (R 4.1.0) #> testthat 3.1.2 2022-01-20 [1] RSPM (R 4.1.0) #> tibble 3.1.6 2021-11-07 [1] RSPM (R 4.1.0) #> usethis 2.1.5 2021-12-09 [1] RSPM (R 4.1.0) #> utf8 1.2.2 2021-07-24 [1] RSPM (R 4.1.0) #> vctrs 0.3.8 2021-04-29 [1] RSPM (R 4.1.0) #> withr 2.5.0 2022-03-03 [1] RSPM (R 4.1.0) #> xfun 0.30 2022-03-02 [1] RSPM (R 4.1.0) #> yaml 2.3.5 2022-02-21 [1] RSPM (R 4.1.0) #> #> [1] /usr/local/lib/R/site-library #> [2] /usr/local/lib/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```
gaborcsardi commented 2 years ago

Well, I have similar questions as for the sessioninfo issue. What is remotes doing wrong? Why? What do you think it should do instead? Thanks!

muschellij2 commented 2 years ago

Remotes isn't doing anything wrong, devtools has the issue. I went to put an issue on devtools, but it says to choose remotes issues if it has to do with package installation, so where should this really live?

image

gaborcsardi commented 2 years ago

I don't mind if it lives here. So the same questions apply to devtools then. Thanks!

gaborcsardi commented 10 months ago

Please reopen with more information if you still have this problem. I.e. what is remotes/devtools doing wrong? Why? What do you think it should do instead? Thanks!