search

Bioconductor / BiocManager

CRAN Package For Managing Bioconductor Packages
http://bioconductor.github.io/BiocManager/
73 stars 24 forks source link

BiocManager 1.30.13 not working with R 4.1 #100

Closed biobenkj closed 3 years ago

biobenkj commented 3 years ago

Hello Bioconductor team! I recently updated BiocManager from 1.30.12 to 1.30.13 and when I go to update packages via BiocManager::install("mypackage") I will either get an error with something like Error in readRDS(dest) or will not attempt to install/error and return the prompt. However, when rolling back to 1.30.12, it will install packages as expected. Is this expected behavior and the latest BiocManager is only intended to run with R 4.2? Thanks for your time!

My current sessionInfo

R version 4.1.0 beta (2021-05-04 r80263)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
biobenkj commented 3 years ago

To reproduce with BiocManager version 1.30.13

> packageVersion("BiocManager")
[1] ‘1.30.13’
> BiocManager::install("DESeq2")
'getOption("repos")' replaces Bioconductor standard repositories, see
'?repositories' for details

replacement repositories:
    CRAN: https://cran.rstudio.com/

Bioconductor version 3.13 (BiocManager 1.30.13), R 4.1.0 beta (2021-05-04 r80263)
Installing package(s) 'DESeq2'
also installing the dependency ‘geneplotter’

>

If rolling back to 1.30.12

> packageVersion("BiocManager")
[1] ‘1.30.12’
> BiocManager::install("DESeq2")
'getOption("repos")' replaces Bioconductor standard repositories, see
'?repositories' for details

replacement repositories:
    CRAN: https://cran.rstudio.com/

Bioconductor version 3.13 (BiocManager 1.30.12), R 4.1.0 beta (2021-05-04 r80263)
Installing package(s) 'DESeq2'
also installing the dependency ‘geneplotter’

trying URL 'https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/geneplotter_1.69.0.tgz'
Content type 'application/x-gzip' length 1552718 bytes (1.5 MB)
==================================================
downloaded 1.5 MB

trying URL 'https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/DESeq2_1.31.21.tgz'
Content type 'application/x-gzip' length 2781825 bytes (2.7 MB)
==================================================
downloaded 2.7 MB

The downloaded binary packages are in
    /var/folders/j9/z2kt4fys04q5xz1m3nv96pv40000gq/T//RtmpqGXcev/downloaded_packages
Old packages: 'BiocManager'
Update all/some/none? [a/s/n]: 
n
>
LiNk-NY commented 3 years ago

Hi Ben, @biobenkj

You should be getting a warning with BiocManger 1.30.13 if you already have the same version of DESeq2. 

Warning message:
package(s) not installed when version(s) same as current; use `force = TRUE` to
  re-install: 'DESeq2'

You are using a non-standard CRAN repository so that can also be a factor... CRAN: https://cran.rstudio.com/

Best, Marcel

vramirez4 commented 3 years ago

I have this same issue. Installing BiocManager on a new laptop running Mac OS Big Sur on ARM chip.

R version 4.0.5

When trying to install packages with BiocManager I produce the following error.

> BiocManager::install("gdsfmt")
'getOption("repos")' replaces Bioconductor standard repositories, see '?repositories' for
details

replacement repositories:
    CRAN: https://cran.rstudio.com/

Bioconductor version 3.12 (BiocManager 1.30.13), R 4.0.5 (2021-03-31)
Installing package(s) 'BiocVersion', 'gdsfmt'
Error in readRDS(dest) : error reading from connection
biobenkj commented 3 years ago

Hi Marcel @LiNk-NY,

Thanks for the quick response! I hadn't installed DESeq2 before attempting the above with 1.30.13, so there shouldn't have been an issue with having/re-installing a package with the same version. To check and see if the non-standard CRAN repo stated above might be an issue, I've set the CRAN repo to one that I've used in the past:

BiocManager::repositories()
'getOption("repos")' replaces Bioconductor standard repositories, see
'?repositories' for details

replacement repositories:
    CRAN: https://repo.miserver.it.umich.edu/cran

                                                BioCsoft 
           "https://bioconductor.org/packages/3.13/bioc" 
                                                 BioCann 
"https://bioconductor.org/packages/3.13/data/annotation" 
                                                 BioCexp 
"https://bioconductor.org/packages/3.13/data/experiment" 
                                           BioCworkflows 
      "https://bioconductor.org/packages/3.13/workflows" 
                                               BioCbooks 
          "https://bioconductor.org/packages/3.13/books" 
                                                    CRAN 
               "https://repo.miserver.it.umich.edu/cran"

After changing these things, the problem persists if I install a new package:

> packageVersion("BiocManager")
[1] ‘1.30.13’
> BiocManager::install("SpatialExperiment")
'getOption("repos")' replaces Bioconductor standard repositories, see
'?repositories' for details

replacement repositories:
    CRAN: https://repo.miserver.it.umich.edu/cran

Bioconductor version 3.13 (BiocManager 1.30.13), R 4.1.0 beta (2021-05-04 r80263)
Installing package(s) 'SpatialExperiment'
also installing the dependencies ‘DropletUtils’, ‘magick’

Error in readRDS(dest) : error reading from connection

Rolling back to 1.30.12:

> packageVersion("BiocManager")
[1] ‘1.30.12’
> BiocManager::install("SpatialExperiment")
Bioconductor version 3.13 (BiocManager 1.30.12), ?BiocManager::install for help
'getOption("repos")' replaces Bioconductor standard repositories, see
'?repositories' for details

replacement repositories:
    CRAN: https://repo.miserver.it.umich.edu/cran

Bioconductor version 3.13 (BiocManager 1.30.12), R 4.1.0 beta (2021-05-04 r80263)
Installing package(s) 'SpatialExperiment'
also installing the dependencies ‘DropletUtils’, ‘magick’

trying URL 'https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/DropletUtils_1.11.12.tgz'
Content type 'application/x-gzip' length 2193184 bytes (2.1 MB)
==================================================
downloaded 2.1 MB

trying URL 'https://repo.miserver.it.umich.edu/cran/bin/macosx/contrib/4.1/magick_2.7.2.tgz'
Content type 'application/octet-stream' length 38926278 bytes (37.1 MB)
==================================================
downloaded 37.1 MB

trying URL 'https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/SpatialExperiment_1.1.700.tgz'
Content type 'application/x-gzip' length 1850989 bytes (1.8 MB)
==================================================
downloaded 1.8 MB

The downloaded binary packages are in
    /var/folders/j9/z2kt4fys04q5xz1m3nv96pv40000gq/T//RtmpkjiIA2/downloaded_packages
Old packages: 'BiocManager'
Update all/some/none? [a/s/n]: 
n
LiNk-NY commented 3 years ago

Hi Ben, @biobenkj

The standard repository is https://cloud.r-project.org. Can you try that? It seems like these errors happen only on Mac / M1 architectures. I don't have access to one unfortunately. Can you provide a traceback()?

Best, Marcel

biobenkj commented 3 years ago

Hi Marcel, @LiNk-NY

Here's the traceback() for dittoSeq with BiocManager 1.30.13 and using the https://cloud.r-project.org CRAN repo.

14: stop(e)
13: .inet_error(e)
12: value[[3L]](cond)
11: tryCatchOne(expr, names, parentenv, handlers[[1L]])
10: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
9: doTryCatch(return(expr), name, parentenv, handler)
8: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), 
       names[nh], parentenv, handlers[[nh]])
7: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatch({
       install.packages(...)
   }, error = function(e) {
       .inet_error(e)
       invisible(NULL)
   }, warning = function(w) {
       msg <- conditionMessage(w)
       if (grepl("not available", msg)) 
           msg <- gsub("this version of R", paste0("Bioconductor version ", 
               "'", version(), "'"), msg)
       .warning(msg)
   })
5: withCallingHandlers({
       tryCatch({
           install.packages(...)
       }, error = function(e) {
           .inet_error(e)
           invisible(NULL)
       }, warning = function(w) {
           msg <- conditionMessage(w)
           if (grepl("not available", msg)) 
               msg <- gsub("this version of R", paste0("Bioconductor version ", 
                   "'", version(), "'"), msg)
           .warning(msg)
       })
   }, warning = .inet_warning)
4: .inet_install.packages(pkgs = doing, lib = lib, repos = repos, 
       ...)
3: .install_repos(pkgs, old_pkgs, instPkgs = instPkgs, lib = lib, 
       repos = repos, checkBuilt = checkBuilt, force = force, ...)
2: .install(pkgs, vout[["out_of_date"]], instPkgs = inst, repos = repos, 
       checkBuilt = checkBuilt, force = force, ...)
1: BiocManager::install("dittoSeq")

and my current sessionInfo

R version 4.1.0 beta (2021-05-04 r80263)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] BiocManager_1.30.13 compiler_4.1.0      tools_4.1.0
biobenkj commented 3 years ago

Apologies, for the update to the traceback() I didn't run it properly the first time. Doh.

LiNk-NY commented 3 years ago

Thanks Ben, @biobenkj We'll look into it. I'll ask someone with a Mac to reproduce and see what's going on. Best, Marcel

mtmorgan commented 3 years ago

Hi @biobenkj , @vramirez4 -- Is this reproducible with

db = available.packages(repos = BiocManager::repositories())

? If so can you narrow this down to a single repository, using e.g., BiocManager::repositories()[1] ? And does it help to avoid using the cached version of the repository

db = available.packages(repos = BiocManager::repositories(), ignore_repo_cache = TRUE)

Can you also provide the value of contrib.url(BiocManager::repositories()) ?

vjcitn commented 3 years ago

Short review: This is a mystery, as I had tested BiocManager::install on M1 for Marcel a few days ago without incident. For the nonce, 

install.packages("SpatialExperiment", type="source", repos=BiocManager::repositories())

works. But I made some comments while in the still unexplained error state:

Error in readRDS(dest) : error reading from connection

Enter a frame number, or 0 to exit

1: BiocManager::install("SpatialExperiment") 2: .install(pkgs, vout[["out_of_date"]], instPkgs = inst, repos = repos, check 3: .install_repos(pkgs, old_pkgs, instPkgs = instPkgs, lib = lib, repos = repo 4: .inet_install.packages(pkgs = doing, lib = lib, repos = repos, ...) 5: withCallingHandlers({ tryCatch({ install.packages(...) }, e 6: tryCatch({ install.packages(...) }, error = function(e) { .inet_err 7: tryCatchList(expr, classes, parentenv, handlers) 8: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), names 9: doTryCatch(return(expr), name, parentenv, handler) 10: tryCatchList(expr, names[-nh], parentenv, handlers[-nh]) 11: tryCatchOne(expr, names, parentenv, handlers[[1]]) 12: value[3] 13: .inet_error(e)

- Comment 2: setting type="source" does not rectify
- Comment 3: `db = available.packages(repos = BiocManager::repositories())` works

db = available.packages(repos = BiocManager::repositories()) dim(db) # same if ignore_repo_cache=TRUE [1] 20857 17

- Comment 4: it seems to take two attempts to trigger; first attempt on a fresh R session just fails silently:

BiocManager::install("SpatialExperiment") Bioconductor version 3.13 (BiocManager 1.30.13), ?BiocManager::install for help Bioconductor version 3.13 (BiocManager 1.30.13), R 4.1.0 alpha (2021-04-26 r80229) Installing package(s) 'SpatialExperiment' also installing the dependencies 'sitmo', 'dqrng', 'scuttle', 'DropletUtils', 'magick'

Old packages: 'rsvg', 'V8' Update all/some/none? [a/s/n]: n

BiocManager::install("SpatialExperiment") Bioconductor version 3.13 (BiocManager 1.30.13), R 4.1.0 alpha (2021-04-26 r80229) Installing package(s) 'SpatialExperiment' also installing the dependencies 'sitmo', 'dqrng', 'scuttle', 'DropletUtils', 'magick'

Error in readRDS(dest) : error reading from connection

Enter a frame number, or 0 to exit

1: BiocManager::install("SpatialExperiment") 2: .install(pkgs, vout[["out_of_date"]], instPkgs = inst, repos = repos, check 3: .install_repos(pkgs, old_pkgs, instPkgs = instPkgs, lib = lib, repos = repo 4: .inet_install.packages(pkgs = doing, lib = lib, repos = repos, ...) 5: withCallingHandlers({ tryCatch({ install.packages(...) }, e 6: tryCatch({ install.packages(...) }, error = function(e) { .inet_err 7: tryCatchList(expr, classes, parentenv, handlers) 8: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]), names 9: doTryCatch(return(expr), name, parentenv, handler) 10: tryCatchList(expr, names[-nh], parentenv, handlers[-nh]) 11: tryCatchOne(expr, names, parentenv, handlers[[1]]) 12: value[3] 13: .inet_error(e)

Selection: 

- Comment 5: when we get to install.packages via .inet_install.packages, `type` is "both" and `type2` is "mac.binary.big-sur-arm64"
- Comment 6: We crash here:

debug: av2 <- available.packages(contriburl = contrib.url(repos, type2), method = method, ...) Browse[6]> Error in readRDS(dest) : error reading from connection

vjcitn commented 3 years ago

One problem is here, I think:

> contrib.url(repos=BiocManager::repositories(), type="binary")
[1] "https://bioconductor.org/packages/3.13/bioc/bin/macosx/big-sur-arm64/contrib/4.1"           
[2] "https://bioconductor.org/packages/3.13/data/annotation/bin/macosx/big-sur-arm64/contrib/4.1"
[3] "https://bioconductor.org/packages/3.13/data/experiment/bin/macosx/big-sur-arm64/contrib/4.1"
[4] "https://bioconductor.org/packages/3.13/workflows/bin/macosx/big-sur-arm64/contrib/4.1"      
[5] "https://bioconductor.org/packages/3.13/books/bin/macosx/big-sur-arm64/contrib/4.1"          
[6] "https://cloud.r-project.org/bin/macosx/big-sur-arm64/contrib/4.1"

While

> .Platform$pkgType
[1] "mac.binary.big-sur-arm64"

and we only have, e.g.,

https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/SpatialExperiment_1.1.700.tgz

Now since SpatialExperiment is a source-only package, one can wget the tgz and any dependencies and install with R CMD INSTALL and use. But this won't work for packages with compiled code:

PC002284:R vincentcarey$ wget https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/DESeq2_1.31.21.tgz
--2021-05-08 09:48:12--  https://bioconductor.org/packages/3.13/bioc/bin/macosx/contrib/4.1/DESeq2_1.31.21.tgz
Resolving bioconductor.org (bioconductor.org)... 13.35.78.86, 13.35.78.120, 13.35.78.19, ...
Connecting to bioconductor.org (bioconductor.org)|13.35.78.86|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2781825 (2.7M) [application/x-gzip]
Saving to: ‘DESeq2_1.31.21.tgz’

DESeq2_1.31.21.tgz            100%[=================================================>]   2.65M  4.66MB/s    in 0.6s    

2021-05-08 09:48:13 (4.66 MB/s) - ‘DESeq2_1.31.21.tgz’ saved [2781825/2781825]

PC002284:R vincentcarey$ R41 CMD INSTALL DESeq2_1.31.21.tgz 
1/9 packages newly attached/loaded, see sessionInfo() for details.
* installing to library '/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library'
* installing *binary* package 'DESeq2' ...
* DONE (DESeq2)
PC002284:R vincentcarey$ R
-bash: R: command not found
PC002284:R vincentcarey$ R41

R version 4.1.0 alpha (2021-04-26 r80229)
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

1/10 packages newly attached/loaded, see sessionInfo() for details.
> library(DESeq2)
Error: package or namespace load failed for 'DESeq2' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/DESeq2/libs/DESeq2.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/DESeq2/libs/DESeq2.so, 6): no suitable image found.  Did find:
    /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/DESeq2/libs/DESeq2.so: mach-o, but wrong architecture
    /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/DESeq2/libs/DESeq2.so: mach-o, but wrong architecture

Enter a frame number, or 0 to exit   

1: library(DESeq2)
2: suppressPackageStartupMessages({
    libdata = base::library(...)
    newse
3: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRest
4: base::library(...)
5: tryCatch({
    attr(package, "LibPath") <- which.lib.loc
    ns <- loadName
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatchOne(expr, names, parentenv, handlers[[1]])
8: value[[3]](cond)

so one has to build from source.

mtmorgan commented 3 years ago

I don't think the lack of binary repositories is 'it'. For instance see Gordon's email https://stat.ethz.ch/pipermail/bioc-devel/2021-May/018090.html on Windows. Also, type = "both" is supposed to work so that when the binary fails, R tries source -- this is needed, I think (?) for the annotation and experiment data packages which are only available as 'source'. Also I'd stick with installation in a consistent manner, e.g., setting options(pkgType = "source") and using BiocManager in the same R session.

There could be problems with the binaries from the build system -- x86 -- versus from a local install -- arm. Maybe it's helpful to think of that as separate from the issue of the corrupt RDS file...?

mtmorgan commented 3 years ago

Can you try, in a new R session

if (!"remotes" %in% rownames(installed.packages()))
    install.packages("remotes", repos = "https://cloud.bioconductor.org")
remotes::install_github("Bioconductor/BiocManager", ref = "ghost-binary-repo")

then restart R. You should have

> packageVersion("BiocManager")
[1] '1.30.13.1'

Can you then try to use standard BiocManager::install() commands?

gksmyth commented 3 years ago

BiocManager 1.30.13.1 works for me under R4.1.0beta for Windows.

I am installing from source when necessary.

vramirez4 commented 3 years ago

It worked fine with version 1.30.13.1.

Thanks!

biobenkj commented 3 years ago

Also can confirm @mtmorgan that it worked for me! Thanks!