search

Bioconductor / bioconductor_full

[DEPRECATED] Docker Images which include a complete installation of all software needed to build all Bioconductor packages
http://bioconductor.org/help/docker/
GNU General Public License v2.0
25 stars 4 forks source link

installation of package ‘BiocVersion’ had non-zero exit status with devel 3.11 #17

Open lwaldron opened 5 years ago

lwaldron commented 5 years ago

I tried running BiocManager::install() to upgrading my packages library with the new 3.11 container. Packages updated except for BiocVersion, which is on site-library, due to insufficient permissions. I started R inside the container like this:

   docker run -ti --user rstudio \
        -v $HOME:/home/rstudio \
        -v $HOME/R/bioc-devel:/usr/local/lib/R/host-site-library \
        -w /home/rstudio \
        bioconductor/bioconductor_full:devel R

Here is what happens:

> BiocManager::install()
Bioconductor version 3.11 (BiocManager 1.30.9), R Under development (unstable)
  (2019-11-06 r77380)
Installation path not writeable, unable to update packages: mgcv, nlme,
  survival
Old packages: 'BiocVersion'
Update all/some/none? [a/s/n]: a
trying URL 'https://bioconductor.org/packages/3.11/bioc/src/contrib/BiocVersion_3.11.1.tar.gz'
Content type 'application/x-gzip' length 980 bytes
==================================================
downloaded 980 bytes

Bioconductor version 3.11 (BiocManager 1.30.9), ?BiocManager::install for help
* installing *source* package ‘BiocVersion’ ...
mv: cannot move '/usr/local/lib/R/site-library/BiocVersion' to '/usr/local/lib/R/site-library/00LOCK-BiocVersion/BiocVersion': Permission denied
ERROR: cannot remove earlier installation, is it in use?
* removing ‘/usr/local/lib/R/site-library/BiocVersion’

The downloaded source packages are in
    ‘/tmp/RtmpobSDKb/downloaded_packages’
Warning message:
In install.packages(update[instlib == l, "Package"], l, contriburl = contriburl,  :
  installation of package ‘BiocVersion’ had non-zero exit status
> .libPaths()
[1] "/usr/local/lib/R/host-site-library" "/usr/local/lib/R/site-library"     
[3] "/usr/local/lib/R/library"          
> BiocManager::valid()

* sessionInfo()

R Under development (unstable) (2019-11-06 r77380)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 10 (buster)

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.5.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C             
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] BiocManager_1.30.9

loaded via a namespace (and not attached):
[1] compiler_4.0.0 tools_4.0.0   

Bioconductor version '3.11'

  * 1 packages out-of-date
  * 0 packages too new

create a valid installation with

  BiocManager::install("BiocVersion", update = TRUE, ask = FALSE)

more details: BiocManager::valid()$too_new, BiocManager::valid()$out_of_date

Warning message:
1 packages out-of-date; 0 packages too new 

> system("ls -l /usr/local/lib/R/site-library/BiocVersion" )
total 24
-rw-rw-r-- 1 root root 1106 Nov 14 17:00 DESCRIPTION
drwxrwxr-x 2 root root 4096 Nov 14 17:00 help
drwxrwxr-x 2 root root 4096 Nov 14 17:00 html
-rw-rw-r-- 1 root root   62 Nov 14 17:00 INDEX
drwxrwxr-x 2 root root 4096 Nov 14 17:00 Meta
-rw-rw-r-- 1 root root  121 Nov 14 17:00 NAMESPACE

> system("whoami")
rstudio
>
nturaga commented 5 years ago

@lwaldron This is happening because the bioc_docker images install packages install the R packages as root (https://github.com/Bioconductor/bioc_docker/blob/master/out/devel_base/Dockerfile#L30).

It is an issue which might be carried forward.

I'm working on a looking into this now.

LiNk-NY commented 5 years ago

Hi @lwaldron @nturaga , I was unable to reproduce the issue with the current bioconductor_full:devel container. I had installed BiocVersion 3.11.0 in $HOME/R/bioc-devel with R devel. I'm using Mac 10.14.5 (18F132). Docker Desktop 2.1.0.5 (40693).

I loaded docker via:

docker run -ti --user rstudio \
        -v $HOME:/home/rstudio \
        -v $HOME/R/bioc-devel:/usr/local/lib/R/host-site-library \
        -w /home/rstudio \
        bioconductor/bioconductor_full:devel R
> BiocManager::install("BiocVersion")
Bioconductor version 3.11 (BiocManager 1.30.10), R Under development (unstable)
  (2019-11-19 r77438)
Installing package(s) 'BiocVersion'
trying URL 'https://bioconductor.org/packages/3.11/bioc/src/contrib/BiocVersion_3.11.1.tar.gz'
Content type 'application/x-gzip' length 980 bytes
==================================================
downloaded 980 bytes

Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
* installing *source* package 'BiocVersion' ...
** using staged installation
** help
*** installing help indices
** building package indices
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
** testing if installed package can be loaded from temporary location
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
** testing if installed package can be loaded from final location
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
** testing if installed package keeps a record of temporary installation path
* DONE (BiocVersion)

The downloaded source packages are in
    ‘/tmp/Rtmpq1pTOJ/downloaded_packages’
Old packages: 'beachmat', 'car', 'ddalpha', 'GEOquery', 'Hmisc', 'prodlim',
  'quantreg', 'RcppEigen', 'RCy3', 'scales', 'scater', 'selectr', 'SeqArray'
Update all/some/none? [a/s/n]: n

root:root is present before and after BiocVersion update

> system("ls -l /usr/local/lib/R/site-library/BiocVersion" )
total 24
-rw-rw-r-- 1 root root 1106 Nov 20 21:56 DESCRIPTION
drwxrwxr-x 2 root root 4096 Nov 20 21:56 help
drwxrwxr-x 2 root root 4096 Nov 20 21:56 html
-rw-rw-r-- 1 root root   62 Nov 20 21:56 INDEX
drwxrwxr-x 2 root root 4096 Nov 20 21:56 Meta
-rw-rw-r-- 1 root root  121 Nov 20 21:56 NAMESPACE

I did notice that my version of BiocManager is newer. Perhaps it is best to do a fresh pull of the bioconductor_full:devel image?

-Marcel

nturaga commented 5 years ago

Hi @LiNk-NY ,

I think the update of the 3.11 is an artifact of the updated docker image which I pushed recently.

I'm unable to reproduce it as well.

Maybe the update of BiocManager fixes it?

LiNk-NY commented 5 years ago

Ah, I am able to reproduce if I explicitly try to install in 

BiocManager::install("BiocVersion", lib = "/usr/local/lib/R/site-library")
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
* installing *source* package 'BiocVersion' ...
mv: cannot move '/usr/local/lib/R/site-library/BiocVersion' to '/usr/local/lib/R/site-library/00LOCK-BiocVersion/BiocVersion': Permission denied
ERROR: cannot remove earlier installation, is it in use?
LiNk-NY commented 5 years ago

The changes from BiocManager 1.30.9 to 1.30.10 were minimal.

nturaga commented 5 years ago

Ah, I am able to reproduce if I explicitly try to install in

BiocManager::install("BiocVersion", lib = "/usr/local/lib/R/site-library")
Bioconductor version 3.11 (BiocManager 1.30.10), ?BiocManager::install for help
* installing *source* package 'BiocVersion' ...
mv: cannot move '/usr/local/lib/R/site-library/BiocVersion' to '/usr/local/lib/R/site-library/00LOCK-BiocVersion/BiocVersion': Permission denied
ERROR: cannot remove earlier installation, is it in use?

Right, but the user rstudio doesn't have permissions to that. And that is a remnant of inheritance from bioc_docker's devel_base2:latest image.

I seems like we understand the reason behind the permissions issue installing it in "/usr/local/lib/R/site-library". But not the reason behind, why it tried to install the BiocVersion package in site-library. It should automatically install in** host-site-library.

LiNk-NY commented 5 years ago

If the image installed BiocVersion 3.11.0 and there were no other versions of BiocVersion available, I think it would try to update this version in site-library. But working with BiocManager AFAIK, it should not touch packages in locations that cannot be written in (pkgs are silently ignored)... :thinking:

nturaga commented 5 years ago

So, does that mean it's an issue in BiocManager::install() ?

LiNk-NY commented 5 years ago

Based on the small 'test' that BiocManager does to see if the directory is write-able: file.access("/usr/local/lib/R/site-library/", 2L) == 0 is TRUE. The directory is technically write-able but the existing contents are owned by root:root. The BiocManager behavior here looks correct to me. For example, in the case that packages were installed using a sudo account, a user shouldn't be able to update them without elevation. So to answer your question, no it's not.

nturaga commented 5 years ago

Ok, then we need to redo the Dockerfile in devel_base2 and release_base2 images within bioc_docker then, and make sure that those libraries are owned by rstudio.

mtmorgan commented 5 years ago

one complexity here is the discussion of users -- if there are two users, then either the libraries should be installed for each user, or the libraries should be writeable by a shared group. A shared group solution wouldn't be good in a traditional environment, because user installations would clobber one another; maybe it's ok in Dockerland? Or maybe there should just be one user ('bioc')...

nturaga commented 5 years ago

@lwaldron @LiNk-NY @mtmorgan

Just an update on the original issue, i've posted the issue on the rocker repository and it seems to be stemming from there.

https://github.com/rocker-org/rocker-versioned/issues/177