rocker-org / rocker-versioned

Run current & prior versions of R using docker
https://hub.docker.com/r/rocker/r-ver
GNU General Public License v2.0
297 stars 169 forks source link

Fix site-library permissions with debian:buster #179

Closed riccardoporreca closed 4 years ago

riccardoporreca commented 4 years ago

See #177 for background and discussions.

Updated r-ver/devel rebuilt locally as rocker/r-ver:devel-site-library-g_ws. Backported to the latest image (3.6.1) to be buster-ready.

Comparison of permissions versus existing rocker/r-ver:devel and rocker/r-ver:3.6.1

test_site_library_mode() {
  docker run --rm rocker/$1 bash -c \
  'find /usr/local/lib/R/site-library* -ls -maxdepth 1'
}
for img in r-ver:devel r-ver:devel-site-library-g_ws \
           r-ver:3.6.1 r-ver:3.6.1-site-library-g_ws
do echo $img && test_site_library_mode $img; done
## r-ver:devel
##  13377366      4 drwxrwxr-x   4 root     staff        4096 Nov 24 22:36 /usr/local/lib/R/site-library
##  13377367      4 drwxrwxr-x   8 root     root         4096 Nov 24 22:36 /usr/local/lib/R/site-library/docopt
##  13377400      4 drwxrwxr-x  12 root     root         4096 Nov 24 22:36 /usr/local/lib/R/site-library/littler
## r-ver:devel-site-library-g_ws
##   5642717      4 drwxrwsr-x   4 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library
##   5642718      4 drwxrwxr-x   8 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library/docopt
##   5642751      4 drwxrwxr-x  12 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library/littler
## r-ver:3.6.1
##  12978538      4 drwxrwsr-x   4 root     staff        4096 Nov 24 22:33 /usr/local/lib/R/site-library
##  12978539      4 drwxrwxr-x   8 root     staff        4096 Nov 24 22:33 /usr/local/lib/R/site-library/docopt
##  12978572      4 drwxrwxr-x  12 root     staff        4096 Nov 24 22:33 /usr/local/lib/R/site-library/littler
## r-ver:3.6.1-site-library-g_ws
##   5120977      4 drwxrwsr-x   4 root     staff        4096 Dec  2 22:13 /usr/local/lib/R/site-library
##   5120978      4 drwxrwxr-x   8 root     staff        4096 Dec  2 22:13 /usr/local/lib/R/site-library/docopt
##   5121011      4 drwxrwxr-x  12 root     staff        4096 Dec  2 22:13 /usr/local/lib/R/site-library/littler

Similarly, for rocker/rstudio:devel, re-building locally based on the updated rocker/r-ver:devel fixes the permission issues as the staff group is now preserved:

test_site_library_mode_rstudio_install() {
  docker run --rm --user rstudio rocker/$1 bash -c \
  'find /usr/local/lib/R/site-library* -ls -maxdepth 1 \
   && install2.r xfun docopt \
   && find /usr/local/lib/R/site-library* -ls -maxdepth 1'
}
for img in rstudio:devel rstudio:devel-site-library-g_ws
do echo $img && test_site_library_mode_rstudio_install $img; done
## rstudio:devel
##   8128063      4 drwxrwxr-x   4 root     staff        4096 Dec  3 01:48 /usr/local/lib/R/site-library
##   8128064      4 drwxrwxr-x   8 root     root         4096 Dec  3 01:48 /usr/local/lib/R/site-library/docopt
##   8128098      4 drwxrwxr-x  12 root     root         4096 Dec  3 01:48 /usr/local/lib/R/site-library/littler
## [...]
## * installing *source* package ‘xfun’ ...
## [...]
## * installing *source* package ‘docopt’ ...
## [...]
## mv: cannot move '/usr/local/lib/R/site-library/docopt' to '/usr/local/lib/R/site-library/00LOCK-docopt/docopt': Permission denied
## ERROR: cannot remove earlier installation, is it in use?
## [...]
##   8128063      4 drwxrwxr-x   1 root     staff        4096 Dec  4 00:08 /usr/local/lib/R/site-library
##   8128064      4 drwxrwxr-x   8 root     root         4096 Dec  3 01:48 /usr/local/lib/R/site-library/docopt
##   8128098      4 drwxrwxr-x  12 root     root         4096 Dec  3 01:48 /usr/local/lib/R/site-library/littler
##  12460546      4 drwxrwxr-x   7 rstudio  rstudio      4096 Dec  4 00:08 /usr/local/lib/R/site-library/xfun
## rstudio:devel-site-library-g_ws
##   5642717      4 drwxrwsr-x   4 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library
##   5642718      4 drwxrwxr-x   8 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library/docopt
##   5642751      4 drwxrwxr-x  12 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library/littler
## [...]
## * installing *source* package ‘xfun’ ...
## [...]
## * installing *source* package ‘docopt’ ...
## [...]
##   5642717      8 drwxrwsr-x   1 root     staff        4096 Dec  4 00:09 /usr/local/lib/R/site-library
##  12724692      4 drwxrwxr-x   8 rstudio  staff        4096 Dec  4 00:09 /usr/local/lib/R/site-library/docopt
##   5642751      4 drwxrwxr-x  12 root     staff        4096 Dec  2 23:08 /usr/local/lib/R/site-library/littler
##  12460546      4 drwxrwxr-x   7 rstudio  staff        4096 Dec  4 00:08 /usr/local/lib/R/site-library/xfun

Note that the example above for rocker/rstudio:devel reproduces the error in Bioconductor/bioconductor_full#17, based on which #177 was opened by @nturaga

riccardoporreca commented 4 years ago

Note that this PR also includes the removal of re-setting root:group ownership in rocker/verse (done on both latest and develop). This does not seem to be required, as assessed using the update images built locally (no-chown-site-library included in the tag), checking if any permission was not root:staff:

test_site_library_root_staff_verse() {
  docker run --rm --user rstudio rocker/$1 bash -c \
  'find /usr/local/lib/R/site-library -ls | grep -E -v "root\s+staff" | head -n 5'
}
for img in verse:3.6.1 verse:3.6.1-no-chown-site-library \
           verse:devel verse:devel-no-chown-site-library-g_ws  
do echo -e "\n"$img && test_site_library_root_staff_verse $img; done
## 
## verse:3.6.1
## 
## verse:3.6.1-no-chown-site-library
## 
## verse:devel
##  16518352      4 drwxrwxr-x   7 root     root         4096 Nov 14 06:17 /usr/local/lib/R/site-library/tufte
##  16518374      4 drwxrwxr-x   2 root     root         4096 Nov 14 06:17 /usr/local/lib/R/site-library/tufte/html
##  16518376      4 -rw-rw-r--   1 root     root         1335 Nov 14 06:17 /usr/local/lib/R/site-library/tufte/html/R.css
##  16518375      4 -rw-rw-r--   1 root     root         1896 Nov 14 06:17 /usr/local/lib/R/site-library/tufte/html/00Index.html
##  16518377      4 drwxrwxr-x   3 root     root         4096 Nov 14 06:17 /usr/local/lib/R/site-library/tufte/rmarkdown
## 
## verse:devel-no-chown-site-library-g_ws
riccardoporreca commented 4 years ago

Note that some minor legacy / temporary statements and comments have been also cleaned-up.

cboettig commented 4 years ago

🎉 Thanks!

nturaga commented 4 years ago

Hi @cboettig , any idea when we'll be seeing the built docker image for this published on Dockerhub?

cboettig commented 4 years ago

@nturaga thanks, good question! devel and latest tags are supposed to be rebuilt nightly, but looking like for some reason those tags stopped getting automatically rebuilt 3 days ago... See: https://hub.docker.com/r/rocker/r-ver/builds

Maybe need to relink things between GitHub and the ever-changing DockerHub/Cloud/Hub thing...

cc @noamross @eddelbuettel at some point we might want to revisit details on image build strategies anyway. I wonder if GitHub actions setup might be better than the DockerHub + cron job curl triggers we have for these nightly builds...

nturaga commented 4 years ago

I see, seems like issue with the connection with Dockerhub or how the builds have been set up. Any chance this one build can be manually triggered on Dockerhub?

cboettig commented 4 years ago

@nturaga yup, I've already manually tickled the build, so that's building now and should be done within the hour.

Still not entirely sure what went wrong with the autobuilds, but I've refreshed the GitHub-DockerHub link for now so we'll see if the cron triggers manage to kick off the new builds automatically when they fire off tonight.

nturaga commented 4 years ago

Thank you @cboettig ! and also thanks to @riccardoporreca for the PR!

nturaga commented 4 years ago

@cboettig I'm going to bother you one more time. Is it possible to kick off a build for the rocker/rstudio:devel Dockerfile as well, since it inherits from the rocker/r-ver:devel. It seems there wasn't any automatic cascading of builds.

My apologies for the bother! https://hub.docker.com/r/rocker/rstudio/builds

cboettig commented 4 years ago

@nturaga good call, queued now. Note you can also build the Dockerfile locally, though of course it's nicer to have it from hub