Missing path for r-cran-* apt packages

[ClaytonJY]

I'm not sure if this is per-package or for all of them, but at least some r-cran-* apt packages install to /usr/lib/R/site-library, which is not present in .libPaths() for e.g. the rocker/rstudio image:

➤ docker run --rm rocker/rstudio R -q -e ".libPaths()"
> .libPaths()
[1] "/usr/local/lib/R/site-library" "/usr/local/lib/R/library"

This means that when installing e.g. r-cran-dbi, after success you won't be able to library(DBI) until you edit the paths or do something like library(DBI, lib.loc = "/usr/lib/R/site-library").

Would it make sense to add this to the path somewhere on the rocker side?

I also notice that some dependency of r-cran-dbi modifies the path to add /usr/lib/R/library, which makes me wonder if this pathing should be fixed elsewhere:

docker run --rm -e ROOT=TRUE rocker/rstudio /bin/bash -c "sudo apt update && sudo apt install -y r-cran-dbi && R -q -e '.libPaths()' && ls -l /usr/lib/R/site-library"
... # truncated
> .libPaths()
[1] "/usr/local/lib/R/site-library" "/usr/local/lib/R/library"     
[3] "/usr/lib/R/library"           
> 
> 
total 4
drwxr-xr-x 7 root root 4096 Sep  5 15:19 DBI

Oddly, I've never seen this lack-of-path problem on my local linux machines.

Is there a different way to use R packages from apt inside rocker images?

[eddelbuettel]

You filed this under rocker-versioned, but you seem to use an unversioned container. Which is it?

The default behaviour for the underlying R package is what we see in r-base aka rocker/r-base:

edd@rob:~$ docker run --rm -ti r-base Rscript -e ".libPaths()"
[1] "/usr/local/lib/R/site-library" "/usr/lib/R/site-library"
[3] "/usr/lib/R/library"
edd@rob:~$

For the containers derived off rocker/r-base this should always work.

If and when we build R from source as the versioned variants do this may be different. In those cases the binaries may be a slight mismatch though as the design is for MRAN, and source installations from it.

[ClaytonJY]

I used rocker/rstudio to demonstrate (because that's how I discovered this), which is based on rocker/rstudio-stable, which in turn is on top of rocker/r-ver, not r-base.

This appears to be an issue in rocker/r-ver specifically (thus in scope for this repo, I hope); changing to rocker/r-ver in your example yields

➤ docker run --rm -ti rocker/r-ver Rscript -e ".libPaths()"
[1] "/usr/local/lib/R/site-library" "/usr/local/lib/R/library"

Thus compared to r-base, r-ver removes /usr/lib/R/site-library, and replaces /usr/lib/R/library with /usr/local/lib/R/library (insert /local/).

I don't know what the ideal set of lib paths is, but if r-base is the template, it seems r-ver could use some updating?

[cboettig]

@ClaytonJY Looks like you're running the rstudio image, which is in the version-stable stack:

docker run --rm -e ROOT=TRUE rocker/rstudio ...

In the version-stable stack, R and R packages are installed from source, and we do not advise installing the pre-built debian binaries -- this will also install the binary version of R shipped in the debian:stretch repos, and not the latest (or tagged) version.

If you're looking for DBI, it's also already installed in the rocker/tidyverse image.

[ClaytonJY]

@cboettig ah that makes sense. I'm actually looking for RQuantLib (DBI was easier to reproduce since r-cran-rquantlib is too old for newer R version), so if I want that along with RStudio, I need to stick to install.packages?

[cboettig]

@ClaytonJY yup, install from source with install.packages (or the install2.r script for a more friendly syntax in Dockerfiles, see, e.g. the tidyverse Dockerfile). Looks like it has system dependencies, so you'll want to install those first, e.g. apt update && apt -y install libquantlib0-dev.

It's a bit more of a pain on the versioned images since you have to identify these system deps like libquantlib0-dev, yourself, but on the upside no need to worry about the binary r-cran-* not being the version you want.

[cboettig]

Sorry this wasn't clearer. I added a note on the README, but no doubt people will still miss that, https://github.com/rocker-org/rocker-versioned#notes. Advice / feedback on documenting this stuff more clearly always welcome

[ClaytonJY]

I think that note is immensely helpful, and exactly the guidance I needed here, thank you for adding that @cboettig!

Documenting this stuff is tricky largely because the audience is presumably pretty unsavvy about Docker and system-level stuff in general (myself included), but I think you're doing a fine job and I particularly appreciate the quick and helpful responses when I can't figure it out myself. I sure as hell don't know what I'd do without all these rocker images!

[cboettig]

Thanks! yeah, I just worry it's easy for a lot to get buried in the README. I'm trying to duplicate that info over at https://www.rocker-project.org too, but yeah, questions always welcome!

[eddelbuettel]

Just in case that was unclear, if you use a Debian-based container with binary R then it is as simple as one call: apt install r-cran-rquantlib and it just works. If you opt for versioned then you value something else more highly and may incur compile cost. It all depends on what you want.

[ClaytonJY]

@eddelbuettel got it. I generally want RStudio for interactive work, so I default to the rstudio or tidyverse containers, but perhaps I should consider using rstudio:testing for work like this.

FWIW I did get this working locally w/ a tidyverse-based Dockerfile, now just waiting for Dockerhub to finish building: https://hub.docker.com/r/methodsconsultants/rocker-tidyverse-rquantlib/. Thanks!

[cboettig]

@ClaytonJY just a heads up that rstudio:testing actually does need to be patched because the build is failing right now -- a consequence of the apt-pinning in the r-base stack means that occasionally I need to keep tweaking whether a binary comes from debian:testing or debian:unstable (since packages migrate though those layers with the 'sliding' version).

Yup, building your own Dockerfile so you don't have to wait for things to compile on the fly is my approach too; your https://github.com/MethodsConsultants/rocker-tidyverse-RQuantLib/blob/master/Dockerfile looks great. (minor comment, no need to set the cran mirror, that is set automatically -- on latest to the RStudio CDN just like you are doing, and to MRAN on the versioned tags like .3.5.1). Otherwise looks good to me!

[ClaytonJY]

@cboettig thanks for the feedback; I probably copied that from somewhere, and copied it again and again not knowing I didn't need it. I'm all for minimalism!