search

rocker-org / rocker-versioned

Run current & prior versions of R using docker
https://hub.docker.com/r/rocker/r-ver
GNU General Public License v2.0
297 stars 169 forks source link

Personal library installs are not the default #108

Closed matthijskooijman closed 5 years ago

matthijskooijman commented 6 years ago

I've been setting up the rocker/rstudio docker image for a multi-user workshop, and am running into some problems with library installs.

I've found that the default "rstudio" user that is created has write permission to /usr/local/lib/R/site-library (because it is in the staff group), and installing packages work fine for that user.

Then I created some new users (using adduser as documented). Since these users are not in the staff group, they cannot install into /usr/local/lib/R/site-library. R nicely detects this and asks "Would you like to use a personal library instead?". If I ask yes, installation fails, because it still tries to install into /usr/local.

This seems to be caused because $R_LIBS_USER points to /usr/local. Looking around shows that this happens in the Dockerfile. I'm not too familiar with how R manages its library paths, but this setting seems wrong to me?

I realize that I can just put all users in the staff group, but then they will share their libraries, which I think is not what I want to happen.

Ideally, I would like the rstudio user (or any user with write permission) to install packages into /usr/local, for system-wide usage, and have all other users install libraries into their homedirs.

I tried removing the $R_LIBS_USER setting override (by manually editing /usr/local/lib/R/etc/Renviron), falling back to the default setting in ~/R (which is defined in Renviron a few lines up). That seems to work as expected: All users have their homedir as the first path in .libPaths() and install libraries there, while the rstudio user only has /usr/local paths. I'm not entirely sure why the latter is (I would expect rstdio to also list its homedir), but perhaps R is smart and only adds $R_LIBS_USER to the libPaths() when the regular path is not writable?

eddelbuettel commented 6 years ago

It's a standard Debian package feature for which I am responsible.

Add the user to the group, or create a new grop with a name you like better and add them to that -- or just make the directory 0777.

The question belongs on r-sig-debian. This has nothing to do with Rocker per se.

matthijskooijman commented 6 years ago

@eddelbuettel, I'm a bit confused. Are you talking about the staff group? As I mentioned, I can of course add the user to that group (or any other group I like), but then all users will install packages into the same directory, while I want each user to have their own library directory (but also allow using some system-wide preinstalled packages).

I believe this problem is induced by the dockerfile, which does make this a rocker problem (if I'm right, of course): https://github.com/rocker-org/rocker-versioned/blob/9ad0922a7cb6932dfacc5ecc387048a1e1cd6e59/r-ver/devel/Dockerfile#L115

I just noticed I had pasted the wrong Dockerfile link in my first comment (it pointed to R sources instead). I've now updated that link with the link shown in this comment.

eddelbuettel commented 6 years ago

See https://salsa.debian.org/edd/r-base/blob/master/debian/r-base-core.postinst#L51

Every .deb package of R, ie Debian package r-base-core, has been doing that for 15 years. That would include Ubuntu and other derivatives (but is of course different if you install from source as some other Rocker containers do -- so if you have a problem with the versioned stack talk to @cboetting about it). I was referencing the standard package which is used in the Rocker containers I build using the .deb package.

And the meta issue still holds: you design your setup for your students. If you want them to share / be able to write to that directory, give them proper rights or change the directory.

cboettig commented 5 years ago

@matthijskooijman thanks for the report. I am closing this since the behavior reported is as expected:

cboettig commented 5 years ago

(also changing title to reflect the issue more precisely -- as @matthijskooijman notes above, it is not the case that personal library installs are impossible).

matthijskooijman commented 5 years ago

Hm, I'm not sure I'm getting my point across here entirely. Let me have another go:

If I want all users to share a single library dir, I should put the users in the staff group. That makes sense, and I can see that his is the recommended setup.

If I do not want to give (all) users write access to the system-wide directory, I should of course not add them to the staff group. R seems to support this scenario, since it already distinguishes between $R_LIBS_DIR and $R_LIBS_USER. Additionally, when there is no write access to $R_LIBS_DIR, R conveniently asks "Would you like to use a personal library instead?".

On a default Debian install of R, answering "yes" to that question installs stuff into my homedir. On a rocker installation, I get a permission error, since it still tried to install into /usr/local. This is the core of the bug I think exists.

Maybe rocker simply does not support this scenario? If so, I guess documentation should state this and state that you must always add new users to staff, or modify their variables to point at their own personal library (IIRC, there is no such documentation now). It might also be good to disable the "Would you like to use a personal library instead?" prompt, then.

Alternatively, I think that removing the $R_LIBS_USER override in the dockerfile might also fix this problem: https://github.com/rocker-org/rocker-versioned/blob/9ad0922a7cb6932dfacc5ecc387048a1e1cd6e59/r-ver/devel/Dockerfile#L115

AFAICS, this should allow people to answer "yes" to the question and have stuff installed in their homedir. For users that are in the staff group, I believe nothing changes, and installation still happens into /usr/local. So if the currently supported cases are 1) put users into staff or 2) setup $R_LIBS_USER manually, I believe that changing the default for $R_LIBS_USER should not affect either of these supported cases, only add support for a new case.

Having said that: I'm not familiar with the details of how R handles this stuff, so there might certainly be side effects that I'm not seeing (and there probably is a reason why $R_LIBS_USER is overridden in the first place, of course, though the original commit 3ef50c53c6b2aaf225dd5fe0ee919fc617868367 does not seem to tell me), so my suggested fix might not be right one.

However, I do believe that there is a bug here, even if just a documentation bug...

cboettig commented 5 years ago

Hey @matthijskooijman, sorry if I miss-understood and I welcome your engagement on this. thanks.

So I believe this comes down to the fact that, as you say, the Rocker versioned images here set R_LIBS_USER env var, whereas of course this is unset out-of-the box. You may be right that we don't need to set that just to get the right library since we do have R_LIBS. I think this might have been related to RStudio behavior in it's package installer GUI, so I'm just going to poke around at that-- clearly I didn't document the motivation behind setting R_LIBS_USER as well in the commit!

matthijskooijman commented 5 years ago

sorry if I miss-understood

No problem :-)

whereas of course this is unset out-of-the box.

It actually is set out of the box, in the same file that the dockerfile modifies (so currently, you get an Renviron file that sets the variable to the default, and then a few lines later changes it to the /usr/local path).

If you're going to poke around, note what I wrote in the original post:

I tried removing the $R_LIBS_USER setting override (by manually editing /usr/local/lib/R/etc/Renviron), falling back to the default setting in ~/R (which is defined in Renviron a few lines up). That seems to work as expected: All users have their homedir as the first path in .libPaths() and install libraries there, while the rstudio user only has /usr/local paths. I'm not entirely sure why the latter is (I would expect rstdio to also list its homedir), but perhaps R is smart and only adds $R_LIBS_USER to the libPaths() when the regular path is not writable?

Note that here "all users" means "the users without write permission (in /usr/local/lib/R/site-library)", whereas "the rstudio user" means "the user with write permission".

cboettig commented 5 years ago

Thanks. Poking around, just a few follow ups here.

It actually is set out of the box Sorry for the vague language, by box I meant in debian:stable it is not set, whereas as it is set in these rocker images, as I had stated.

It is set in these images because, as I suspected RStudio sets R_LIBS_USER automatically if it is not already set, and this defaults to the user's home location. Because consistency between code run in the R console and code run from RStudio is an important goal of these images, I set R_LIBS_USER explicitly in the image so as to have consistent behavior in both environments.

It does appear though that even when R_LIBS_USER is set to HOME, install.packages() still defaults to the /usr/local/... path when the user has write permission there, as you noted. I don't see a mention of that behavior in the documentation for .libPaths(), and have a vague memory of that not always being the case -- i.e. the top .libPaths() value (default install location) would become this HOME location. Now it seems that's not the case and I don't see any side-effect of having RStudio setting $R_LIBS_USER to, e.g. /home/bob/R/x86_64-pc-linux-gnu-library/3.5 while the cli version has this env var unset. Still not super comfortable having different env var settings in RStudio vs cli, but maybe in this context it is worth unsetting R_LIBS_USER ...

matthijskooijman commented 5 years ago

Thanks for the followup. To be honest, I'm not familiar enough with the R environment to really be of further help, though...

ldfreight commented 5 years ago

Hello,

was there a conclusion here? My docker file installs by default all packages to /use/local/lib/R/sit-library, which is not accessible when docker runs under a different user.

What is the recommended approach? Should I force in the dockerfile to install as a different user?

eddelbuettel commented 5 years ago

which is not accessible when docker runs under a different user

No, it is always readable by all and hence provides all libraries to all users.

If you want another user to write there, then you as admin have to add the user to the group that can write there (or make the directory world-writeable).