ZackBarry commented 3 years ago

Hey all -- I'm trying to execute the following image hierarchy:

Base image with R 4.0.5, RStudio 1.4.x, renv >= 0.13, and some other stuff (gcloud, mlflow, ...)
Modeling image with user code and packages via renv.lock

I've forklifted the RStudio installation code from your rocker-versioned Dockerfile and installed it into the base image. The modeling image simply copies the codebase into /home/rstudio and runs renv::restore(). The codebase includes .Rprofile with the line source("renv/activate.R").

I'm having trouble getting RStudio to load the correct environment, though. If I run a container and open an R session via the command line within the /home/rstudio directory, the renv environment loads without issue and the modeling scripts work. If I run a container and then use the /init command to load RStudio, it tries to boostrap renv rather than using existing installs:

# Bootstrapping renv 0.13.2 --------------------------------------------------
* Downloading renv 0.13.2 ... OK (downloaded source)
* Installing renv 0.13.2 ... Done!
Error installing renv:
======================
Error: ERROR: no permission to install to directory ‘/home/rstudio/renv/library/R-4.0/x86_64-pc-linux-gnu’
Warning messages:
1: In system2(r, args, stdout = TRUE, stderr = TRUE) :
  running command ''/usr/lib/R/bin/R' --vanilla CMD INSTALL -l '/home/rstudio/renv/library/R-4.0/x86_64-pc-linux-gnu' '/tmp/Rtmpc0ATgf/renv_0.13.2.tar.gz' 2>&1' had status 1
2: Failed to find an renv installation: the project will not be loaded.
Use `renv::activate()` to re-initialize the project.

I added some print statements to renv's activate.R function to check that both command line R session and RStudio had the same values for project, libpath, and version, and they were the same:

[1] "/home/rstudio"
[1] "/home/rstudio/renv/library/R-4.0/x86_64-pc-linux-gnu"
[1] "0.13.2"

I've tried setting export ROOT=true and DISABLE_AUTH=true, running sudo /init, and using /etc/services.d/rstudio/run directly, but all have the same result.

Edit: I should also mention that I wasn't able to overwrite files with RStudio, it would only let me save copies.

cboettig commented 3 years ago

@ZackBarry It sounds like something is wrong with your permissions. Can you try testing with our RStudio image, rocker/rstudio, instead? It has always worked for me with renv.

You point out:

I've forklifted the RStudio installation code from your rocker-versioned Dockerfile and installed it into the base image.

but it's not clear what precisely what you are doing. You've linked to the old, 3.x, debian based images, while our 4.x tagged images are based on Ubuntu 20.04; see README. HTH

ZackBarry commented 3 years ago

Thanks for the suggestion, @cboettig. I tested out the rocker/rstudio:4.0.5 image and am running into what seems to be the same issue.

Here's what I tried:

Create a new R 4.0.5 project locally using RStudio IDE, call it test
Write a simple script, plot.R
```
library(dplyr)
library(ggplot2)
```

data.frame(a = rnorm(100)) %>% ggplot(aes(x = a)) + geom_density()

* Snapshot packages with `renv::init()`
    * Project directory now has `renv.lock` and `renv` files
    * `.Rprofile` has the line `source("renv/activate.R")`
* Write a simple Dockerfile with `renv::restore()`

FROM rocker/rstudio:4.0.5

WORKDIR /home/rstudio

Install requirements with renv

COPY .Rprofile . COPY renv.lock . ADD renv renv

Install requirements with renv

RUN R -e "renv::consent(provided = TRUE); renv::restore()"

Copy project files (okay to rewrite renv files)

COPY / ./

* Build image to Google Container Registry and spin up a pod

kubectl run --env "PASSWORD=test" --image gcr.io//:test rstudio kubectl port-forward rstudio 8787:8787

* Log in to RStudio and get this log

R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.

Bootstrapping renv 0.13.2 --------------------------------------------------

Downloading renv 0.13.2 ... OK (downloaded source)
Installing renv 0.13.2 ... Done! Error installing renv:

Error: ERROR: no permission to install to directory ‘/home/rstudio/renv/library/R-4.0/x86_64-pc-linux-gnu’ Warning messages: 1: In system2(r, args, stdout = TRUE, stderr = TRUE) : running command ''/usr/local/lib/R/bin/R' --vanilla CMD INSTALL -l '/home/rstudio/renv/library/R-4.0/x86_64-pc-linux-gnu' '/tmp/Rtmp2KrcKP/renv_0.13.2.tar.gz' 2>&1' had status 1 2: Failed to find an renv installation: the project will not be loaded. Use renv::activate() to re-initialize the project.
renv::activate() Error in loadNamespace(name) : there is no package called ‘renv

I also tried running

kubectl run --env "PASSWORD=test" --env "DISABLE_AUTH=true" --env "ROOT=true" --image gcr.io/<my-org>/<my-proj>:test rstudio

but that resulted in the same error as above.

cboettig commented 3 years ago

@ZackBarry Ah, I see, you are running a bunch of COPY commands in your Dockerfile. These run as root, and this results in those files being owned by root and not by the default rstudio user. You probably meant to switch to USER rstudio before copying. (Also, copying / seems like a very bad idea..., you probably just want to follow the advice in https://rstudio.github.io/renv/articles/docker.html after switching the USER....).

ZackBarry commented 3 years ago

@cboettig Thank you for these suggestions! Unfortunately I tried some variations of them and they all lead to errors.

Include USER rstudio before COPY, get error when building:

[91m# Bootstrapping renv 0.13.2 --------------------------------------------------
[0m[91m* Downloading renv 0.13.2 ... [0m[91mOK (downloaded source)
[0m[91m* Installing renv 0.13.2 ... [0m[91mDone!
[0m[91mError installing renv:
======================
Error: ERROR: cannot cd to directory ‘/home/rstudio/renv/library/R-4.0/x86_64-pc-linux-gnu’

Same build error if I tried to give rtudio root privileges like RUN usermod -aG sudo rstudio before switching user

Use the --vanilla flag after USER rstudio as suggested by https://rstudio.github.io/renv/articles/docker.html, get error when building:

Step 8/9 : RUN R --vanilla -s -e "renv::restore()"
---> Running in 840a6f8608e3
[91mError in loadNamespace(name) : there is no package called ‘renv’

Install renv before the restore command (full Dockerfile for clarity)
```
FROM rocker/rstudio:4.0.5
```

WORKDIR /home/rstudio

RUN usermod -aG sudo rstudio

USER rstudio

RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))" RUN R -e "remotes::install_github('rstudio/renv@0.13.2')"

Install requirements with renv

COPY .Rprofile . COPY renv.lock . ADD renv renv

Install requirements with renv

RUN R --vanilla -s -e "renv::restore()"

Copy project files (okay to rewrite renv files)

COPY / ./

the image builds, but I get this error when trying to boot RStudio:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0. s6-chown: fatal: unable to chown /var/run/s6/etc/cont-init.d/userconf: Operation not permitted s6-chmod: fatal: unable to change mode of /var/run/s6/etc/cont-init.d/userconf: Operation not permitted s6-chown: fatal: unable to chown /var/run/s6/etc/services.d/rstudio/run: Operation not permitted s6-chmod: fatal: unable to change mode of /var/run/s6/etc/services.d/rstudio/run: Operation not permitted s6-chown: fatal: unable to chown /var/run/s6/etc/services.d/rstudio/finish: Operation not permitted s6-chmod: fatal: unable to change mode of /var/run/s6/etc/services.d/rstudio/finish: Operation not permitted [s6-init] ensuring user provided files have correct perms...exited 0. [fix-attrs.d] applying ownership & permissions fixes... [fix-attrs.d] done. [cont-init.d] executing container initialization scripts... [cont-init.d] userconf: executing... /var/run/s6/etc/cont-init.d/userconf: line 12: /usr/local/lib/R/etc/Renviron: Permission denied Changing password for rstudio. chpasswd: (user rstudio) pam_chauthtok() failed, error: Authentication token manipulation error chpasswd: (line 1, user rstudio) password not changed /var/run/s6/etc/cont-init.d/userconf: line 100: /etc/R/Renviron.site: Permission denied /var/run/s6/etc/cont-init.d/userconf: line 101: /etc/R/Renviron.site: Permission denied [cont-init.d] userconf: exited 1. [cont-init.d] done. [services.d] starting services s6-supervise (child): fatal: unable to exec run: Permission denied s6-supervise rstudio: warning: unable to spawn ./run - waiting 10 seconds [services.d] done. s6-supervise rstudio: warning: unable to spawn ./run - waiting 10 seconds s6-supervise (child): fatal: unable to exec run: Permission denied s6-supervise (child): fatal: unable to exec run: Permission denied s6-supervise rstudio: warning: unable to spawn ./run - waiting 10 seconds s6-supervise (child): fatal: unable to exec run: Permission denied


same error with `--env "ROOT=true" --env "DISABLE_AUTH=true"` 

Is it possible to run RStudio as root user to get around this? Or anything else I can try?

cboettig commented 3 years ago

Sorry, but you are running into issues here that have nothing to do with rocker or RStudio per se, you simply need to pay more attention linux file permissions and existence. In the above last example Dockerfile, you don't switch back to USER root, so your image is trying to run as the rstudio user, which, as the errors tell you, does not have permission to run the init system, which runs as root. (granting the rstudio user sudo privileges is not sufficient since these aren't run with sudo).

FWIW, I don't recommend creating a custom Dockerfile when using renv unless your project depends on custom apt-get libraries. There is usually no need to copy over the local renv library of all packages, since explicit versions of all packages are already pinned in the renv lockfile. It's usually sufficient just to use a vanilla rocker image, like rocker/rstudio:4.0.5, clone your project from GitHub (or copy just the project files and lockfile -- note renv .gitignore's the actual package library since all you need is the lockfile) , and let renv handle the installation.

ZackBarry commented 3 years ago

Gotcha, yeah I'm definitely still learning some of these ins and outs - thank you for all the valuable feedback @cboettig . I was hoping to get renv to install the packages at build time rather than run time to save time. I'll keep trying there, but good to know I can fall back to letting renv handle it at run time.

ZackBarry commented 3 years ago

Hopefully this isn't too hacky, but I figured out a way to make this work how I wanted. For context, the image I'm trying to build is dual purpose - 1) run a script upon container spinup then terminate, 2) spin up RStudio with access to project files and packages for development purposes. There is just 1 project per image, so I don't need renv to handle multiple workspaces in the same image. Packages can be installed globally, and renv/activate.R doesn't need to be called when RStudio starts. This realization really helped.

Here's what I did:

Install packages globally following instructions here.
Run RStudio from a folder other than /home/rstudio via these instructions

FROM rocker/rstudio:4.0.5

WORKDIR /app

# Install project packages globally with renv
RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
RUN R -e "remotes::install_github('rstudio/renv@0.13.2')"
COPY renv.lock .
RUN R -e "renv::restore(lockfile = '/app/renv.lock', library = '/usr/local/lib/R/site-library')"

# Copy non-renv project files
COPY avg.R avg.R

# Start RStudio from project folder
RUN echo "session-default-working-dir=/app" >> /etc/rstudio/rsession.conf

ENTRYPOINT ["/bin/bash", "-c"]

Now this can create an RStudio instance

kubectl run --image <my-image> --env "PASSWORD=test" rstudio -- /init
kubectl port-forward rstudio 8787:8787

or execute a project script

kubectl run --image <my-image> rstudio -- "Rscript /app/avg.R"

rocker-org / rocker

RStudio not working with renv #463

Install requirements with renv

Install requirements with renv

Copy project files (okay to rewrite renv files)

Bootstrapping renv 0.13.2 --------------------------------------------------

Installing renv 0.13.2 ... Done! Error installing renv:

Install requirements with renv

Install requirements with renv

Copy project files (okay to rewrite renv files)