Provide support for `{renv}` users

[DianeBeldame]

Assess how to help {renv} users with renv.lock files

[jcpsantiago]

@DianeBeldame we are using renv with a golem app with docker in production. We have a Dockerfile that looks like this (the code to launch the golem app is in a separate script called app_init.R which has the run_app() call:

FROM rocker/r-ver:3.6.1 AS builder

COPY . /app
RUN R CMD build app

FROM rocker/r-ver:3.6.1

ENV RENV_VERSION 0.9.0

RUN R -e "install.packages('remotes')"
RUN R -e "remotes::install_github('rstudio/renv@${RENV_VERSION}')"
COPY --from=builder /myapp/renv.lock renv.lock
RUN R -e "renv::restore()"

COPY --from=builder app_*.tar.gz /app.tar.gz
COPY app_init.R /app_init.R

RUN R -e 'remotes::install_local("/app.tar.gz", dependencies = FALSE)'

EXPOSE 3838
CMD R -e "options(shiny.port=3838,shiny.host='0.0.0.0'); source('app_init.R')"

We .dockerignore both the renv folder and the .Rprofile. For simple apps you probably don't need the "two step" process described in the Dockerfile above though.

With this setup you don't get the nice caching of layers as with installing your packages one by one, so it takes a lot longer to build. I'm sure there's an optimization there we can tap into.

I'm building a repository with an example, barebones golem app with this setup, I'll link it here once its done.

What was difficult for you using golem + renv?

[VincentGuyader]

Hi,

I 've made a branch here : https://github.com/ThinkR-open/golem/tree/renv-test a few day ago.

with a renv=TRUE parameter in the add_dockerfile function. (see https://github.com/ThinkR-open/golem/blob/renv-test/R/dock_from_renv.R )

Buy default it will use your renv.lock file you have, if not it will create one for you:

  if ( ! file.exists(path)){
    lock <- renv::snapshot(lockfile = NULL, confirm = FALSE)
    renv_lockfile_write(lock, file = path)
    }

Right now I don't know if I'm gonna go any further or not with renv inside golem, I need to take a step back to get the pros and cons.

Indeed not being able to use the docker cache system for each of the package is problematic BUT if you use the amaizing renv cache system it could be really usefull! (this requires a number of different applications to be deployed)

I'll take your comments and any PR on the test-renv branch you could make !

Regards

[ColinFay]

The only drawback right now is that we have 20 deps, and adding {renv} will generate a note on CRAN.

Switching the Dockerfile creation to {dockerfiler} would make things easier

[ColinFay]

Been toying a little bit more with that.

One thing I don't like about {renv} in docker is that you can't benefit from docker cache: if your renv::restore() takes very long to run but fails, you don't benefit from any cache.

So we should mimic the current behaviour (with one line per package), with renv::restore()

[ColinFay]

Also, the snapshot should be done with snapshot = "explicit"

[KoderKow]

What is the recommended approach when our work environment does not support docker? To note my environment has RStudio Connect. I have come across a few situations where I developed a golem app and shared the repo with IT to deploy and it fails to deploy because their environment has different package versions. It seems like {renv} is a great candidate for making sure an app is deployed with the same package versions it was developed with.

@ColinFay I believe I saw you mention on Twitter that if you use {renv} and golem that you should only snapshot the DESCRIPTION file. Is this a good approach for specific package version deployments?

[ColinFay]

@KoderKow The "issue" with Connect right now is that it bundles its own flavour of {renv} (as far as I know), so basically it guesses the packages from the project and builds its own file.

That's for example what you force by doing rsconnect::writeManifest().

At this point in time I'm not entirely sure how both interact. I will have a closer look and let you know :)

[DianeBeldame]

Live from weekly meeting, 2 ideas :

• handle renv.lock before installing the package
• parse renv.lock to create remotes::install_version "à la main"

[statnmap]

A new test today with a combination of {renv} and Docker cache requires:

• a .dockerignore with renv and .Rprofile ignored as suggested by @jcpsantiago :

  .RData
  .Rhistory
  .git
  .gitignore
  manifest.json
  rsconnect/
  Rproj.user
  .Rprofile
  renv/

• A Dockerfile like this that forces {renv} to use the classical path to install dependencies with renv::restore(library = .libPaths()):

  • Remember to create the dockerfile when you are in your package, with renv::status() up-to-date, so that currently installed versions are the same as in "renv.lock"
  • Note that sub-dependencies probably still don't have the correct version and will be re-installed by renv::restore()
  • Note also that when you change some code in your project, then renv::restore() will be called again as it appears after WORKDIR

    
    FROM rocker/geospatial:4.0.1
    RUN apt-get update && apt-get install -y  gdal-bin git-core libcairo2-dev libcurl4-openssl-dev libgdal-dev libgeos-dev libgeos++-dev libgit2-dev libicu-dev libpng-dev libpq-dev libproj-dev libssl-dev libudunits2-dev libxml2-dev make pandoc pandoc-citeproc zlib1g-dev && rm -rf /var/lib/apt/lists/*
    RUN echo "options(repos = c(CRAN = 'https://packagemanager.rstudio.com/all/__linux__/focal/latest'), download.file.method = 'libcurl')" >> /usr/local/lib/R/etc/Rprofile.site
    RUN echo "options(warn = 2);" >> $R_HOME/etc/Rprofile.site

Dependencies as extracted from DESCRIPTION file as usual

RUN Rscript -e 'install.packages("remotes")' RUN Rscript -e 'remotes::install_version("stringr",upgrade="never", version = "1.4.0")' RUN Rscript -e 'remotes::install_version("glue",upgrade="never", version = "1.4.2")' RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "2.0.1")' RUN Rscript -e 'remotes::install_version("future",upgrade="never", version = "1.21.0")' RUN Rscript -e 'remotes::install_version("cartography",upgrade="never", version = "2.4.2")' RUN Rscript -e 'remotes::install_github("Thinkr-open/golem@aaae5c8788802a7b4aef4df23691902a286dd964")'

renv part

RUN Rscript -e 'remotes::install_github("rstudio/renv")' RUN mkdir /build_zone ADD . /build_zone WORKDIR /build_zone

RUN Rscript -e 'source("renv/activate.R")'

Still need to restore in case sub-dependencies versions are not good

RUN Rscript -e 'renv::restore(library = .libPaths())' RUN Rscript -e 'remotes::install_local(upgrade = FALSE)' RUN rm -rf /build_zone

Remove option to warn and stop

RUN head -n -1 $R_HOME/etc/Rprofile.site

RUN echo "options(warn = -1);" >> $R_HOME/etc/Rprofile.site

EXPOSE 3838 CMD ["R", "-e", "options('shiny.port'=3838,shiny.host='0.0.0.0');mygolem::run_app()"]

[VincentGuyader]

HI,

with golem > 0.3.3, you can now use :

golem::add_dockerfile_with_renv(output_dir = "deploy")

or

golem::add_dockerfile_with_renv(output_dir = "deploy",lockfile = "renv.lock")

to create your docker image. this use natively {renv} and it's more reliable than previous functions.