Issue with 'digest' package when running Renv inside Docker + Shiny

[capncodewash]

Hi there, I have a very similar issue to the one reported on the RStudio forums in the thread "How to use renv with shiny-server in Docker" - renv claims that the 'digest' package is not available, even though it is listed in my renv.lock file, and I can see that it gets built from source and installed during Docker build.

Here is the error, taken from the Shiny startup log at /var/log/shiny-server/ :

* The project and lockfile are out of sync -- use `renv::status()` for more details.
Warning message:
The following package(s) are missing entries in the cache:

        askpass, assertthat, backports, base64enc, BH, broom,
        callr, cellranger, cli, clipr, colorspace, crayon,
        crosstalk, curl, data.table, DBI, dbplyr, desc, digest,
        dplyr, DT, ellipsis, evaluate, fansi, farver, fastmap,
        forcats, fs, generics, ggplot2, ggwordcloud, glue, gtable,
        haven, here, hexbin, highr, hms, htmltools, htmlwidgets,
        httpuv, httr, hunspell, igraph, isoband, ISOcodes,
        janeaustenr, jsonlite, knitr, labeling, later, lattice,
        lazyeval, lexRankr, lifecycle, lubridate, magrittr,
        markdown, MASS, mime, modelr, munsell, nlme, NLP, openssl,
        pdftools, pillar, pkgbuild, pkgconfig, pkgload, plogr,
        plotly, plyr, png, praise, prettyunits, processx, progress,
        promises, ps, purrr, qpdf, R6, RColorBrewer, Rcpp,
        RcppEigen, readr, readxl, rematch, reprex, reshape2, rlang,
        rmarkdown, rprojroot, RSpectra, rstudioapi, rvest, scales,
        selectr, shiny, shinydashboard, shinyFiles, shinyhelper,
        shinyjs, shinyWidgets, slam, Sn [... truncated] 
Error in loadNamespace(name) : there is no package called ‘digest’
Calls: local ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

Excerpt of Renv.lock:

    "digest": {
      "Package": "digest",
      "Version": "0.6.25",
      "Source": "CRAN"
    },

Excerpt from Docker build command:

Installing digest [0.6.25] ...
        OK [built from source]

My Dockerfile looks like this:

FROM rocker/shiny:3.6.3

RUN apt-get update && apt-get install -y libxml2 libxml2-dev libgsl-dev libssl-dev libjpeg-dev libpoppler-cpp-dev

RUN sed -i 's/3838/3838 0.0.0.0/' /etc/shiny-server/shiny-server.conf

COPY . /srv/shiny-server/

WORKDIR /srv/shiny-server
COPY renv.lock renv.lock

RUN echo '.libPaths("/srv/shiny-server/renv/library/R-3.6/x86_64-pc-linux-gnu")' >> /usr/local/lib/R/etc/Rprofile.site

RUN R -e 'renv::restore(library="/srv/shiny-server/renv/library/R-3.6/x86_64-pc-linux-gnu")'

RUN R -e "renv::status()"

# allow permission
RUN sudo chown -R shiny:shiny /srv/shiny-server

USER shiny

EXPOSE 3838

# run app
CMD ["/usr/bin/shiny-server.sh"]

Versions in use:

• rocker/shiny Docker image version 3.6.3
• renv version 0.10.0
• R version 3.6.3 (inheriting from the Docker image)
• Docker version 19.03.8, build afacb8b on macOS

I can see that the packages I requested are indeed in the /srv/shiny-server/renv/library/R-3.6/x86_64-pc-linux-gnu directory, but they don't seem to be available to Shiny.

I added an 'renv::status()' line to the Dockerfile as a diagnostic step during the Docker build. This is the part of output - showing that 'digest' is indeed installed:

* Project '/srv/shiny-server' loaded. [renv 0.10.0]
* The project and lockfile are out of sync -- use `renv::status()` for more details.
> renv::status()
The following package(s) are installed but not recorded in the lockfile:
          _
  nlme      [3.1-148]
  lattice   [0.20-41]
  Matrix    [1.2-18]
  renv      [0.10.0]
  MASS      [7.3-51.6]
  mgcv      [1.8-31]

Use `renv::snapshot()` to add these packages to your lockfile.

The following package(s) are out of sync:

        Package   Lockfile Version   Library Version
             BH           1.72.0-3          1.72.0-3
            DBI              1.1.0             1.1.0
             DT               0.12              0.12
       ISOcodes         2019.04.22        2019.04.22
            NLP              0.2-0             0.2-0
             R6              2.4.1             2.4.1
   RColorBrewer              1.1-2             1.1-2
       RSpectra             0.16-0            0.16-0
           Rcpp            1.0.4.6           1.0.4.6
      RcppEigen          0.3.3.7.0         0.3.3.7.0
      SnowballC              0.6.0             0.6.0
        askpass                1.1               1.1
     assertthat              0.2.1             0.2.1
      backports              1.1.7             1.1.7
      base64enc              0.1-3             0.1-3
          broom              0.5.4             0.5.4
          callr              3.4.3             3.4.3
     cellranger              1.1.0             1.1.0
            cli              2.0.2             2.0.2
          clipr              0.7.0             0.7.0
     colorspace              1.4-1             1.4-1
         crayon              1.3.4             1.3.4
      crosstalk            1.1.0.1           1.1.0.1
           curl                4.3               4.3
     data.table             1.12.8            1.12.8
         dbplyr              1.4.2             1.4.2
           desc              1.2.0             1.2.0
         digest             0.6.25            0.6.25
          dplyr              0.8.4             0.8.4

Can anyone advise on what is going wrong, or provide a working example Dockerfile with R 3.6.x, Renv, and Shiny?

[kevinushey]

You probably need to set RENV_PATHS_CACHE to point to a stable path to be used for the renv cache (e.g. in .Renviron or similar).

Also, are you sure the library path is set correctly as you expect?

If you haven't seen it already, https://rstudio.github.io/renv/articles/docker.html documents what I know to be best practice thus far for using renv in Docker. If there's extra information I can add, please let me know.

[Michael-T64]

Hi,

I'm getting the same issue when trying to deploy my Shiny app to the Shiny Server.

su: ignore --preserve-environment, it's mutually exclusive to --login.

• The project and lockfile are out of sync -- use renv::status() for more details. Warning message: The following package(s) are missing entries in the cache:

  assertthat, base64enc, BH, BiocGenerics, BiocManager, BiocVersion, Cairo, callr, cellranger, cli, colorspace, commonmark, crayon, crosstalk, DBI, desc, digest, dplyr, DT, ellipsis, evaluate, fansi, farver, fastmap, forcats, generics, ggplot2, glue, gtable, highr, hms, htmltools, htmlwidgets, httpuv, isoband, jsonlite, knitr, labeling, later, lattice, lazyeval, lifecycle, magrittr, markdown, MASS, Matrix, mgcv, mime, munsell, nlme, pillar, pkgbuild, pkgconfig, pkgload, plogr, plyr, praise, prettyunits, processx, progress, promises, ps, purrr, R6, RColorBrewer, Rcpp, readxl, rematch, reshape2, rlang, rmarkdown, RMySQL, rprojroot, rstudioapi, scales, shiny, shinyjs, shinythemes, sourcetools, stringi, stringr, testthat, tibble, tidyselect, tinytex, utf8, vctrs, viridisLite, withr, writexl, xfun, xtable, yaml

These packages will need to be reinstalled.

Error in loadNamespace(name) : there is no package called ‘digest’ Calls: local ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart Execution halted

I've checked where RENV_PATHS_CACHE points to using renv::paths$cache() and the directory definitely exists and definitely contains a bunch of packages. (".../.local/share/renv/cache/v5/R-3.6/x86_64-redhat-linux-gnu")

Would you be able to elaborate on what you mean by "set RENV_PATHS_CACHE to point to a stable path to be used for the renv cache" is there a different existing directory that you could suggest I use? (or should I make a new one and copy all of the r packages there?)

I have to admit I only have a very shallow understanding of how renv works under the hood so any level of specificity you're willing to offer would be greatly appreciated!

Thanks!

EDIT: It might be worth mentioning that I'm not using docker. If that changes anything.

EDIT 2: In case anyone else comes across this issue and wants a work-around, renv::isolate() turns off the cache entirely.

[kevinushey]

You could try comparing the symlink paths in the renv library folder; e.g.

paths <- list.files(renv::paths$library(), full.names = TRUE)
Sys.readlink(paths)

to see if the symlink paths match up with what is expected for the cache.

You could also try calling renv::restore() explicitly before launching the application, to make sure the library (and symlinks) are all up-to-date before running the application.

[shruthirames]

hello everyone,

I am facing similar issue. The docker run errors with the following messgae-

I have installed renv(), COPY renv.lock /srv/shiny-server/<>renv.lock, renv.restore() invoked. Still no luck. Any inputs will be highly appreciated!

[zoncrd]

@capncodewash, this is probably a user permissions issue.

As defined into your dockerfile, the expression renv::restore() execute into an R console running as root user. This will cause the global renv cache to be created at /root/.local/share/renv/cache/...

You can easily check it by inspecting the symlinks paths in your renv library directory with the command ls /srv/shiny-server/renv/library/R-3.6/x86_64-pc-linux-gnu -al. In regards to the digest package you mentioned, you should see something like this (apart from the R version)

According to the Shiny server default conf, R Shiny processes run as shiny user, but your R project library points to resources owned by the root user. To solve the problem it is necessary to run the renv::restore() expression into an R console running as shiny user, doing so the global renv cache will be created at /home/shiny/.local/share/renv/cache/....

Below a minimal version of a working dockerfile that solves the problem:

FROM rocker/shiny:4.0.3

RUN apt-get update && apt-get install -y <YOUR_DEPENDENCIES> && apt-get clean && rm -rf /var/lib/apt/lists/*

WORKDIR /srv/shiny-server

COPY --chown=shiny:shiny . ./test_app

RUN cd ./test_app && sudo -u shiny R -e 'renv::restore()'

[tylerlittlefield]

I am also experiencing this exact issue. I'm running shiny-server in a docker container with tyluRp/shiny-server-arm. I have basically:

1. Ran the code in the README of that repository
2. Cloned a shiny app with git clone in the ~/shiny-server/shiny-apps directory
3. Navigate to ~/shiny-server/shiny-apps/myapp directory and run R -e 'renv::restore(confirm = FALSE)'

I was able to get the app running with:

docker run \
    --detach \
    --publish 3838:3838 \
    --volume shiny-apps:/srv/shiny-server/ \
    --volume shiny-logs:/var/log/shiny-server/ \
    --volume shiny-conf:/etc/shiny-server/ \
    --volume /home/pirate/.local/share/renv/:/home/pirate/.local/share/renv/ \ # <---- added this part
    --name shiny-server \
    tylurp/shiny-server-arm

But now I am failing to load a whole bunch of assets. I think I just need to understand Docker more, and how to get my apps working inside a container properly. It could also have something to do with the way I've configured nginx.

Update: Yes, the assets thing seems to be something I've done wrong with nginx configuration. With the container running using the above chunk, it's showing all the CSS and JS with http://<my private ip>:3838/shiny-examples/001-hello/. I removed the following from my nginx config file and all is working:

        #location ~* .(jpg|jpeg|png|gif|ico|css|js)$ {
        #       expires 1s;
        #}

Went a bit off topic but tldr including --volume /home/pirate/.local/share/renv/:/home/pirate/.local/share/renv/ seemed to let my app run.

[kevinushey]

That makes sense. The renv cache needs to be on a volume visible to Docker as well.

[hadley]

Looks like this is resolved by the suggestions in #599