ThinkR-open / dockerfiler

Easy Dockerfile Creation from R
https://thinkr-open.github.io/dockerfiler/
Other
169 stars 25 forks source link

[BUG] Dockerfile_base fails to execute when generated with `add_dockerfile_with_renv_shinyproxy()` #66

Closed campbead closed 8 months ago

campbead commented 9 months ago

add_dockerfile_with_renv_shinyproxy() generates files normally as expected. However the current version of renvseems to be causing difficulty with deploying.

Here is the Dockerfile_base produced by add_dockerfile_with_renv_shinyproxy() for my project:

FROM rocker/verse:4.2.2
RUN mkdir -p /usr/local/lib/R/etc/ /usr/lib/R/etc/
RUN echo "options(renv.config.pak.enabled = TRUE, repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
RUN R -e 'install.packages(c("renv","remotes"))'
COPY renv.lock renv.lock
RUN R -e 'renv::restore()'

However when this files is used to created a docker container using this command docker build -f Dockerfile_base --progress=plain -t golem.test.project_base .

I get the following error:

Screenshot 2023-10-26 at 7 52 53 PM

I have attempted fix by changing Dockerfile_base to force an install of an older version of renv to:

FROM rocker/verse:4.2.2
RUN mkdir -p /usr/local/lib/R/etc/ /usr/lib/R/etc/
RUN echo "options(renv.config.pak.enabled = TRUE, repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
RUN R -e 'install.packages("remotes")'
RUN R -e 'remotes::install_version("renv", version = "0.17.3")'
COPY renv.lock renv.lock
RUN R -e 'renv::restore()'

which results in a successful build when using the docker build command.

This seems to be a problem with renv but am asking for help here for two reasons.

  1. I am not sure exactly how to document this issue for a bug report to renv and any help with that would be awesome
  2. The current way Dockerfile_base files are created using add_dockerfile_with_renv_shinyproxy() is using the most current version of renv and not the version of renv used in the lockfile. It would be nice to at least have this option with this package prevent this particular bug.

Thank you for your help.

To Reproduce Steps to reproduce the behavior:

  1. create new golem project with some minimal amount of content
  2. run renv::init()to create a lockfile
  3. run add_dockerfile_with_renv_shinyproxy(lockfile = "renv.lock")
  4. run the first docker build command in the generated README file

session info

─ Session info ──────────────────────────────────────────────────────────────────────────────── setting value version R version 4.2.2 (2022-10-31) os macOS Ventura 13.3.1 system aarch64, darwin20 ui RStudio language (EN) collate en_US.UTF-8 ctype en_US.UTF-8 tz Pacific/Auckland date 2023-10-26 rstudio 2023.09.1+494 Desert Sunflower (desktop) pandoc NA

─ Packages ──────────────────────────────────────────────────────────────────────────────────── ! package * version date (UTC) lib source P attempt 0.3.1 2020-05-03 [?] CRAN (R 4.2.0) P cachem 1.0.6 2021-08-19 [?] CRAN (R 4.2.0) P callr 3.7.3 2022-11-02 [?] CRAN (R 4.2.0) P cli 3.5.0 2022-12-20 [?] CRAN (R 4.2.0) P config 0.3.1 2020-12-17 [?] CRAN (R 4.2.0) P crayon 1.5.2 2022-09-29 [?] CRAN (R 4.2.0) P desc 1.4.2 2022-09-08 [?] CRAN (R 4.2.0) P devtools 2.4.5 2022-10-11 [?] CRAN (R 4.2.0) P digest 0.6.31 2022-12-11 [?] CRAN (R 4.2.0) P dockerfiler 0.2.1 2023-01-18 [?] CRAN (R 4.2.0) P ellipsis 0.3.2 2021-04-29 [?] CRAN (R 4.2.0) P fastmap 1.1.0 2021-01-25 [?] CRAN (R 4.2.0) P fs 1.5.2 2021-12-08 [?] CRAN (R 4.2.0) P glue 1.6.2 2022-02-24 [?] CRAN (R 4.2.0) P golem 0.4.1 2023-06-05 [?] CRAN (R 4.2.0) P htmltools 0.5.4 2022-12-07 [?] CRAN (R 4.2.0) P htmlwidgets 1.6.0 2022-12-15 [?] CRAN (R 4.2.0) P httpuv 1.6.7 2022-12-14 [?] CRAN (R 4.2.0) P jsonlite 1.8.4 2022-12-06 [?] CRAN (R 4.2.0) P later 1.3.0 2021-08-18 [?] CRAN (R 4.2.0) P lifecycle 1.0.3 2022-10-07 [?] CRAN (R 4.2.0) P magrittr 2.0.3 2022-03-30 [?] CRAN (R 4.2.0) P memoise 2.0.1 2021-11-26 [?] CRAN (R 4.2.0) P mime 0.12 2021-09-28 [?] CRAN (R 4.2.0) P miniUI 0.1.1.1 2018-05-18 [?] CRAN (R 4.2.0) P pak 0.6.0 2023-08-29 [?] RSPM (R 4.2.0) P pkgbuild 1.4.0 2022-11-27 [?] CRAN (R 4.2.0) P pkgload 1.3.2 2022-11-16 [?] CRAN (R 4.2.0) P prettyunits 1.1.1 2020-01-24 [?] CRAN (R 4.2.0) P processx 3.8.0 2022-10-26 [?] CRAN (R 4.2.0) P profvis 0.3.7 2020-11-02 [?] CRAN (R 4.2.0) P promises 1.2.0.1 2021-02-11 [?] CRAN (R 4.2.0) P ps 1.7.2 2022-10-26 [?] CRAN (R 4.2.0) P purrr 1.0.0 2022-12-20 [?] CRAN (R 4.2.0) P R6 2.5.1 2021-08-19 [?] CRAN (R 4.2.0) P Rcpp 1.0.9 2022-07-08 [?] CRAN (R 4.2.0) P remotes 2.4.2 2021-11-30 [?] CRAN (R 4.2.0) P renv 1.0.3 2023-09-19 [?] RSPM (R 4.2.0) P rlang 1.0.6 2022-09-24 [?] CRAN (R 4.2.0) P rprojroot 2.0.3 2022-04-02 [?] CRAN (R 4.2.0) P rstudioapi 0.14 2022-08-22 [?] CRAN (R 4.2.0) P sessioninfo 1.2.2 2021-12-06 [?] CRAN (R 4.2.0) P shiny 1.7.4 2022-12-15 [?] CRAN (R 4.2.0) P stringi 1.7.8 2022-07-11 [?] CRAN (R 4.2.0) P stringr 1.5.0 2022-12-02 [?] CRAN (R 4.2.0) P urlchecker 1.0.1 2021-11-30 [?] CRAN (R 4.2.0) P usethis 2.1.6 2022-05-25 [?] CRAN (R 4.2.0) P vctrs 0.5.1 2022-11-16 [?] CRAN (R 4.2.0) P withr 2.5.0 2022-03-03 [?] CRAN (R 4.2.0) P xtable 1.8-4 2019-04-21 [?] CRAN (R 4.2.0) P yaml 2.3.6 2022-10-18 [?] CRAN (R 4.2.0)

[1] /Users/adamc/Library/Caches/org.R-project.R/R/renv/library/golem.test.actions-7c1a2f83/R-4.2/aarch64-apple-darwin20 [2] /Users/adamc/Library/Caches/org.R-project.R/R/renv/sandbox/R-4.2/aarch64-apple-darwin20/fd29d0b8

P ── Loaded and on-disk path mismatch.

VincentGuyader commented 9 months ago

thanks a lot,

i will have a look to this

campbead commented 9 months ago

I have found the pakpackage causes the problem observed here with the renv::restore() command.

Modifiying Dockerfile_base so to disable pak during the renv::restore() command results in a successful build:

FROM rocker/verse:4.2.2
RUN mkdir -p /usr/local/lib/R/etc/ /usr/lib/R/etc/
RUN echo "options(renv.config.pak.enabled = TRUE, repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
RUN R -e 'install.packages(c("renv","remotes"))'
COPY renv.lock renv.lock
RUN echo "options(renv.config.pak.enabled = FALSE)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
RUN R -e 'renv::restore()'

output from docker build command:

FROM rocker/verse:4.2.2
RUN mkdir -p /usr/local/lib/R/etc/ /usr/lib/R/etc/
RUN echo "options(renv.config.pak.enabled = TRUE, repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
RUN R -e 'install.packages(c("renv","remotes"))'
COPY renv.lock renv.lock
COPY renv_run.R renv_run.R
RUN R -e 'source("renv_run.R")'
adam@ahab:~/github-action-deploy/pak_option$ vim  Dockerfile_base
adam@ahab:~/github-action-deploy/pak_option$ sudo docker build -f Dockerfile_base --progress=plain -t golem.test.project_base .
Sending build context to Docker daemon  44.03kB
Step 1/7 : FROM rocker/verse:4.2.2
 ---> dadab58f18c1
Step 2/7 : RUN mkdir -p /usr/local/lib/R/etc/ /usr/lib/R/etc/
 ---> Using cache
 ---> c73357013ce0
Step 3/7 : RUN echo "options(renv.config.pak.enabled = TRUE, repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
 ---> Using cache
 ---> b415f5e2bec4
Step 4/7 : RUN R -e 'install.packages(c("renv","remotes"))'
 ---> Using cache
 ---> cd478876509a
Step 5/7 : COPY renv.lock renv.lock
 ---> Using cache
 ---> bcb15c5eb906
Step 6/7 : RUN echo "options(renv.config.pak.enabled = FALSE)" | tee /usr/local/lib/R/etc/Rprofile.site | tee /usr/lib/R/etc/Rprofile.site
 ---> Running in bfaeb3a719fd
options(renv.config.pak.enabled = FALSE)
Removing intermediate container bfaeb3a719fd
 ---> f50ae6d794a4
Step 7/7 : RUN R -e 'renv::restore()'
 ---> Running in 531d6bc09f93

R version 4.2.2 (2022-10-31) -- "Innocent and Trusting"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> renv::restore()
The following package(s) will be updated:

# CRAN -----------------------------------------------------------------------
- highr        [repo: RSPM -> CRAN; ver: 0.9 -> 0.10]
- remotes      [2.4.2.1 -> 2.4.2]
- xfun         [repo: RSPM -> CRAN; ver: 0.35 -> 0.36]
- attachment   [* -> 0.3.1]
- attempt      [* -> 0.3.1]
- config       [* -> 0.3.1]
- golem        [* -> 0.4.1]
- here         [* -> 1.0.1]

# RSPM -----------------------------------------------------------------------
- rematch      [1.0.1 -> 2.0.0]
- uuid         [1.1-0 -> 1.1-1]
- parsedate    [* -> 1.3.1]
- rhub         [* -> 1.1.2]

# Repository -----------------------------------------------------------------
- renv         [repo: CRAN -> *; ver: 1.0.3 -> 0.17.3]

# Downloading packages -------------------------------------------------------
- Querying repositories for available source packages ... Done!
- Downloading highr from CRAN ...               OK [37.9 Kb in 1.3s]
- Downloading xfun from CRAN ...                OK [376.2 Kb in 2.1s]
- Downloading uuid from RSPM ...                OK [44.5 Kb in 1.2s]
- Downloading rematch from RSPM ...             OK [15.1 Kb in 1.2s]
- Downloading remotes from CRAN ...             OK [388.9 Kb in 2.0s]
- Downloading renv from Repository ...          OK [1.9 Mb in 2.8s]
- Downloading attachment from CRAN ...          OK [223.7 Kb in 2.1s]
- Downloading attempt from CRAN ...             OK [104.5 Kb in 1.2s]
- Downloading config from CRAN ...              OK [78.1 Kb in 1.8s]
- Downloading golem from CRAN ...               OK [1.1 Mb in 1.2s]
- Downloading here from CRAN ...                OK [51.1 Kb in 1.2s]
- Downloading parsedate from RSPM ...           OK [59.7 Kb in 1.4s]
- Downloading rhub from RSPM ...                OK [956 Kb in 1.7s]
Successfully downloaded 13 packages in 29 seconds.

# Installing packages --------------------------------------------------------
- Installing xfun ...                           OK [installed binary and cached in 0.39s]
- Installing highr ...                          OK [installed binary and cached in 0.34s]
- Installing uuid ...                           OK [installed binary and cached in 0.33s]
- Installing rematch ...                        OK [installed binary and cached in 0.32s]
- Installing remotes ...                        OK [installed binary and cached in 0.37s]
- Installing renv ...                           OK [installed binary and cached in 0.54s]
- Installing attachment ...                     OK [installed binary and cached in 0.58s]
- Installing attempt ...                        OK [installed binary and cached in 0.47s]
- Installing config ...                         OK [installed binary and cached in 0.35s]
- Installing here ...                           OK [installed binary and cached in 0.35s]
- Installing golem ...                          OK [installed binary and cached in 0.68s]
- Installing parsedate ...                      OK [installed binary and cached in 0.33s]
- Installing rhub ...                           OK [installed binary and cached in 0.75s]

The following loaded package(s) have been updated:
- renv
Restart your R session to use the new versions.

>
>
Removing intermediate container 531d6bc09f93
 ---> 1fdbb4112c24
Successfully built 1fdbb4112c24
Successfully tagged golem.test.project_base:latest
VincentGuyader commented 8 months ago

same error for me :

image

VincentGuyader commented 8 months ago

👍 no issue with :

golem::add_dockerfile_with_renv_shinyproxy(output_dir = "deploy")

image

👎 issue with :

golem::add_dockerfile_with_renv_shinyproxy(lockfile = "renv.lock",output_dir = "deploy")

image

👍 no issue with :

attachment::create_renv_for_prod() golem::add_dockerfile_with_renv_shinyproxy(output_dir = "deploy",lockfile = "renv.lock.prod" )

👎 issue with :

attachment::create_renv_for_dev(output = "renv.lock.dev") golem::add_dockerfile_with_renv_shinyproxy(output_dir = "deploy",lockfile = "renv.lock.dev" )

but not the same issue :

image

👎 issue with :

image

VincentGuyader commented 8 months ago

each time setting renv.config.pak.enabled = FALSE correct the issue...

VincentGuyader commented 8 months ago

I will set FALSE by default... the issue seems more an issue with pak than renv or dockerfiler

campbead commented 8 months ago

Perhaps this bugfix can be its own release version? I would definitely help my deployment pipeline if it were available off CRAN.

VincentGuyader commented 8 months ago

Can you please confirm me that the branch https://github.com/ThinkR-open/dockerfiler/tree/rc_2.2 correct this issue ?

We made a cran submission for version 2.2 last week. We have one strange issue on macos do manage before the release

Regards

campbead commented 8 months ago

Yes I can confirm that using rc_2.2 allowed a succesful golem::add_dockerfile_with_renv_shinyproxy(lockfile = "renv.lock",output_dir = "deploy") execution.

Thank you.

VincentGuyader commented 8 months ago

0.2.2 is "on his way on cran"