asiripanich / emdash

An e-mission deployer's dashboard. See https://github.com/e-mission/e-mission-docs.
https://emdash.amarin.dev
Other
6 stars 3 forks source link

Docker build fails #63

Closed shankari closed 3 years ago

shankari commented 3 years ago

The current docker build on master fails.

It gets past all of the initial steps correctly,

 => [ 2/10] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic  72.9s
 => [ 3/10] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li  0.3s
 => [ 4/10] RUN R -e 'install.packages("remotes")'                                                       5.6s
 => [ 5/10] RUN R -e 'install.packages("devtools")'                                                    561.0s
 => [ 6/10] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                         6.3s
 => [ 7/10] RUN mkdir /build_zone                                                                        0.3s
 => [ 8/10] ADD . /build_zone                                                                            0.5s
 => [ 9/10] WORKDIR /build_zone                                                                          0.0s

but fails on the last step due to some download errors?

 => ERROR [10/10] RUN R -e 'remotes::install_local(upgrade="never")'                                    71.7s

#14 71.61 Error in download.file(url, destfile, method, mode = "wb", ...) :
#14 71.61   (converted from warning) downloaded length 288991 != reported length 581347
#14 71.61 Error in download.packages(pkgs, destdir = tmpd, available = available,  :
#14 71.61   (converted from warning) download of package ‘e1071’ failed
#14 71.61 Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
#14 71.66 Execution halted
Here is a more detailed log of that step

``` => ERROR [10/10] RUN R -e 'remotes::install_local(upgrade="never")' 71.7s ... #14 0.436 > remotes::install_local(upgrade="never") #14 3.671 Installing 109 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, curl, data.table, datamods, DBI, diffobj, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, Rcpp, readr, readxl, rematch, rio, rmarkdown, roxygen2, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, waldo, webshot, wk, xtable #14 3.673 Installing packages into ‘/usr/local/lib/R/site-library’ #14 3.673 (as ‘lib’ is unspecified) #14 4.464 trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz' #14 4.871 Content type 'application/x-gzip' length 79261 bytes (77 KB) #14 4.871 ================================================== #14 4.897 downloaded 77 KB #14 4.897 #14 4.899 trying URL 'https://cran.rstudio.com/src/contrib/backports_1.2.1.tar.gz' #14 5.034 Content type 'application/x-gzip' length 21184 bytes (20 KB) #14 5.034 ================================================== #14 5.036 downloaded 20 KB ... #14 10.60 ================================================== #14 10.60 downloaded 10 KB #14 10.60 #14 10.60 trying URL 'https://cran.rstudio.com/src/contrib/dplyr_1.0.7.tar.gz' #14 10.91 Content type 'application/x-gzip' length 852103 bytes (832 KB) #14 10.92 ================================================== #14 11.06 downloaded 832 KB #14 11.06 #14 11.06 trying URL 'https://cran.rstudio.com/src/contrib/DT_0.18.tar.gz' #14 11.36 Content type 'application/x-gzip' length 1578749 bytes (1.5 MB) #14 11.36 ================================================== #14 11.56 downloaded 1.5 MB #14 11.56 #14 11.57 trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz' #14 11.78 Content type 'application/x-gzip' length 581347 bytes (567 KB) #14 11.78 ======================== #14 71.61 downloaded 282 KB #14 71.61 #14 71.61 Error in download.file(url, destfile, method, mode = "wb", ...) : #14 71.61 (converted from warning) downloaded length 288991 != reported length 581347 #14 71.61 Error in download.packages(pkgs, destdir = tmpd, available = available, : #14 71.61 (converted from warning) download of package ‘e1071’ failed #14 71.61 Calls: ... with_envvar -> force -> force -> i.p -> download.packages #14 71.66 Execution halted ```

shankari commented 3 years ago

@allenmichael099 (in case you want to follow along)

Trying to downgrade since this is currently broken.

Downgrading

``` $ git checkout 72340fb34454e011ac53069b399463925c200d6c M docker_build_and_run.sh Note: switching to '72340fb34454e011ac53069b399463925c200d6c'. HEAD is now at 72340fb Use install_github for DTedit ```

Rebuilding, failed while trying to install package remotes

``` $ docker image rm emdash Error: No such image: emdash kshankar-35069s:emdash kshankar$ docker build -t emdash . [+] Building 327.7s (9/37) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 3.07kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.2s => => transferring context: 1.70MB 0.2s => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag 0.0s => CACHED [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho 0.0s => CACHED [ 4/33] RUN R -e 'install.packages("remotes")' 0.0s => CANCELED [ 5/33] RUN R -e 'install.packages("devtools")' 327.6s context canceled kshankar-35069s:emdash kshankar$ docker build -t emdash --no-cache . [+] Building 80.6s (10/37) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 43B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.1s => => transferring context: 79.23kB 0.1s => CACHED [ 1/33] FROM docker.io/rocker/r-ver:4.0.2 0.0s => [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic 76.0s => [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li 0.4s => [ 4/33] RUN R -e 'install.packages("remotes")' 1.9s => [ 5/33] RUN R -e 'install.packages("devtools")' 1.7s => ERROR [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' 0.6s ------ > [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")': #9 0.384 #9 0.384 R version 4.0.2 (2020-06-22) -- "Taking Off Again" #9 0.384 Copyright (C) 2020 The R Foundation for Statistical Computing #9 0.384 Platform: x86_64-pc-linux-gnu (64-bit) #9 0.384 #9 0.384 R is free software and comes with ABSOLUTELY NO WARRANTY. #9 0.384 You are welcome to redistribute it under certain conditions. #9 0.384 Type 'license()' or 'licence()' for distribution details. #9 0.384 #9 0.384 R is a collaborative project with many contributors. #9 0.384 Type 'contributors()' for more information and #9 0.384 'citation()' on how to cite R or R packages in publications. #9 0.384 #9 0.384 Type 'demo()' for some demos, 'help()' for on-line help, or #9 0.384 'help.start()' for an HTML browser interface to help. #9 0.384 Type 'q()' to quit R. #9 0.384 #9 0.519 > remotes::install_github("r-lib/remotes", ref = "97bbf81") #9 0.521 Error in loadNamespace(name) : there is no package called ‘remotes’ #9 0.521 Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart #9 0.521 Execution halted ------ executor failed running [/bin/sh -c R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")']: exit code: 1 kshankar-35069s:emdash kshankar$ docker build -t emdash --no-cache . [+] Building 195.4s (9/37) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 43B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.1s => => transferring context: 79.23kB 0.1s => CACHED [ 1/33] FROM docker.io/rocker/r-ver:4.0.2 0.0s => [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic 70.8s => [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li 0.3s => [ 4/33] RUN R -e 'install.packages("remotes")' 4.9s => CANCELED [ 5/33] RUN R -e 'install.packages("devtools")' 119.3s context canceled kshankar-35069s:emdash kshankar$ docker build -t emdash . [+] Building 0.6s (10/37) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 43B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.1s => => transferring context: 79.23kB 0.1s => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2 0.0s => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag 0.0s => CACHED [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho 0.0s => CACHED [ 4/33] RUN R -e 'install.packages("remotes")' 0.0s => CACHED [ 5/33] RUN R -e 'install.packages("devtools")' 0.0s => ERROR [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' 0.5s ------ > [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")': #9 0.372 #9 0.372 R version 4.0.2 (2020-06-22) -- "Taking Off Again" #9 0.372 Copyright (C) 2020 The R Foundation for Statistical Computing #9 0.372 Platform: x86_64-pc-linux-gnu (64-bit) #9 0.372 #9 0.372 R is free software and comes with ABSOLUTELY NO WARRANTY. #9 0.372 You are welcome to redistribute it under certain conditions. #9 0.372 Type 'license()' or 'licence()' for distribution details. #9 0.372 #9 0.372 R is a collaborative project with many contributors. #9 0.372 Type 'contributors()' for more information and #9 0.372 'citation()' on how to cite R or R packages in publications. #9 0.372 #9 0.372 Type 'demo()' for some demos, 'help()' for on-line help, or #9 0.372 'help.start()' for an HTML browser interface to help. #9 0.372 Type 'q()' to quit R. #9 0.372 #9 0.465 > remotes::install_github("r-lib/remotes", ref = "97bbf81") #9 0.466 Error in loadNamespace(name) : there is no package called ‘remotes’ #9 0.466 Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart #9 0.466 Execution halted ------ executor failed running [/bin/sh -c R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")']: exit code: 1 ```

Commented out the DTEdit lines

``` diff --git a/Dockerfile b/Dockerfile index caf55a4..9393843 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,8 +3,8 @@ RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl')" >> /usr/local/lib/R/etc/Rprofile.site RUN R -e 'install.packages("remotes")' RUN R -e 'install.packages("devtools")' -RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' -RUN Rscript -e 'devtools::install_github("jbryer/DTedit")' +# RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' +# RUN Rscript -e 'devtools::install_github("jbryer/DTedit")' RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")' RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")' RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")' ``` ``` => [11/31] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")' 11.2s => ERROR [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.7. 407.8s ------ > [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.7.8")': #15 407.8 Error in download_version_url(package, version, repos, type) : #15 407.8 version '2.7.8' is invalid for package 'mapview' #15 407.8 Calls: -> download_version_url #15 407.8 Execution halted ------ executor failed running [/bin/sh -c Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.7.8")']: exit code: 1 ```

shankari commented 3 years ago
Upgraded mapview to 2.10.0. worked!

``` kshankar-35069s:emdash kshankar$ docker build -t emdash . [+] Building 2185.4s (36/36) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 43B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [ 1/31] FROM docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.1s => => transferring context: 91.56kB 0.1s => CACHED [ 2/31] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag 0.0s => CACHED [ 3/31] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho 0.0s => CACHED [ 4/31] RUN R -e 'install.packages("remotes")' 0.0s => CACHED [ 5/31] RUN R -e 'install.packages("devtools")' 0.0s => CACHED [ 6/31] RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")' 0.0s => CACHED [ 7/31] RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1") 0.0s => CACHED [ 8/31] RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0") 0.0s => CACHED [ 9/31] RUN Rscript -e 'remotes::install_version("data.table",upgrade="never", version = "1. 0.0s => CACHED [10/31] RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "1.5" 0.0s => CACHED [11/31] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0 0.0s => [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")' 1073.7s => [13/31] RUN Rscript -e 'remotes::install_version("ggplot2",upgrade="never", version = "3.3.0")' 19.0s => [14/31] RUN Rscript -e 'remotes::install_version("mongolite",upgrade="never", version = "2.2.0")' 30.2s => [15/31] RUN Rscript -e 'remotes::install_version("shinydashboard",upgrade="never", version = "0.7.1 6.7s => [16/31] RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.14")' 8.2s => [17/31] RUN Rscript -e 'remotes::install_version("flexdashboard",upgrade="never", version = "0.5.2 15.4s => [18/31] RUN Rscript -e 'remotes::install_version("esquisse",upgrade="never", version = "0.3.0")' 19.1s => [19/31] RUN Rscript -e 'remotes::install_version("lubridate",upgrade="never", version = "1.7.8")' 32.7s => [20/31] RUN Rscript -e 'remotes::install_version("dplyr",upgrade="never", version = "1.0.2")' 18.0s => [21/31] RUN Rscript -e 'remotes::install_version("leaflet",upgrade="never", version = "2.0.3")' 9.3s => [22/31] RUN Rscript -e 'remotes::install_version("janitor",upgrade="never", version = "2.0.1")' 22.2s => [23/31] RUN Rscript -e 'remotes::install_version("tidyr",upgrade="never", version = "1.1.0")' 26.5s => [24/31] RUN Rscript -e 'remotes::install_version("shinycssloaders",upgrade="never", version = "0. 323.5s => [25/31] RUN Rscript -e 'remotes::install_version("plotly",upgrade="never", version = "4.9.2.1")' 21.1s => [26/31] RUN Rscript -e 'remotes::install_version("randomcoloR",upgrade="never", version = "1.1.0.1 48.7s => [27/31] RUN Rscript -e 'remotes::install_version("sf",upgrade="never", version = "0.9-5")' 127.6s => [28/31] RUN mkdir /build_zone 0.3s => [29/31] ADD . /build_zone 0.2s => [30/31] WORKDIR /build_zone 0.0s => [31/31] RUN R -e 'remotes::install_local(upgrade="never")' 365.2s => exporting to image 17.6s => => exporting layers 17.6s => => writing image sha256:fc2836ca5a8cc9fb78f8a48bb35f85d0f7173bc04a1391433f0f2086a7d2a8cb 0.0s => => naming to docker.io/library/emdash 0.0s ```

shankari commented 3 years ago
Restoring DTEdit, fails again

``` $ docker build -t emdash . [+] Building 8.1s (11/37) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 3.07kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.2s => => transferring context: 82.26kB 0.2s => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2 0.0s => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag 0.0s => CACHED [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho 0.0s => CACHED [ 4/33] RUN R -e 'install.packages("remotes")' 0.0s => CACHED [ 5/33] RUN R -e 'install.packages("devtools")' 0.0s => [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' 7.5s => ERROR [ 7/33] RUN Rscript -e 'devtools::install_github("jbryer/DTedit")' 0.5s ------ > [ 7/33] RUN Rscript -e 'devtools::install_github("jbryer/DTedit")': #10 0.471 Error in loadNamespace(name) : there is no package called ‘devtools’ #10 0.471 Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart #10 0.471 Execution halted ------ executor failed running [/bin/sh -c Rscript -e 'devtools::install_github("jbryer/DTedit")']: exit code: 1 ```

allenmichael099 commented 3 years ago

We wondered if the default download timeout was too low, so Shankari is attempting to build with an altered timeout.

Based on this from https://search.r-project.org/R/refmans/utils/html/download.file.html: The timeout for many parts of the transfer can be set by the option timeout which defaults to 60 seconds. This is often insufficient for downloads of large files (50MB or more) and so should be increased when download.file is used in packages to do so. Note that the user can set the default timeout by the environment variable R_DEFAULT_INTERNET_TIMEOUT in recent versions of R, so to ensure that this is not decreased packages should use something like

options(timeout = max(300, getOption("timeout")))

And from https://stackoverflow.com/questions/35282928/how-do-i-set-a-timeout-for-utilsdownload-file-in-r,

I think we just need to alter timeout within the options call, setting to whatever number of seconds we want

RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl',timeout = 300)" >> /usr/local/lib/R/etc/Rprofile.site
shankari commented 3 years ago

Fixed the error there is no package called ‘devtools’ by changing it to remotes like everything else.

-RUN Rscript -e 'devtools::install_github("jbryer/DTedit")'
+RUN Rscript -e 'remotes::install_github("jbryer/DTedit")'
But still ran into a timeout/install issue

``` => ERROR [ 7/33] RUN Rscript -e 'remotes::install_github("jbryer/DTedit")' 57.4s ------ > [ 7/33] RUN Rscript -e 'remotes::install_github("jbryer/DTedit")': #10 0.938 Downloading GitHub repo jbryer/DTedit@master #10 4.417 base64enc (NA -> 0.1-3 ) [CRAN] #10 4.417 bslib (NA -> 0.2.5.1) [CRAN] #10 4.417 crosstalk (NA -> 1.1.1 ) [CRAN] #10 4.417 DT (NA -> 0.18 ) [CRAN] #10 4.417 htmltools (NA -> 0.5.1.1) [CRAN] #10 4.417 htmlwidgets (NA -> 1.5.3 ) [CRAN] #10 4.417 httpuv (NA -> 1.6.1 ) [CRAN] #10 4.417 jquerylib (NA -> 0.1.4 ) [CRAN] #10 4.417 later (NA -> 1.2.0 ) [CRAN] #10 4.417 lazyeval (NA -> 0.2.2 ) [CRAN] #10 4.417 promises (NA -> 1.2.0.1) [CRAN] #10 4.417 sass (NA -> 0.4.0 ) [CRAN] #10 4.417 shiny (NA -> 1.6.0 ) [CRAN] #10 4.417 sourcetools (NA -> 0.1.7 ) [CRAN] #10 4.417 xtable (NA -> 1.8-4 ) [CRAN] #10 4.419 Installing 15 packages: base64enc, bslib, crosstalk, DT, htmltools, htmlwidgets, httpuv, jquerylib, later, lazyeval, promises, sass, shiny, sourcetools, xtable #10 4.421 Installing packages into ‘/usr/local/lib/R/site-library’ #10 4.421 (as ‘lib’ is unspecified) #10 5.124 trying URL 'https://cran.rstudio.com/src/contrib/base64enc_0.1-3.tar.gz' #10 5.326 Content type 'application/x-gzip' length 7833 bytes #10 5.326 ================================================== #10 5.326 downloaded 7833 bytes ... #10 8.747 trying URL 'https://cran.rstudio.com/src/contrib/shiny_1.6.0.tar.gz' #10 9.008 Content type 'application/x-gzip' length 4170497 bytes (4.0 MB) #10 57.31 #10 57.31 downloaded 63 KB #10 57.31 #10 57.31 Error in download.file(url, destfile, method, mode = "wb", ...) : #10 57.31 (converted from warning) downloaded length 64755 != reported length 4170497 #10 57.31 Error in download.packages(pkgs, destdir = tmpd, available = available, : #10 57.31 (converted from warning) download of package ‘shiny’ failed #10 57.31 Calls: ... with_envvar -> force -> force -> i.p -> download.packages #10 57.32 Execution halted ------ executor failed running [/bin/sh -c Rscript -e 'remotes::install_github("jbryer/DTedit")']: exit code: 1 ```

Trying after upping the timeout.

shankari commented 3 years ago

Got past that point, but failed again. I wonder if there are limitations on disk size.

Failed again during download

``` => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 3.08kB 0.0s => [internal] load .dockerignore 0.0s => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.1s => => transferring context: 94.60kB 0.1s => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag 0.0s => [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li 0.3s => [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' 14.0s => [ 7/33] RUN Rscript -e 'remotes::install_github("jbryer/DTedit")' 405.3s => [ 8/33] RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")' 5.3s => [ 9/33] RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")' 17.8s => [10/33] RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")' 33.0s => [11/33] RUN Rscript -e 'remotes::install_version("data.table",upgrade="never", version = "1.12.8") 36.9s => [12/33] RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "1.5")' 5.9s => [13/33] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")' 15.0s => ERROR [14/33] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10 417.8s ------ > [14/33] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")': #17 2.347 Downloading package from url: https://cran.rstudio.com/src/contrib/mapview_2.10.0.tar.gz #17 6.622 Installing 40 packages: classInt, colorspace, cpp11, DBI, e1071, farver, geojsonsf, geometries, ggplot2, gridExtra, gtable, isoband, jsonify, labeling, leafem, leaflet, leaflet.providers, leafpop, munsell, plyr, png, proxy, rapidjsonr, raster, RColorBrewer, s2, satellite, scales, servr, sf, sfheaders, sp, svglite, systemfonts, units, uuid, viridis, viridisLite, webshot, wk #17 6.626 Installing packages into ‘/usr/local/lib/R/site-library’ #17 6.626 (as ‘lib’ is unspecified) #17 7.878 trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz' #17 8.094 Content type 'application/x-gzip' length 403884 bytes (394 KB) #17 8.094 ================================================== #17 8.190 downloaded 394 KB #17 8.190 #17 8.196 trying URL 'https://cran.rstudio.com/src/contrib/colorspace_2.0-2.tar.gz' #17 8.603 Content type 'application/x-gzip' length 2196563 bytes (2.1 MB) #17 8.603 ================================================== #17 8.945 downloaded 2.1 MB #17 8.945 #17 8.949 trying URL 'https://cran.rstudio.com/src/contrib/cpp11_0.3.1.tar.gz' #17 9.145 Content type 'application/x-gzip' length 217952 bytes (212 KB) #17 9.145 ================================================== #17 9.200 downloaded 212 KB #17 9.200 #17 9.205 trying URL 'https://cran.rstudio.com/src/contrib/DBI_1.1.1.tar.gz' #17 9.617 Content type 'application/x-gzip' length 743802 bytes (726 KB) #17 9.617 ================================================== #17 9.690 downloaded 726 KB #17 9.690 #17 9.697 trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz' #17 10.22 Content type 'application/x-gzip' length 581347 bytes (567 KB) #17 10.22 ================================================== #17 10.28 downloaded 567 KB #17 10.28 #17 10.28 trying URL 'https://cran.rstudio.com/src/contrib/farver_2.1.0.tar.gz' #17 10.79 Content type 'application/x-gzip' length 1281864 bytes (1.2 MB) #17 10.79 ================================================= #17 417.8 downloaded 1.2 MB #17 417.8 #17 417.8 Error in download.file(url, destfile, method, mode = "wb", ...) : #17 417.8 (converted from warning) downloaded length 1261298 != reported length 1281864 #17 417.8 Error in download.packages(pkgs, destdir = tmpd, available = available, : #17 417.8 (converted from warning) download of package ‘farver’ failed #17 417.8 Calls: ... with_envvar -> force -> force -> i.p -> download.packages #17 417.8 Execution halted ------ executor failed running [/bin/sh -c Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")']: exit code: 1 ```

shankari commented 3 years ago

Aha! https://forums.docker.com/t/cannot-change-default-20g-size-limit-on-windows-docker-host/31879 But that is only on windows.

And the successfully built image is only 3GB.

emdash                                         latest        fc2836ca5a8c   4 hours ago     2.62GB
shankari commented 3 years ago

Going back to the successfully built image (checkout to 72340fb34454e011ac53069b399463925c200d6c and remove the DTEdit stuff)

It built again successfully ``` docker build -t emdash . [+] Building 733.9s (11/35) [+] Building 735.4s (11/35) [+] Building 5145.0s (36/36) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 43B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.0.2 0.0s => [internal] load build context 0.1s => => transferring context: 79.23kB 0.1s => [ 1/31] FROM docker.io/rocker/r-ver:4.0.2 0.0s => CACHED [ 2/31] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag 0.0s => CACHED [ 3/31] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho 0.0s => CACHED [ 4/31] RUN R -e 'install.packages("remotes")' 0.0s => CACHED [ 5/31] RUN R -e 'install.packages("devtools")' 0.0s => CACHED [ 6/31] RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")' 0.0s => [ 7/31] RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")' 352.5s => [ 8/31] RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")' 632.7s => [ 9/31] RUN Rscript -e 'remotes::install_version("data.table",upgrade="never", version = "1.12.8") 24.0s => [10/31] RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "1.5")' 4.6s => [11/31] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")' 10.5s => [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")' 2284.5s => [13/31] RUN Rscript -e 'remotes::install_version("ggplot2",upgrade="never", version = "3.3.0")' 476.8s => [14/31] RUN Rscript -e 'remotes::install_version("mongolite",upgrade="never", version = "2.2.0")' 29.7s => [15/31] RUN Rscript -e 'remotes::install_version("shinydashboard",upgrade="never", version = "0.7.1 6.7s => [16/31] RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.14")' 7.7s => [17/31] RUN Rscript -e 'remotes::install_version("flexdashboard",upgrade="never", version = "0.5.2 15.5s => [18/31] RUN Rscript -e 'remotes::install_version("esquisse",upgrade="never", version = "0.3.0")' 63.4s => [19/31] RUN Rscript -e 'remotes::install_version("lubridate",upgrade="never", version = "1.7.8")' 30.8s => [20/31] RUN Rscript -e 'remotes::install_version("dplyr",upgrade="never", version = "1.0.2")' 18.4s => [21/31] RUN Rscript -e 'remotes::install_version("leaflet",upgrade="never", version = "2.0.3")' 13.1s => [22/31] RUN Rscript -e 'remotes::install_version("janitor",upgrade="never", version = "2.0.1")' 21.2s => [23/31] RUN Rscript -e 'remotes::install_version("tidyr",upgrade="never", version = "1.1.0")' 27.7s => [24/31] RUN Rscript -e 'remotes::install_version("shinycssloaders",upgrade="never", version = "0.3" 6.7s => [25/31] RUN Rscript -e 'remotes::install_version("plotly",upgrade="never", version = "4.9.2.1")' 20.5s => [26/31] RUN Rscript -e 'remotes::install_version("randomcoloR",upgrade="never", version = "1.1.0.1 51.3s => [27/31] RUN Rscript -e 'remotes::install_version("sf",upgrade="never", version = "0.9-5")' 151.8s => [28/31] RUN mkdir /build_zone 0.4s => [29/31] ADD . /build_zone 0.4s => [30/31] WORKDIR /build_zone 0.0s => [31/31] RUN R -e 'remotes::install_local(upgrade="never")' 888.1s => exporting to image 5.7s => => exporting layers 5.7s => => writing image sha256:fc0e6b055d52c8963d1753e4fb775af2bd8de7d7abbe360be7aa86b1970f0063 0.0s => => naming to docker.io/library/emdash 0.0s ```

And manually installing the packages works

docker run followed by copy/pasted R script ``` R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' R version 4.0.2 (2020-06-22) -- "Taking Off Again" Copyright (C) 2020 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > remotes::install_github("r-lib/remotes", ref = "97bbf81") Downloading GitHub repo r-lib/remotes@97bbf81 ✔ checking for file ‘/tmp/Rtmp7dzsLc/remotesd4e89362a/r-lib-remotes-97bbf81/DESCRIPTION’ ... ─ preparing ‘remotes’: ✔ checking DESCRIPTION meta-information ... ─ checking for LF line-endings in source and make files and shell scripts ─ checking for empty or unneeded directories ─ building ‘remotes_2.0.2.tar.gz’ Installing package into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) * installing *source* package ‘remotes’ ... ** using staged installation ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (remotes) > > root@a6c68d04f4ea:/src# Rscript -e 'remotes::install_github("jbryer/DTedit")' Downloading GitHub repo jbryer/DTedit@master DT (0.14 -> 0.18 ) [CRAN] magrittr (1.5 -> 2.0.1) [CRAN] shiny (1.5.0 -> 1.6.0) [CRAN] Installing 3 packages: DT, magrittr, shiny Installing packages into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) trying URL 'https://cran.rstudio.com/src/contrib/DT_0.18.tar.gz' Content type 'application/x-gzip' length 1578749 bytes (1.5 MB) ================================================== downloaded 1.5 MB trying URL 'https://cran.rstudio.com/src/contrib/magrittr_2.0.1.tar.gz' Content type 'application/x-gzip' length 265580 bytes (259 KB) ================================================== downloaded 259 KB trying URL 'https://cran.rstudio.com/src/contrib/shiny_1.6.0.tar.gz' Content type 'application/x-gzip' length 4170497 bytes (4.0 MB) ================================================== downloaded 4.0 MB * installing *source* package ‘magrittr’ ... ** package ‘magrittr’ successfully unpacked and MD5 sums checked ** using staged installation ** libs gcc -I"/usr/local/lib/R/include" -DNDEBUG -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c pipe.c -o pipe.o gcc -I"/usr/local/lib/R/include" -DNDEBUG -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c utils.c -o utils.o gcc -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o magrittr.so pipe.o utils.o -L/usr/local/lib/R/lib -lR installing to /usr/local/lib/R/site-library/00LOCK-magrittr/00new/magrittr/libs ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices *** copying figures ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location ** checking absolute paths in shared objects and dynamic libraries ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (magrittr) * installing *source* package ‘shiny’ ... ** package ‘shiny’ successfully unpacked and MD5 sums checked ** using staged installation ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices *** copying figures ** building package indices ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (shiny) * installing *source* package ‘DT’ ... ** package ‘DT’ successfully unpacked and MD5 sums checked ** using staged installation ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (DT) The downloaded source packages are in ‘/tmp/RtmpGEQ84i/downloaded_packages’ ✔ checking for file ‘/tmp/RtmpGEQ84i/remotesbb694ecdc0/jbryer-DTedit-0c248c0/DESCRIPTION’ ... ─ preparing ‘DTedit’: ✔ checking DESCRIPTION meta-information ... ─ checking for LF line-endings in source and make files and shell scripts ─ checking for empty or unneeded directories ─ building ‘DTedit_1.0.0.tar.gz’ Installing package into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) * installing *source* package ‘DTedit’ ... ** using staged installation ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices ** building package indices ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (DTedit) ```
shankari commented 3 years ago

Trying to go to master again

Installing fails

``` [+] Building 19.8s (4/14) [+] Building 20.1s (4/14) [+] Building 21.2s (4/14) [+] Building 732.6s (14/14) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 43B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.1.0 0.6s => [ 1/10] FROM docker.io/rocker/r-ver:4.1.0@sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d1 38.4s => => resolve docker.io/rocker/r-ver:4.1.0@sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d10ef 0.0s => => sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d10ef5d73fd681597 951B / 951B 0.0s => => sha256:d9f4d826c19c1d24c823472294d912b42dca8ddffbd34b2e3d02d5b304f98a4a 4.14kB / 4.14kB 0.0s => => sha256:3b480e2036f0f2c081e4cdd2aaa32e5b6c26d60a397792c2a31ea32d34c57805 18.79kB / 18.79kB 0.2s => => sha256:643395aa8431033c0fe27ebcbe0f7b802599b75c2b08b820905e7c8a9c4e0f24 295.81MB / 295.81MB 27.2s => => extracting sha256:3b480e2036f0f2c081e4cdd2aaa32e5b6c26d60a397792c2a31ea32d34c57805 0.1s => => extracting sha256:643395aa8431033c0fe27ebcbe0f7b802599b75c2b08b820905e7c8a9c4e0f24 11.0s => [internal] load build context 0.1s => => transferring context: 91.85kB 0.1s => [ 2/10] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic 66.4s => [ 3/10] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li 0.3s => [ 4/10] RUN R -e 'install.packages("remotes")' 4.8s => [ 5/10] RUN R -e 'install.packages("devtools")' 535.3s => [ 6/10] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' 6.9s => [ 7/10] RUN mkdir /build_zone 0.3s => [ 8/10] ADD . /build_zone 0.3s => [ 9/10] WORKDIR /build_zone 0.0s => ERROR [10/10] RUN R -e 'remotes::install_local(upgrade="never")' 79.1s #14 79.01 #14 79.01 Error in download.file(url, destfile, method, mode = "wb", ...) : #14 79.01 (converted from warning) downloaded length 26586 != reported length 215365 #14 79.01 Error in download.packages(pkgs, destdir = tmpd, available = available, : #14 79.01 (converted from warning) download of package ‘janitor’ failed #14 79.01 Calls: ... with_envvar -> force -> force -> i.p -> download.packages #14 79.05 Execution halted ------ executor failed running [/bin/sh -c R -e 'remotes::install_local(upgrade="never")']: exit code: 1 ```

Installing without `remotes::install_local(upgrade="never")'` succeeds

``` $ docker build -t emdash . [+] Building 9.5s (15/15) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 1.06kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/rocker/r-ver:4.1.0 1.2s => [auth] rocker/r-ver:pull token for registry-1.docker.io 0.0s => [1/9] FROM docker.io/rocker/r-ver:4.1.0@sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d10ef 0.0s => [internal] load build context 0.1s => => transferring context: 92.87kB 0.1s => CACHED [2/9] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagem 0.0s => CACHED [3/9] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method 0.0s => CACHED [4/9] RUN R -e 'install.packages("remotes")' 0.0s => CACHED [5/9] RUN R -e 'install.packages("devtools")' 0.0s => CACHED [6/9] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")' 0.0s => CACHED [7/9] RUN mkdir /build_zone 0.0s => [8/9] ADD . /build_zone 0.2s => [9/9] WORKDIR /build_zone 0.0s => exporting to image 7.8s => => exporting layers 7.8s => => writing image sha256:9fcdad76e5532bb51a0a3690e70fa892ed7fdac68021340309276fe2f22d477a 0.0s => => naming to docker.io/library/emdash 0.0s ```

Trying to install within the container instead of image build fails while installing crosstalk

``` docker run -it -p 1234:80 -v $PWD:/src --name emdash emdash /bin/bash root@b7efc26cf072:/build_zone# cd /src/ CODE_OF_CONDUCT.md emdash.Rproj _pkgdown.yml DESCRIPTION .git/ ports.txt dev/ .github/ R/ docker_build_and_run.sh .gitignore .Rbuildignore Dockerfile inst/ README.md Dockerfile.orig LICENSE README.Rmd dockerfile.patch LICENSE.md test-data/ .Dockerfile.swp man/ tests/ docker_run.sh NAMESPACE docker_start_test_mongodb.sh NEWS.md root@b7efc26cf072:/build_zone# R -e 'remotes::install_local(upgrade="never")' R version 4.1.0 (2021-05-18) -- "Camp Pontanezen" Copyright (C) 2021 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > remotes::install_local(upgrade="never") Installing 105 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, data.table, datamods, DBI, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, jsonlite, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, readr, readxl, rematch, rio, rmarkdown, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, webshot, wk, xtable Installing packages into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz' Content type 'application/x-gzip' length 79261 bytes (77 KB) ================================================== downloaded 77 KB trying URL 'https://cran.rstudio.com/src/contrib/backports_1.2.1.tar.gz' Content type 'application/x-gzip' length 21184 bytes (20 KB) ================================================== downloaded 20 KB trying URL 'https://cran.rstudio.com/src/contrib/base64enc_0.1-3.tar.gz' Content type 'application/x-gzip' length 7833 bytes ================================================== downloaded 7833 bytes trying URL 'https://cran.rstudio.com/src/contrib/bit_4.0.4.tar.gz' Content type 'application/x-gzip' length 279723 bytes (273 KB) ================================================== downloaded 273 KB trying URL 'https://cran.rstudio.com/src/contrib/bit64_4.0.5.tar.gz' Content type 'application/x-gzip' length 135091 bytes (131 KB) ================================================== downloaded 131 KB trying URL 'https://cran.rstudio.com/src/contrib/bslib_0.2.5.1.tar.gz' Content type 'application/x-gzip' length 3810412 bytes (3.6 MB) ================================================== downloaded 3.6 MB trying URL 'https://cran.rstudio.com/src/contrib/cellranger_1.1.0.tar.gz' Content type 'application/x-gzip' length 63857 bytes (62 KB) ================================================== downloaded 62 KB trying URL 'https://cran.rstudio.com/src/contrib/checkmate_2.0.0.tar.gz' Content type 'application/x-gzip' length 168716 bytes (164 KB) ================================================== downloaded 164 KB trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz' Content type 'application/x-gzip' length 403884 bytes (394 KB) ================================================== downloaded 394 KB trying URL 'https://cran.rstudio.com/src/contrib/colorspace_2.0-2.tar.gz' Content type 'application/x-gzip' length 2196563 bytes (2.1 MB) ================================================== downloaded 2.1 MB trying URL 'https://cran.rstudio.com/src/contrib/config_0.3.1.tar.gz' Content type 'application/x-gzip' length 69056 bytes (67 KB) ================================================== downloaded 67 KB trying URL 'https://cran.rstudio.com/src/contrib/cpp11_0.3.1.tar.gz' Content type 'application/x-gzip' length 217952 bytes (212 KB) ================================================== downloaded 212 KB trying URL 'https://cran.rstudio.com/src/contrib/credentials_1.3.1.tar.gz' Content type 'application/x-gzip' length 230470 bytes (225 KB) ================================================== downloaded 225 KB trying URL 'https://cran.rstudio.com/src/contrib/crosstalk_1.1.1.tar.gz' Content type 'application/x-gzip' length 676543 bytes (660 KB) == downloaded 32 KB Error in download.file(url, destfile, method, mode = "wb", ...) : (converted from warning) downloaded length 32768 != reported length 676543 Error in download.packages(pkgs, destdir = tmpd, available = available, : (converted from warning) download of package ‘crosstalk’ failed Calls: ... with_envvar -> force -> force -> i.p -> download.packages Execution halted ```

And in attempt

``` root@b7efc26cf072:/build_zone# echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', timeout = 300)" >> /usr/local/lib/R/etc/Rprofile.site root@b7efc26cf072:/build_zone# R -e 'remotes::install_local(upgrade="never")' R version 4.1.0 (2021-05-18) -- "Camp Pontanezen" Copyright (C) 2021 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > remotes::install_local(upgrade="never") Installing 105 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, data.table, datamods, DBI, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, jsonlite, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, readr, readxl, rematch, rio, rmarkdown, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, webshot, wk, xtable Installing packages into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz' Content type 'application/x-gzip' length 79261 bytes (77 KB) =============== downloaded 24 KB Error in download.file(url, destfile, method, mode = "wb", ...) : (converted from warning) downloaded length 25333 != reported length 79261 Error in download.packages(pkgs, destdir = tmpd, available = available, : (converted from warning) download of package ‘attempt’ failed Calls: ... with_envvar -> force -> force -> i.p -> download.packages Execution halted ```

And, even after switching to the mounted drive, and increasing the timeout, fails in contrib

``` root@b7efc26cf072:/build_zone# cd /src/ root@b7efc26cf072:/src# R -e 'remotes::install_local(upgrade="never")' R version 4.1.0 (2021-05-18) -- "Camp Pontanezen" Copyright (C) 2021 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > remotes::install_local(upgrade="never") Installing 105 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, data.table, datamods, DBI, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, jsonlite, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, readr, readxl, rematch, rio, rmarkdown, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, webshot, wk, xtable Installing packages into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz' Content type 'application/x-gzip' length 79261 bytes (77 KB) ================================================== downloaded 77 KB trying URL 'https://cran.rstudio.com/src/contrib/backports_1.2.1.tar.gz' Content type 'application/x-gzip' length 21184 bytes (20 KB) ================================================== downloaded 20 KB ... ================================================== downloaded 212 KB trying URL 'https://cran.rstudio.com/src/contrib/credentials_1.3.1.tar.gz' Content type 'application/x-gzip' length 230470 bytes (225 KB) === downloaded 15 KB Error in download.file(url, destfile, method, mode = "wb", ...) : (converted from warning) downloaded length 15999 != reported length 230470 Error in download.packages(pkgs, destdir = tmpd, available = available, : (converted from warning) download of package ‘credentials’ failed Calls: ... with_envvar -> force -> force -> i.p -> download.packages Execution halted ```

And in every one of the failures, we were partway through downloading a file, the download failed, and then we got the error about the downloaded length

trying URL 'https://cran.rstudio.com/src/contrib/crosstalk_1.1.1.tar.gz'
Content type 'application/x-gzip' length 676543 bytes (660 KB)
==

downloaded 32 KB

trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz'
Content type 'application/x-gzip' length 79261 bytes (77 KB)
===============
downloaded 24 KB

trying URL 'https://cran.rstudio.com/src/contrib/credentials_1.3.1.tar.gz'
Content type 'application/x-gzip' length 230470 bytes (225 KB)
===
downloaded 15 KB

So it looks like the problem is really with the network downloads failing randomly. Downloading dependencies as part of remotes::install_local(upgrade="never") seems to be fragile, when we were downloading dependencies using remotes::install_version, it was a lot more stable.

That is the main difference between:

shankari commented 3 years ago

Trying to workaround by creating the docker container with dependencies from 72340fb34454e011ac53069b399463925c200d6c and then switching to master and running R -e 'remotes::install_local(upgrade="never")'

error

Mapping without trajectories
Warning: Error in st_geometry.sf: attr(obj, "sf_column") does not point to a geometry column.
Did you rename it, without setting st_geometry(obj) <- "newname"?
  55: stop
  54: st_geometry.sf
  52: [.sf
  50: dropListColumns
  49: create_filters
  48: <observer>
   5: runApp
   3: print.shiny.appobj
   1: emdash::run_app
disconnecting from the emission collections..

giving up at this point since I am not sure whether or not this is related to docker.

shankari commented 3 years ago

Final try:

Created an instance from the published image ``` docker run -it -p 1234:80 -v $PWD:/src --name emdash amarins/emdash /bin/bash Unable to find image 'amarins/emdash:latest' locally latest: Pulling from amarins/emdash a31c7b29f4ad: Already exists 8ed946bf07db: Already exists 80b12a566f68: Already exists 7efaf74e9032: Pull complete ae6bfc8e2aa9: Pull complete e774a89efc36: Pull complete a8634475f08b: Pull complete bfd17ff19f89: Pull complete 7062d71fc235: Pull complete 1fb9a5c6b03e: Pull complete 4f4fb700ef54: Pull complete f9f587febd40: Pull complete Digest: sha256:7cc00d5478c4e312a5b553992e5c9366e9c118451aa87ba86561aacfc67f824e Status: Downloaded newer image for amarins/emdash:latest ```
Tried to run from the mounted directory, it failed with the same error (`"sf_column") does not point to a geometry column.`) ``` root@4b4af9e1ff2c:/build_zone# cd /src/ > remotes::install_local(upgrade="never") Skipping install of 'emdash' from a local remote, the SHA1 (1.4.1.90) has not changed since last install. Use `force = TRUE` to force installation > > root@4b4af9e1ff2c:/src# R -e 'remotes::install_local(upgrade="never", force=TRUE)' R version 4.1.0 (2021-05-18) -- "Camp Pontanezen" Copyright (C) 2021 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > remotes::install_local(upgrade="never", force=TRUE) ✔ checking for file ‘/tmp/Rtmp0FOmhQ/file2356edbae4/src/DESCRIPTION’ ... ─ preparing ‘emdash’: ✔ checking DESCRIPTION meta-information ... ─ checking for LF line-endings in source and make files and shell scripts ─ checking for empty or unneeded directories Omitted ‘LazyData’ from DESCRIPTION ─ building ‘emdash_1.4.1.9001.tar.gz’ Installing package into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified) * installing *source* package ‘emdash’ ... ** using staged installation ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices *** copying figures ** building package indices ** testing if installed package can be loaded from temporary location ** testing if installed package can be loaded from final location ** testing if installed package keeps a record of temporary installation path * DONE (emdash) > > root@4b4af9e1ff2c:/src# R -e "options('shiny.port'=80,shiny.host='0.0.0.0');emdash::run_app(mongo_url = '$MONGO_URL', anon_locations=$ANON_LOCATIONS)" R version 4.1.0 (2021-05-18) -- "Camp Pontanezen" Copyright (C) 2021 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > options('shiny.port'=80,shiny.host='0.0.0.0');emdash::run_app(mongo_url = 'mongodb://host.docker.internal:27017', anon_locations=FALSE) No config file given, the default config file will be used. Loading required package: shiny Listening on http://0.0.0.0:80 First trip: 2020-09-22 Last trip: 2021-08-01 Warning: 'esquisserUI' is deprecated. Use 'esquisse_ui' instead. See help("Deprecated") and help("esquisse-deprecated"). Warning: 'filterDF_UI / filterDF' is deprecated. Use 'datamods::filter_data_ui / datamods::filter_data_server' instead. See help("Deprecated") and help("esquisse-deprecated"). Running: mod_load_data_server Running: mod_load_trips_server Warning: 'esquisserServer' is deprecated. Use 'esquisse_server' instead. See help("Deprecated") and help("esquisse-deprecated"). About to load participants merging trip summaries with participants merging server call summaries with participants Finished merging server calls Finished loading participants Participants size is: 17.2 kB kb About to load Bike Check in Finished loading Bike Check in About to load Polar Bear Finished loading Polar Bear The dates reactive values are: 2021-07-26 to 2021-08-02 Window_width is 7 days get_n_trips_in_query: The time stamps for that date range are: 1627257600, 1627862400 There are 17 trips and 905 locations in the date range Load_trips observed About to load trips Finished query, about to clean trips Finished cleaning trips Finished loading trips Trips size is: 54 kB Warning in grSoftVersion() : unable to load shared object '/usr/local/lib/R/modules//R_X11.so': libXt.so.6: cannot open shared object file: No such file or directory Warning: Removed 1 rows containing non-finite values (stat_count). Mapping without trajectories Warning: Error in st_geometry.sf: attr(obj, "sf_column") does not point to a geometry column. Did you rename it, without setting st_geometry(obj) <- "newname"? 54: stop 53: st_geometry.sf 51: [.sf 49: dropListColumns 48: create_filters 47: 4: runApp 3: print.shiny.appobj 1: emdash::run_app disconnecting from the emission collections.. ```
shankari commented 3 years ago

Final final try (run the image directly), @asiripanich this still fails with the same error:

$ docker run -d -p 1234:80 -v $PWD:/src --name emdash amarins/emdash
4941cac11b45fd02e8fae24313402f427ad97f56d91134645bafe911de300875

$ docker logs -f emdash
R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

> options('shiny.port'=80,shiny.host='0.0.0.0');emdash::run_app(mongo_url = 'mongodb://host.docker.internal:27017', anon_locations=FALSE)
No config file given, the default config file will be used.
Loading required package: shiny

Listening on http://0.0.0.0:80
First trip: 2020-09-22
Last trip:  2021-08-01
Warning: 'esquisserUI' is deprecated.
Use 'esquisse_ui' instead.
See help("Deprecated") and help("esquisse-deprecated").
Warning: 'filterDF_UI / filterDF' is deprecated.
Use 'datamods::filter_data_ui / datamods::filter_data_server' instead.
See help("Deprecated") and help("esquisse-deprecated").
Running: mod_load_data_server
Running: mod_load_trips_server
Warning: 'esquisserServer' is deprecated.
Use 'esquisse_server' instead.
See help("Deprecated") and help("esquisse-deprecated").
About to load participants
merging trip summaries with participants
merging server call summaries with participants
Finished merging server calls
Finished loading participants
Participants size is: 17.2 kB kb
About to load Bike Check in
Finished loading Bike Check in
About to load Polar Bear
Finished loading Polar Bear
The dates reactive values are: 2021-07-26 to 2021-08-02
Window_width is 7 days
get_n_trips_in_query: The time stamps for that date range are:
1627257600, 1627862400
There are 17 trips and 905 locations in the date range
Load_trips observed
About to load trips
Finished query, about to clean trips
Finished cleaning trips
Finished loading trips
Trips size is: 54 kB
Warning in grSoftVersion() :
  unable to load shared object '/usr/local/lib/R/modules//R_X11.so':
  libXt.so.6: cannot open shared object file: No such file or directory
Warning: Removed 1 rows containing non-finite values (stat_count).
Mapping without trajectories
Warning: Error in st_geometry.sf: attr(obj, "sf_column") does not point to a geometry column.
Did you rename it, without setting st_geometry(obj) <- "newname"?
  54: stop
  53: st_geometry.sf
  51: [.sf
  49: dropListColumns
  48: create_filters
  47: <observer>
   4: runApp
   3: print.shiny.appobj
   1: emdash::run_app
disconnecting from the emission collections..
asiripanich commented 3 years ago

Bad news.. I no longer can build images on Docker Hub. Their free tier benefits have changed. Only paying customers are allowed to use their CI.

shankari commented 3 years ago

@asiripanich we can use github packages instead. https://docs.github.com/en/actions/guides/publishing-docker-images

But I don't think that the most recent error is related to building images. I can use the published image since there have not been any docker changes since then, mount my local directory, rebuild and re-run. But I'm still getting the error above, and the only issues related to it that we can find seem to be about importing sf correctly. https://github.com/afrimapr/afrihealthsites/issues/1

shankari commented 3 years ago

Fortunately, we got the bright idea of trying with the docker image on @allenmichael099's laptop, and we have now narrowed this down to a data issue.

I'm going to try to dump my database so we can see if we can reproduce this on Michael's laptop and figure out the data issue.

This is no longer a docker build issue, so tracking this in https://github.com/asiripanich/emdash/issues/68