AlexsLemonade / refinebio

Refine.bio harmonizes petabytes of publicly available biological data into ready-to-use datasets for cancer researchers and AI/ML scientists.
https://www.refine.bio/
Other
129 stars 19 forks source link

Docker cache miss during deploy #3340

Open arkid15r opened 1 year ago

arkid15r commented 1 year ago

Context

During deploy process some docker images still being rebuilt from scratch.

Problem or idea

Smasher:

#9 importing cache manifest from ccdlstaging/dr_smasher_cache:latest
#9 DONE 0.3s

#6 [ 1/12] FROM docker.io/ccdlstaging/dr_base:v1.45.7-dev@sha256:776a8c605a6a04914572f59d0be4989b8c9fe05011d62466bd167af90868c2f3
#6 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 1.56MB 0.1s done
#5 DONE 0.1s

#10 [ 2/12] WORKDIR /home/user
#10 CACHED

#11 [ 3/12] COPY workers/R/dependencies/smasher/renv.lock .
#11 DONE 0.1s

#12 [ 4/12] COPY common/R/renv_load.R .
#12 DONE 0.0s

#13 [ 5/12] RUN Rscript renv_load.R
#13 0.463 Installing package into ‘/usr/local/lib/R/site-library’
#13 0.463 (as ‘lib’ is unspecified)
#13 2.414 trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/BiocInstaller_1.28.0.tar.gz'
#13 2.709 Content type 'application/octet-stream' length 18624 bytes (18 KB)
#13 2.722 ==================================================

Salmon:

#9 importing cache manifest from ccdlstaging/dr_salmon_cache:latest
#9 DONE 0.2s

#6 [ 1/14] FROM docker.io/ccdlstaging/dr_base:v1.45.7-dev@sha256:776a8c605a6a04914572f59d0be4989b8c9fe05011d62466bd167af90868c2f3
#6 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 14.06kB 0.1s done
#5 DONE 0.1s

#10 [ 2/14] WORKDIR /home/user
#10 CACHED

#11 [ 3/14] COPY workers/R/dependencies/salmon/renv.lock .
#11 CACHED

#12 [ 4/14] COPY common/R/renv_load.R .
#12 DONE 0.1s

#13 [ 5/14] RUN Rscript renv_load.R
#13 0.358 Installing package into ‘/usr/local/lib/R/site-library’
#13 0.358 (as ‘lib’ is unspecified)
#13 2.507 trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/BiocInstaller_1.28.0.tar.gz'
#13 2.803 Content type 'application/octet-stream' length 18624 bytes (18 KB)
#13 2.821 ==================================================

Solution or next step

Figure out the layer cache invalidation reason and fix it.