coder / envbuilder

Build development environments from a Dockerfile on Docker, Kubernetes, and OpenShift. Enable developers to modify their development environment quickly.
Apache License 2.0
171 stars 28 forks source link

coder/kaniko: support multi-stage builds with DoCacheProbe #230

Closed johnstcn closed 3 months ago

johnstcn commented 5 months ago

https://github.com/coder/kaniko/pull/12 added support for performing a 'fake' build (e.g. ProbeCache)

However, this does not appear to work with multi-stage builds (https://github.com/coder/kaniko/pull/12#issuecomment-2160534514)

Add support for multi-stage builds. This will require changes to our Kaniko fork.

mafredri commented 4 months ago

To support multi-stage cache probes, we'll need a way to deduce the hash of dependencies between stages when operating in cache probe-mode.

Some ideas off the top of my head:

  1. Use a tar reader/filesystem on the previous stage cache archive
    • Kaniko uses os and reads files directly, this will fail unless the previous stage has been extracted
    • Potentially slow, may require decompression and reading of the entire archive
    • Faster than normal extraction, small files are part of the tar stream and don't add overhead
  2. Allow cached stage to be extracted
    • Slow, potentially very little benefit in cache probing
  3. Modify cache archive/image to include metadata (i.e. file hashes of cross-stage dependencies)?
    • Probably doable, we can pre-compute cross-stage dependencies
    • Probably jank
  4. Through some magic, allow the stage archive/hash to represent the cache-ability of the directive

I'm open to better ideas here.