"RUN --mount=type=cache" caches not persisted

[charles-dyfis-net]

This isn't a request for a code change as much as for documentation, presuming that there is in fact a reasonable way to reuse the ./cache directory for arbitrary user-provided content from inside the container. If there isn't, then this might be into feature-request territory (which I'd be happy to take a shot at if given some guidance about what an implementation would look like).

Insofar as oci-build-task uses Buildkit, extended syntax is available in Dockerfiles using it; for example:

RUN --mount=type=cache,target=/go-cache env GOCACHE=/go-cache go build

(similar use cases include Maven jar caches, Nix store caches, etc).

Unfortunately, even with caches: [{"path": "cache"}] enabled, this doesn't actually work to reuse downloaded Go modules across builds -- there's an error of the form:

 (*service).Write failed                       error="rpc error: code = Canceled desc = context canceled

...and as far as I can tell, the ./cache directory isn't mapped into the container context.

Is there a reasonable way to cache content inside of oci-build-task?

[ei-grad]

The relevant issue in buildkit - https://github.com/moby/buildkit/issues/1512

[ei-grad]

To sum up, buildkit type=cache mounts are not considered an image layers, and are not exported by the buildkit's --export-cache functionality on which the oci-build-task relies. There are some workarounds suggested, which imply running of two extra build operations (possibly defined in a separate Dockerfile(s)):

• first - to import the cache contents into the type=cache mounts
• second - to export type=cache mounts contents out into some persistent storage

Implementation details could vary, should it rely on docker layer caching, CI caching functionality, or just use rclone to sync the volumes with S3/whatever.

[ei-grad]

Also, there is one a quite different suggestion, which looks more like a real solution to the problem. It proposes to use a dedicated buildkitd server, which would execute actual container build jobs, and clients inside the CI jobs would only send a context to it. I like this solution, but not sure this fits well into the Concourse philosophy.