Reducing duplication across multiple build targets

[cpuguy83]

I currently have a project that requires build 6 different components across 7 distros with 3 CPU architectures. In this project there ends up being a ton of duplication in Dockerfiles. Making changes to anything means I really need to make changes to everything. A bug in a change means a bug everywhere and a change everywhere.

Changes at this scale (which isn't even that large) are extremely difficult to keep track of, be it in one big Dockerfile or across many Dockerfiles (either way is pretty much equal IME). Where I want some uniformity there's always some fat-fingering, things put in Line X when I meant to put in line X+1... basically lots of room for human error.

One potential solution that occurred to me is something along the lines of a new instruction called TEMPLATE (let the bike-shedding commence). TEMPLATE would be treated pretty much like a FROM, the only difference is the context is determined by where it is called from and the stage itself is not cached except within the calling context which can be called multiple times from within different contexts.

As an example

TEMPLATE AS tmpl
RUN echo hello > /tmp/hello
COPY foo /bar

FROM busybox AS builder
RUN echo I'm running some things
COPY --from=tmpl /tmp/hello /hello

Now, this is kind of a weird example because the template is already being executed with the filesystem in the builder stage. Perhaps we need a way to execute a template without having to explicitly copy from it.

TEMPLATE AS build
# install common packages
RUN apt-get update && apt-get install -y libseccomp-dev
WORKDIR /go/src/github.com/opencontainers/runc
RUN --mount=from=src,source=/runc,target=/go/src/github.com/opencontainers/runc make runc

FROM golang AS go

FROM apline AS src
RUN apk add --no-cache git
ARG COMMIT
RUN git clone https://github.com/opencontainers/runc.git && git checkout $COMMIT

FROM ubuntu:18.04
EXECUTE build

FROM ubuntu:20.04
EXECUTE build

FROM ubuntu:22.04
EXECUTE build

FROM debian:buster
EXECUTE build

FROM debian:bullseye
EXECUTE build

Perhaps one could specify a path to a template, where that path would be within the build context:

TEMPLATE /steps.dtmpl AS  common

FROM busybox
EXECUTE common

or

FROM busybox
EXECUTE --from=source=/steps.dtmpl common

Can a template include another template? One may question why so meta, but in the end I don't see why it wouldn't work. It should just be another dependency as far as buildkit would be concerned.

What's stops someone from trying to run apt-get in a busybox container? Nothing. Same as it is today. You'll get an error from the system that apt-get doesn't exist.

This is one of those things that can be easily abused that probably most people don't need but could really soothe the pain of medium to large scale projects.

[sudo-bmitch]

Related: https://github.com/moby/moby/issues/735

[tonistiigi]

I don't really get the first example with COPY --from=tmpl but the second one looks quite similar to this old proposal https://github.com/moby/moby/issues/32100

[brad-jones]

FWIW this sounds similar to Earthly's User Defined Command concept.

see: https://docs.earthly.dev/docs/guides/udc

[mstate]

Or, you could use dagger.io to make a composable Dockerfile. With dagger.io, you could maintain the shared elements independently and compose Dockerfiles.

[cpuguy83]

Definitely understand that dagger supports this, the idea though is to enable this Dockerfile.