Add multi-stage builds for Docker

[ghost]

This issue was originally opened by @finferflu as hashicorp/packer#9462. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

---

Please search the existing issues for relevant feature requests, and use the reaction feature (https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to add upvotes to pre-existing requests.

Feature Description

Add the ability to create multi-stage builds.

Use Case(s)

This is vital to keep the image sizes down, e.g. I have a Dockerfile which I can't translate to Packer:

FROM prom/prometheus:v2.19.0

FROM alpine:3.10.2
RUN apk add gettext

COPY --from=0 /bin/prometheus /bin/prometheus

RUN mkdir -p /prometheus /etc/prometheus && \
chown -R nobody:nogroup etc/prometheus /prometheus
# Run envsubst before Prometheus.
RUN echo $'#!/bin/sh\n\
envsubst < /etc/prometheus/orig.yml > /etc/prometheus/prometheus.yml && \
exec /bin/prometheus "$@"' \
> /etc/prometheus/entrypoint.sh
RUN chmod +x /etc/prometheus/entrypoint.sh
ENTRYPOINT ["/etc/prometheus/entrypoint.sh"]

CMD [ "--config.file=/etc/prometheus/prometheus.yml", \
"--storage.tsdb.path=/prometheus" ]
USER nobody
EXPOSE 9090
VOLUME [ "/prometheus" ]
WORKDIR /prometheus

# This is your local prometheus.yml.
ADD prometheus.yml /etc/prometheus/orig.yml

[brayra]

Here is a Go application which returns the ip of the web caller. It is important to make image as small as possible. With this two stage Dockerfile, it is only 6mb for docker image:

package main

import (
    "net"
    "net/http"
    "strings"
)

func main() {
    http.HandleFunc("/", ExampleHandler)
    if err := http.ListenAndServe(":8080", nil); err != nil {
        panic(err)
    }
}

func ExampleHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Add("Content-Type", "application/json")

    w.Write([]byte(GetIP(r)))
}

// GetIP gets a requests IP address by reading off the forwarded-for
// header (for proxies) and falls back to use the remote address.
func GetIP(r *http.Request) string {
    forwarded := r.Header.Get("X-FORWARDED-FOR")
    ips := strings.Split(forwarded, ", ")
    if forwarded != "" {
        return ips[0]
    }
    ip, _, err := net.SplitHostPort(r.RemoteAddr)
    if err != nil {
        return "Unknown"
    }
    return ip
}

Here is the Dockerfile

FROM golang:alpine AS builder

# Set necessary environmet variables needed for our image
ENV GO111MODULE=off \
    CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64

# Move to working directory /build
WORKDIR /build

# Copy the code into the container
COPY myip.go .

# Build the application
RUN go build -o main .

# Move to /dist directory as the place for resulting binary folder
WORKDIR /dist

FROM scratch

# Need SSL Certificates
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# copy build executable to scratch image
COPY --from=builder /build/main /go/bin/main

# Export necessary port
EXPOSE 8080

# Command to run when starting the container
CMD ["/go/bin/main"]

[leopck]

I'm also facing similar roadblock on my side, we need a multi-stage build for Packer to reduce the size of the image, what's the status of this current issue?

[samuelbsource]

I am going to piggyback this. I was very excited about packer and all but it does not make creating docker images much more straightforward. Single binary images (and distroless images where single binary won't work) should be easy to do without the need for extra steps, otherwise, it defies the point.

Having a provisioner that can extract files from prebuilt images would be a good start.

[russoz]

Hi, I am learning about packer these days (exactly because I have a project to build multiple images out of a template) and packer came in handy as an option for that. But now that I see the multi-stage build is not yet available, I think I will actually build a sample image with and without packer, see how big the difference is.

I wish this could be prioritized somehow.

[github-actions[bot]]

This issue has been synced to JIRA for planning.

JIRA ID: HPR-769

[S0LERA]

Any updates on this?

We are considering the use of packer to improve the use of pipelines for building container images, but the fact that packer doesn't have multi-stage images support is really a stopper for this.

[kumadee]

Hi Folks. We are heavily using packer for our image builds. Recently, we saw the need to use multistage builds to reduce the footprint of our images. It would be really cool if this feature is added soon, otherwise back to writing ContainerFiles/DockerFiles 😭

[yb66]

Perhaps something hacky around this[0] could be used?

You can extract files from an image with the following commands:

container_id=$(docker create "$image")
docker cp "$container_id:$source_path" "$destination_path"
docker rm "$container_id"

[0] https://unix.stackexchange.com/questions/331645/extract-file-from-docker-image

[kumadee]

Sadly, after seeing no progress on this topic, we stopped using packer for container image build and switched to Containerfiles. 😔

[mloskot]

@kumadee

use multistage builds to reduce the footprint of our images.

I've been multi-stage building Windows container images with Dockerfile-s for very long time. I've recently switched to the Packer which is a different approach (run container, commit as image) and I have not noticed any increase size of generate image. I'm not defending Packer's lack of the multi-stage builds, just sharing my observation.

[kmcduffee-verisk]

I have a proposed solution/extension to have this work.

1. Define multiple source images: one with the base/build image and one defining the final image:

source "docker" "build_image" {
    image = "debian:12-slim"
    commit = true
    ...
}

source "docker" "final_image" {
    image = "myapp:build-image"
    commit = true
    ...
}

2. Extend the build block types to accept a <BLOCK LABEL> value, much like most other resource blocks in HCL (see: https://developer.hashicorp.com/packer/docs/templates/hcl_templates):

build "build_image" {
  sources = ["source.docker.debian"]
  ...
  post-processors {
    post-processor "docker-tag" {
      repository =  "myapp"
      tags = ["build-image"]
    }
  }
}

build "final_image" {
  sources = ["source.docker.final_image"]
  ...
  post-processors {
    post-processor "docker-tag" {
      repository =  "myapp"
      tags = ["final-image"]
    }
  }
}

You would probably need/want to support exported properties from the build resources, in order to ensure the builds are done in the order you want (or else use a depends_on or similar)