`docker buildx bake` does not support saving multiple targets to a single tarball

negasora commented 1 year ago

Contributing guidelines

[X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

[X] ... the documentation does not mention anything about my problem
[X] ... there are no open or closed issues that are related to my problem

Description

Attempting to output multiple targets to a single tarball results in a single random target being contained in the tarball, instead of all of them.

Expected behaviour

The generated tarball contains all images

Actual behaviour

The generated tarball contains a random image

Buildx version

github.com/docker/buildx 0.10.3 79e156beb11f697f06ac67fa1fb958e4762c0fab

Docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.10.3
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.16.0
    Path:     /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 18
  Running: 1
  Paused: 0
  Stopped: 17
 Images: 691
 Server Version: 23.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640.m
 runc version: 
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.2.1-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.13GiB
 Name: pudge
 ID: 6IXA:H6AE:DQUM:AHGY:WMLU:ZWJQ:BOJX:LRKK:EOH4:UJ7Y:ZN23:WRMK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

NAME/NODE           DRIVER/ENDPOINT             STATUS  BUILDKIT PLATFORMS
beautiful_shamir *  docker-container                             
  beautiful_shamir0 unix:///var/run/docker.sock running v0.10.5  linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default             docker                                       
  default           default                     running 23.0.1   linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Configuration

With the following files,

docker-bake.hcl:

group "default" {
  targets = ["redis", "python"]
}

target "redis" {
    dockerfile = "Dockerfile_redis"
    output = ["type=docker,dest=images.tar"]
    tags = ["test_redis:latest"]
}

target "python" {
    dockerfile = "Dockerfile_python"
    output = ["type=docker,dest=images.tar"]
    tags = ["test_python:latest"]
}

Dockerfile_python:

FROM python:3.11-alpine

Dockerfile_redis:

FROM redis:7-alpine

docker buildx bake -f docker-bake.hcl

Logs

No response

Additional info

No response

ThomasDebrunner commented 1 year ago

crazy-max commented 1 year ago

The generated tarball contains all images

What's your use case?

Build semantic does not have the same purpose as the docker save. If you really want this, you need to output each tarball and join them with docker save. Would also need syncable output with https://github.com/docker/buildx/pull/1197.

ThomasDebrunner commented 1 year ago

We build images for a remote, offline, embedded system. We have no intention to run the containers on the build machine. To get the images onto the embedded system, we copy the them over as an oci tarball.

In order to get a single tarball with all images, we currently load the images in the local docker, only that we can then do docker save to get them out as a single OCI tarball. Especially for large images, this is very slow. If there's only one image, we can just use buildx bake directly to oci tarball, without loading into local docker, which is much faster.

docker / buildx