bake: support manifests

[jkroepke]

Description

Hi,

I would like to use buildx bake to build windows images.

Images Docker images has the requirement that OS and host matches. In conclusion, I need a docker images for WIndows Server 2019 and Windows Server 2022.

Pushing images to the same tag will create a dedicated manifest of each target and overrides the other targets.

For example

target "windows-nanoserver-ltsc2019" {
  inherits = ["docker-metadata-action"]
  tags = ["app:latest"]

  platforms = ["windows/amd64"]

  target = "windows"
  args = {
    NANOSERVER_VERSION = "ltsc2019"
  }
}

target "windows-nanoserver-ltsc2022" {
  inherits = ["docker-metadata-action"]
  tags = ["app:latest"]

  platforms = ["windows/amd64"]

  target = "windows"
  args = {
    NANOSERVER_VERSION = "ltsc2022"
  }
}

Push targets push images to app:latest, after completion, only one image will be present in the manifest. I would be nice to setup an target which covers the manifest generation.

[tonistiigi]

This is ultimately blocked on platform string format atm not defining "osversion" component so it can't be specified with "platform" and it is merged together on creating manifest index. Some recent discussion on this: https://github.com/moby/buildkit/pull/4387#discussion_r1378284196 .

But maybe we can do some Bake-specific hack to allow this merge of targets from different definitions. Not sure how that would look like though. @crazy-max any ideas?

Atm. it should be possible to build both targets together with different name (or "push-by-digest=true") and then merge them together with buildx imagetools create.

[jkroepke]

Atm. it should be possible to build both targets together with different name (or "push-by-digest=true") and then merge them together with buildx imagetools create.

I'm doing buildx imagetools create at the moment I had the hope this could be integrated into bake, too.

Define "osversion" is not mandatory, since it will inherit from the Base image.

[crazy-max]

But maybe we can do some Bake-specific hack to allow this merge of targets from different definitions. Not sure how that would look like though. @crazy-max any ideas?

I wonder if we could reuse the matrix feature for this case and a new builtin-variable like BAKE_MATRIX_MERGE_RESULTS:

target "windows" {
  name = "windows-${nanoversion}"
  inherits = ["docker-metadata-action"]
  tags = ["app:latest"]
  platforms = ["windows/amd64"]
  matrix = {
    nanoversion = ["ltsc2019", "ltsc2022"]
  }
  target = "windows"
  args = {
    BAKE_MATRIX_MERGE_RESULTS = 1
  }
}

But then platforms = ["windows/amd64"] is wrong as stipulated by OP there would be only one manifest. Need to think a bit more about this.

[tonistiigi]

Define "osversion" is not mandatory, since it will inherit from the Base image.

The logic would be that you define windows/amd64/osver1, windows/amd64/osver2, windows/arm64/osver2 as platforms and then the correct base image would be resolved based on that. Without no extra targets of hacks in Dockerfile being needed.

@crazy-max Theoretically we could just check that 2 targets have the same "mergeable" output definition and if such targets are built together with the same bake command we could just run the merge step afterward.

[jkroepke]

What about more specific configuration for manifests?

target "latest" {
  type = "manifest"
  includes = [target.windows, target.linux]
}

[tonistiigi]

@jkroepke We don't have concept of "type" for target atm nor includes like this. In that case we would probably add a new top-level object (like target, group, variable) instead, but then what makes sure that the properties for target.windows and target.linux are mergeable at all? Or if the output is defined in this type=manifest layer then what happens with the outputs defined in actual targets.

[jkroepke]

If the matrix array would support list of objects, something like this would be also possible:

target "windows" {
  name = "${target.os}-${image.platform}"
  inherits = ["docker-metadata-action"]
  tags = ["app:latest"]
  platforms = [image.platform]
  matrix = {
    image = [
      {platform = "linux/amd64", target = "linux", base = "ubuntu"},
      {platform = "linux/arm64", target = "linux", base = "ubuntu"},
      {platform = "windows/amd64", target = "linux", base = "mcr.microsoft.com/windows/nanoserver:ltsc2019"},
      {platform = "windows/amd64", target = "linux", base = "mcr.microsoft.com/windows/nanoserver:ltsc2022"}
    ]
  }
  target = image.target
  args = {
    BASE_IMAGE = image.base
  }
}

[tonistiigi]

Matrix supports structs, but matrix is for building all combinations of configurations as separate targets. All configs can define their own outputs etc.

[jkroepke]

Okay, just dumped some ideas here that I had.

[crazy-max]

windows/amd64/osver1

Afaik Windows supports variants like windows/arm/v7. I'm afraid we can paint ourselves into a corner if we assume this format without having smth upstream first for the string representation using OSVersion/OSFeatures.

But maybe if we have a prefix like osversion it would be ok to avoid collision with variant:

• windows/amd64/osversion:10.0.17763.4974 (ltsc2019)
• windows/amd64/osversion:10.0.20348.2031 (ltsc2022)