Support for multiple platforms in jib build strategy

gansheer commented 1 year ago

Component

Kubernetes Maven Plugin

Is your enhancement related to a problem? Please describe

We are experimenting with maven plugins providing JIB in the Apache Camel-k project. We really like the results from our tests with the current JKube Kubernetes Maven Plugin (with -Djkube.build.strategy=jib), but it lacks the support for multiple platform we want to introduce as a part of the new feature that is JIB for our project.

Describe the solution you'd like

Our goal is to be able to build the container image using JIB a list of platform architectures (linux/am64, linux/arm64, etc). The ideal solution would be to provide some configuration defining the platforms we want to build images for like this one or anything else that fits more the way the configuration is done in the Kubernetes Maven Plugin or Openshift Maven Plugin.

From previous discussion, the change would be in the JKube Kit and as such could be used in either the Kubernetes Maven Plugin and the Openshift Maven plugin.

Describe alternatives you've considered

I have considered using the other plugins tools for my features : Quarkus Container Extension Jib, Google's Jib Maven Plugin, Fabric8 Docker Plugin. While the Google's Jib Maven Plugin might works for our use case, we find the JKube plugins to be a better choice for the future and would like to use it.

Additional context

There is already some discussion in this issue #1876 .

It was suggested that the change could be:

passing some additional configuration flags to Jib programmatically that we're doing in JibBuildService
using the setPlatform method being available in Jib.from method call here: https://github.com/eclipse/jkube/blob/e573db610e21284a52782e84faeaaf68fe1c4eb7/jkube-kit/build/service/jib/src/main/java/org/eclipse/jkube/kit/service/jib/JibServiceUtil.java#L120

I was unsure of the right component to declare, so I used what is my first requirement, but it should probably be changes in JKube Kit and Kubernetes Maven Plugin and Openshift Maven Plugin.

pavel1tel commented 1 year ago

Hello @manusa @rohanKanojia , I'm ready to make this improvement, but I need your help. I see here 2 options for how we can do it:

Add platform support to the <from> field and do it like this <from>--platform=windows/amd64 --platform=linux/arm64 openjdk:latest</from>(just parse the string)
Or modify <from> and make it an object like here

Another problem is that docker does not support multiple platforms without a manifest, so for a normal docker build, we will most likely have to limit the number of platforms to one

rohanKanojia commented 1 year ago

Another thing I observed is that JIB multi-platform mode doesn't support generating local tar (what our k8s:build does in JIB build strategy). Not 100% sure but we might need to change behavior of this goal when multiple platforms are selected.

manusa commented 1 year ago

Hello @manusa @rohanKanojia , I'm ready to make this improvement, but I need your help. I see here 2 options for how we can do it:

Add platform support to the <from> field and do it like this <from>--platform=windows/amd64 --platform=linux/arm64 openjdk:latest</from>(just parse the string)

Or modify <from> and make it an object like here

Another problem is that docker does not support multiple platforms without a manifest, so for a normal docker build, we will most likely have to limit the number of platforms to one

In order to keep things simple and to keep backwards compatibility, how would adding a fromPlatforms field to BuildConfiguration loop?

This should allow to provide an XML configuration as:

<image>
  <name>final/image:version</name>
  <build>
    <from>base/image:version</from>
    <fromPlatforms>
      <platform>
        <architecture>amd64</architecture>
        <os>linux</os>
      </platform>
      <platform>
        <architecture>arm64</architecture>
        <os>linux</os>
      </platform>
    </fromPlatforms>
  </build>
<image>

Another thing I observed is that JIB multi-platform mode doesn't support generating local tar

We need to find out how to overcome this. Maybe JIB supports generating multiple tarballs, but requires additional config. This would also have repercussions for the push phase.

Another problem is that docker does not support multiple platforms without a manifest, so for a normal docker build, we will most likely have to limit the number of platforms to one

If the previous issue with JIB can't be overcome, then we might need to limit platforms to one regardless of the build mechanism and suggest users to use Maven profiles or properties.

pavel1tel commented 1 year ago

@manusa JIB supports multiple architectures only for registry builds, tar and docker builds do not support multi-arch builds. We can add registry build support, but k8s:build will push images to the registry after the build. At the same time, we will also need to update k8s:push to also support the registry build.

Docker also does not support multi-arch builds by itself, to support it we will also need to generate a manifest file and push images and the manifest. There is also a buildx command in the docker, but it is not present in API, so we are unlikely to be able to use it.

manusa commented 11 months ago

I'll try to create a PoC to support this. The goal is to have a platforms field that can be used to set the target platforms. For JIB it should be straightforward. For Docker we might want to experiment with building each image separately and use docker manifest to merge them. Otherwise, docker buildx should be the way to go, however this won't work from the HTTP API.

eclipse-jkube / jkube