GoogleContainerTools / jib

🏗 Build container images for your Java applications.
Apache License 2.0
13.5k stars 1.42k forks source link

Impossible to build amd-64 docker on Mac M3 machine #4265

Closed borisgob closed 1 week ago

borisgob commented 1 month ago

Starting 3.4.3 version of jib-maven-plugin it is not possible to build amd-64 image on Mac with Mx chip (arm-64) I didn't find a setting, that fix it. There is no problem with 3.4.2 version.

mpeddada1 commented 1 month ago

@borisgob Thanks for filing this issue! Would you be able to share more information on your jib configuration and the error you are currently running into?

pierCo commented 1 month ago

Same issue on Mac M1 after updating to 3.4.3 version 😄

Info

About the base image used : only available for amd64

About plugin configuration: nothing special

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <configuration>
        <from>
            <image>
                ghcr.io/image:XX@shaXX
            </image>
            <auth>
                ...
            </auth>
        </from>
        <to>
            <image>my/image:${docker.image.version}</image>
        </to>
        <container>
            <creationTime>USE_CURRENT_TIMESTAMP</creationTime>
            <mainClass>com.application.Application</mainClass>
            <ports>
                <port>8080</port>
            </ports>
            <user>64325:64325</user>
            <labels>
               ...
            </labels>
        </container>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>dockerBuild</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Error message from maven:

[INFO] --- jib:3.4.3:dockerBuild (default) @ application ---
[INFO] 
[INFO] Containerizing application to Docker daemon as my-app...
[INFO] 
[INFO] Container entrypoint set to [java, -cp, @/app/jib-classpath-file, com.application.Application]
[INFO] Executing tasks:
[INFO] [========================      ] 80.0% complete
[INFO] > building image to Docker daemon

...

[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.3:dockerBuild (default) on project application: The configured platforms don't match the Docker Engine's OS and architecture (linux/arm64) -> [Help 1]

In debug mode:


[DEBUG] TIMING  Building image to Docker daemon
[DEBUG] Containerizing application with the following files:
[DEBUG]         Dependencies:

...

[DEBUG] TIMING  Pulling base image manifest
[DEBUG] TIMING  Preparing application layer builders
[DEBUG] TIMED   Preparing application layer builders : 0.388 ms
[DEBUG] TIMING  Building dependencies layer
[DEBUG] TIMING  Building project dependencies layer
[DEBUG] TIMING  Building resources layer
[DEBUG] TIMING  Building classes layer
[DEBUG] TIMING  Building jvm arg files layer
[DEBUG] TIMED   Pulling base image manifest : 62.52 ms
[DEBUG] TIMING  Pulling base image layer sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8
[DEBUG] TIMING  Pulling base image layer sha256:a21a63612cbe8d148f75173be90696fbe03e2a6e9c901e2c039bcf1bcdeec0b9
[DEBUG] TIMING  Pulling base image layer sha256:92d6f603e71eafc432daa1850d612282a283263fbb867595b6f677946432b353
[DEBUG] TIMING  Pulling base image layer sha256:b8be18af9f33213bbcb3d22cccdf9fd2321c4373d32042c9c425e84dc590fb18
[DEBUG] TIMING  Pulling base image layer sha256:307132abc4f541c0c7c295dbabe80b2fdad4f7a9c6949563fe97be8f77000f38
[DEBUG] TIMING  Pulling base image layer sha256:704a4a6d46b80fc86ad623edf11cb31db58102ba82bf7688b9c8ee651142953f
[DEBUG] TIMING  Pulling base image layer sha256:ce0cd2cbd63b93556aa46f4d50c1e35121b3ebba07a6db67343d1d545a168544
[DEBUG] TIMED   Pulling base image layer sha256:ce0cd2cbd63b93556aa46f4d50c1e35121b3ebba07a6db67343d1d545a168544 : 1.525 ms
[DEBUG] TIMED   Pulling base image layer sha256:b8be18af9f33213bbcb3d22cccdf9fd2321c4373d32042c9c425e84dc590fb18 : 3.186 ms
[DEBUG] TIMED   Pulling base image layer sha256:92d6f603e71eafc432daa1850d612282a283263fbb867595b6f677946432b353 : 4.639 ms
[DEBUG] TIMED   Pulling base image layer sha256:a21a63612cbe8d148f75173be90696fbe03e2a6e9c901e2c039bcf1bcdeec0b9 : 4.67 ms
[DEBUG] TIMED   Pulling base image layer sha256:704a4a6d46b80fc86ad623edf11cb31db58102ba82bf7688b9c8ee651142953f : 3.815 ms
[DEBUG] TIMED   Pulling base image layer sha256:307132abc4f541c0c7c295dbabe80b2fdad4f7a9c6949563fe97be8f77000f38 : 4.28 ms
[DEBUG] TIMED   Pulling base image layer sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8 : 11.745 ms
[DEBUG] Building jvm arg files layer built sha256:da783be6a548852a92c671870d2f0cd5e5e72392f705820b613580d7df6e3b83
[DEBUG] TIMED   Building jvm arg files layer : 101.762 ms
[DEBUG] Building classes layer built sha256:eb5150fbd5f53f3add587da5a5b5c4cf024b94e83aea2b7513e8f3a2922271d7
[DEBUG] TIMED   Building classes layer : 131.291 ms
[DEBUG] Building resources layer built sha256:ef5ada1b8da915665facba8f93fa75ba8e2f85495661a7648a5e15c0b552ccd2
[DEBUG] TIMED   Building resources layer : 159.77 ms
[DEBUG] Building project dependencies layer built sha256:33f341327dd567d515afcdab661a3cb9312127429c21881fbee8809b4271f788
[DEBUG] TIMED   Building project dependencies layer : 229.413 ms
[DEBUG] Building dependencies layer built sha256:891ab74e1ac83d8b055f87125a60f200584cd3f21ef43d15b175f5144a364236
[DEBUG] TIMED   Building dependencies layer : 3594.467 ms
[DEBUG] TIMING  Building container configuration
[INFO] 
[INFO] Container entrypoint set to [java, -cp, @/app/jib-classpath-file, com.application.Application]
[DEBUG] TIMED   Building container configuration : 1.02 ms
[DEBUG] TIMED   Building image to Docker daemon : 3611.31 ms
[INFO] Executing tasks:
[INFO] [========================      ] 80.0% complete
[INFO] > building image to Docker daemon

...

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.4.3:dockerBuild (default) on project application: The configured platforms don't match the Docker Engine's OS and architecture (linux/arm64)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.maven.plugin.MojoExecutionException: The configured platforms don't match the Docker Engine's OS and architecture (linux/arm64)
    at com.google.cloud.tools.jib.maven.BuildDockerMojo.execute (BuildDockerMojo.java:167)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: java.lang.IllegalStateException: The configured platforms don't match the Docker Engine's OS and architecture (linux/arm64)
    at com.google.common.base.Preconditions.checkState (Preconditions.java:512)
    at com.google.cloud.tools.jib.builder.steps.StepsRunner.lambda$loadDocker$18 (StepsRunner.java:628)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly (TrustedListenableFutureTask.java:131)
    at com.google.common.util.concurrent.InterruptibleTask.run (InterruptibleTask.java:75)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run (TrustedListenableFutureTask.java:82)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1583)
borisgob commented 1 month ago

Plugin definition

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <version>3.4.3</version>
    <configuration>
        <from>
            <image>[our privately built jib image for amd-64]</image>
            <auth>
                <username>${env.MAVEN_DEV_USERNAME}</username>
                <password>${env.MAVEN_DEV_PASSWORD}</password>
            </auth>
        </from>
        <to>
            <image>[our service]:${docker.image.version}</image>
        </to>
        <container>
            <creationTime>USE_CURRENT_TIMESTAMP</creationTime>
            <ports>
                <port>8080</port>
            </ports>
            <entrypoint>
                [Our service entry point]
            </entrypoint>
            <user>64325:64325</user>
        </container>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>dockerBuild</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The error I get: The configured platforms don't match the Docker Engine's OS and architecture (linux/arm64)

germondc commented 1 month ago

Getting exactly the same problem running the gradle plugin on M3 for 3.4.3. After reverting to 3.4.2 things work again. Physically setting the platform (in from.platforms) does not help.

hhedin commented 1 month ago

Same Issue here with 3.4.3, but on Ubuntu x86 with podman instead of docker: mvn jib:dockerBuild -Djib.dockerClient.executable=$(which podman). Works without issues in 3.4.2.

mpeddada1 commented 1 month ago

Thank you for the context, all, We're looking into reproducing this issue locally.

@hhedin Could I get a bit more information on the ostype and architecture of your local podman/docker environment? Through docker info -f '{{.OSType}} {{.Architecture}}' (but the podman equivalent). The newly added check in 3.4.3 compares the platform of the image built to the os and architecture of the docker environment and throws an exception if those values don't match.

Additionally, @borisgob and @pierCo could we also get more information on the docker version you are currently using?

borisgob commented 1 month ago

The result is: linux aarch64 Docker version 26.1.1, build 4cf5afa This check is wrong and useless. The docker itself and the previous (3.4.2) version allow to build docker with different platform from the build machine. I don't see a reason to prevent it. You may show a warning, that this docker will not run on this machne. We build docker on dev machine and push it to the testing environment based on linux with Intel processor.

hhedin commented 1 month ago

@mpeddada1 Closest I can get is podman info -f '{{.Host.OS}} {{.Host.Arch}}' resulting in: linux amd64.

Like @borisgob I don´t really understand the purpose of this check. If there is a case to cover here that we don't understand, perhaps it could be opt-in or at least opt-out? This feels like a breaking change/added feature and would perhaps be better suited as a 3.5.x release and not 3.4.x in my opinion.

pierCo commented 1 month ago

Requested info:

$ docker info -f '{{.OSType}} {{.Architecture}}'
linux aarch64
$ docker version 
Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:04 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
Ashishkumar459 commented 4 weeks ago

same issue with me I am using mac M3. ![Uploading Screenshot 2024-06-10 at 11.04.00 PM.png…]()

mihalyr-prospect commented 2 weeks ago

Not just Mac, I cannot build my Amazon Linux based docker image on my Linux laptop anymore, same error. And works with 3.4.2 fails with 3.4.3

Execution failed for task ':jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: The configured platforms don't match the Docker Engine's OS and architecture (/)
jib {
    from {
        image = 'amazoncorretto:21-al2-jdk'
    }
✗ uname -a
Linux eli 6.8.9-300.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May  2 18:59:06 UTC 2024 x86_64 GNU/Linux
✗ docker -v
podman version 5.0.2
borisgob commented 2 weeks ago

@mpeddada1 , when do you expect the fix, when priority is P1?

zasran commented 1 week ago

@mpeddada1 why is this issue closed? The problem is still there (just tried to build on Mac using jib-maven-plugin 3.4.3).

mpeddada1 commented 1 week ago

@zasran the fix for this issue has been merged (https://github.com/GoogleContainerTools/jib/pull/4268). I think github's feature of adding "Fixes ..." in the description automatically closes the connected issue.

It will be introduced in the upcoming release but we're working on addressing one more issue before going ahead with it. Until then, please continue using 3.4.2. We appreciate your patience on this!