search

spring-attic / spring-native

Spring Native is now superseded by Spring Boot 3 official native support
https://docs.spring.io/spring-boot/docs/current/reference/html/native-image.html
Apache License 2.0
2.74k stars 356 forks source link

Buildpacks Error collecting native-compiler info #1642

Closed Tonne-TM closed 2 years ago

Tonne-TM commented 2 years ago

I am always getting the same error when trying to build native image via build packs. (Building directly on the machine works fine)

    [creator]     [1/7] Initializing...                                            (0.0s @ 0.30GB)
    [creator]     Error: Collecting native-compiler info with '/usr/bin/gcc -v' failed
    [creator]     Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
    [creator]     Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception

My Setup:

Full logs of gradle :bootBuildImage:

10:30:30: Executing 'bootBuildImage'...

> Task :compileJava
> Task :processResources UP-TO-DATE
> Task :classes

> Task :generateAot
Execution optimizations have been disabled for task ':generateAot' to ensure correctness due to the following reasons:
  - Additional action of task ':generateAot' was implemented by the Java lambda 'org.springframework.aot.gradle.SpringAotGradlePlugin$$Lambda$703/0x00000008006cb108'. Reason: Using Java lambdas is not supported as task inputs. Please refer to https://docs.gradle.org/7.4.1/userguide/validation_problems.html#implementation_unknown for more details about this problem.
2022-06-15 10:30:31.700  INFO 6626 --- [           main] o.s.a.build.ContextBootstrapContributor  : Detected application class: com.example.springbootnative.SpringBootNativeApplication
2022-06-15 10:30:31.702  INFO 6626 --- [           main] o.s.a.build.ContextBootstrapContributor  : Processing application context
2022-06-15 10:30:31.999  INFO 6626 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data R2DBC repositories in DEFAULT mode.
2022-06-15 10:30:32.003  INFO 6626 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 2 ms. Found 0 R2DBC repository interfaces.
2022-06-15 10:30:32.856  INFO 6626 --- [           main] o.s.a.build.ContextBootstrapContributor  : Processed 142 bean definitions in 1152ms
2022-06-15 10:30:33.026  INFO 6626 --- [           main] o.s.nativex.support.SpringAnalyzer       : Spring Native operating mode: native

> Task :compileAotMainJava

> Task :processAotMainResources
> Task :aotMainClasses
> Task :aotMainJar
warning: unknown enum constant When.MAYBE
  reason: class file for javax.annotation.meta.When not found
Note: /Users/myuser/Downloads/spring-boot-native/build/generated/runtimeSources/aotMain/org/springframework/boot/jdbc/_FactoryProvider.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /Users/myuser/Downloads/spring-boot-native/build/generated/runtimeSources/aotMain/org/springframework/aot/StaticSpringFactories.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning
> Task :bootJarMainClassName UP-TO-DATE
> Task :bootJar

> Task :bootBuildImage
Building image 'docker.io/library/spring-boot-native:0.0.1-SNAPSHOT'

 > Pulling builder image 'docker.io/paketobuildpacks/builder:tiny' ..................................................
 > Pulled builder image 'paketobuildpacks/builder@sha256:627b6ca4d75e5ed15686306483c5f71b84cd7a43f5cf4b4d3320a1f4f4aafa65'
 > Pulling run image 'docker.io/paketobuildpacks/run:tiny-cnb' ..................................................
 > Pulled run image 'paketobuildpacks/run@sha256:49f11f7cf19efc85985e576acb887c48eb59a0b157af180ae0d8903992ffd3ea'
 > Executing lifecycle version v0.14.1
 > Using build cache volume 'pack-cache-d1dc12756c57.build'

 > Running creator
    [creator]     Previous image with name "docker.io/library/spring-boot-native:0.0.1-SNAPSHOT" not found
    [creator]     ===> DETECTING
    [creator]     6 of 14 buildpacks participating
    [creator]     paketo-buildpacks/ca-certificates   3.2.3
    [creator]     paketo-buildpacks/bellsoft-liberica 9.3.6
    [creator]     paketo-buildpacks/syft              1.11.3
    [creator]     paketo-buildpacks/executable-jar    6.2.4
    [creator]     paketo-buildpacks/spring-boot       5.11.0
    [creator]     paketo-buildpacks/native-image      5.2.3
    [creator]     ===> RESTORING
    [creator]     ===> BUILDING
    [creator]     
    [creator]     Paketo CA Certificates Buildpack 3.2.3
    [creator]       https://github.com/paketo-buildpacks/ca-certificates
    [creator]       Launch Helper: Contributing to layer
    [creator]         Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper
    [creator]     
    [creator]     Paketo BellSoft Liberica Buildpack 9.3.6
    [creator]       https://github.com/paketo-buildpacks/bellsoft-liberica
    [creator]       Build Configuration:
    [creator]         $BP_JVM_TYPE                 JRE             the JVM type - JDK or JRE
    [creator]         $BP_JVM_VERSION              17.*            the Java version
    [creator]       Launch Configuration:
    [creator]         $BPL_DEBUG_ENABLED           false           enables Java remote debugging support
    [creator]         $BPL_DEBUG_PORT              8000            configure the remote debugging port
    [creator]         $BPL_DEBUG_SUSPEND           false           configure whether to suspend execution until a debugger has attached
    [creator]         $BPL_HEAP_DUMP_PATH                          write heap dumps on error to this path
    [creator]         $BPL_JAVA_NMT_ENABLED        true            enables Java Native Memory Tracking (NMT)
    [creator]         $BPL_JAVA_NMT_LEVEL          summary         configure level of NMT, summary or detail
    [creator]         $BPL_JFR_ARGS                                configure custom Java Flight Recording (JFR) arguments
    [creator]         $BPL_JFR_ENABLED             false           enables Java Flight Recording (JFR)
    [creator]         $BPL_JMX_ENABLED             false           enables Java Management Extensions (JMX)
    [creator]         $BPL_JMX_PORT                5000            configure the JMX port
    [creator]         $BPL_JVM_HEAD_ROOM           0               the headroom in memory calculation
    [creator]         $BPL_JVM_LOADED_CLASS_COUNT  35% of classes  the number of loaded classes in memory calculation
    [creator]         $BPL_JVM_THREAD_COUNT        250             the number of threads in memory calculation
    [creator]         $JAVA_TOOL_OPTIONS                           the JVM launch flags
    [creator]         Using Java version 17.* from BP_JVM_VERSION
    [creator]       BellSoft Liberica NIK 17.0.3: Contributing to layer
    [creator]         Downloading from https://download.bell-sw.com/vm/22.1.0/bellsoft-liberica-vm-core-openjdk17.0.3+7-22.1.0+1-linux-amd64.tar.gz
    [creator]         Verifying checksum
    [creator]         Expanding to /layers/paketo-buildpacks_bellsoft-liberica/native-image-svm
    [creator]         Adding 127 container CA certificates to JVM truststore
    [creator]         Writing env.build/JAVA_HOME.override
    [creator]         Writing env.build/JDK_HOME.override
    [creator]     
    [creator]     Paketo Syft Buildpack 1.11.3
    [creator]       https://github.com/paketo-buildpacks/syft
    [creator]         Downloading from https://github.com/anchore/syft/releases/download/v0.46.3/syft_0.46.3_linux_amd64.tar.gz
    [creator]         Verifying checksum
    [creator]         Writing env.build/SYFT_CHECK_FOR_APP_UPDATE.default
    [creator]     
    [creator]     Paketo Executable JAR Buildpack 6.2.4
    [creator]       https://github.com/paketo-buildpacks/executable-jar
    [creator]       Class Path: Contributing to layer
    [creator]         Writing env.build/CLASSPATH.delim
    [creator]         Writing env.build/CLASSPATH.prepend
    [creator]     
    [creator]     Paketo Spring Boot Buildpack 5.11.0
    [creator]       https://github.com/paketo-buildpacks/spring-boot
    [creator]       Build Configuration:
    [creator]         $BP_SPRING_CLOUD_BINDINGS_DISABLED   false  whether to contribute Spring Boot cloud bindings support
    [creator]       Launch Configuration:
    [creator]         $BPL_SPRING_CLOUD_BINDINGS_DISABLED  false  whether to auto-configure Spring Boot environment properties from bindings
    [creator]         $BPL_SPRING_CLOUD_BINDINGS_ENABLED   true   Deprecated - whether to auto-configure Spring Boot environment properties from bindings
    [creator]       Class Path: Contributing to layer
    [creator]         Writing env.build/CLASSPATH.append
    [creator]         Writing env.build/CLASSPATH.delim
    [creator]       Image labels:
    [creator]         org.springframework.boot.version
    [creator]     Warning: BOM table is deprecated in this buildpack api version, though it remains supported for backwards compatibility. Buildpack authors should write BOM information to <layer>.sbom.<ext>, launch.sbom.<ext>, or build.sbom.<ext>.
    [creator]     
    [creator]     Paketo Native Image Buildpack 5.2.3
    [creator]       https://github.com/paketo-buildpacks/native-image
    [creator]       Build Configuration:
    [creator]         $BP_BINARY_COMPRESSION_METHOD                Compression mechanism used to reduce binary size. Options: `none` (default), `upx` or `gzexe`
    [creator]         $BP_NATIVE_IMAGE                       true  enable native image build
    [creator]         $BP_NATIVE_IMAGE_BUILD_ARGUMENTS             arguments to pass to the native-image command
    [creator]         $BP_NATIVE_IMAGE_BUILD_ARGUMENTS_FILE        a file with arguments to pass to the native-image command
    [creator]         $BP_NATIVE_IMAGE_BUILT_ARTIFACT              the built application artifact explicitly, required if building from a JAR
    [creator]       Native Image: Contributing to layer
    [creator]         GraalVM 22.1.0 Java 17 CE (Java Version 17.0.3+7-LTS)
    [creator]         Executing native-image -H:+StaticExecutableWithDynamicLibC -H:Name=/layers/paketo-buildpacks_native-image/native-image/com.example.springbootnative.SpringBootNativeApplication -cp /workspace:/workspace/BOOT-INF/classes:/workspace/BOOT-INF/lib/spring-boot-native-0.0.1-SNAPSHOT-aot.jar:/workspace/BOOT-INF/lib/spring-native-0.12.0.jar:/workspace/BOOT-INF/lib/r2dbc-h2-0.9.1.RELEASE.jar:/workspace/BOOT-INF/lib/h2-2.1.212.jar:/workspace/BOOT-INF/lib/spring-data-r2dbc-1.5.0.jar:/workspace/BOOT-INF/lib/r2dbc-pool-0.9.0.RELEASE.jar:/workspace/BOOT-INF/lib/spring-r2dbc-5.3.20.jar:/workspace/BOOT-INF/lib/r2dbc-spi-0.9.1.RELEASE.jar:/workspace/BOOT-INF/lib/spring-webflux-5.3.20.jar:/workspace/BOOT-INF/lib/spring-web-5.3.20.jar:/workspace/BOOT-INF/lib/reactor-pool-0.2.8.jar:/workspace/BOOT-INF/lib/reactor-netty-http-1.0.19.jar:/workspace/BOOT-INF/lib/reactor-netty-core-1.0.19.jar:/workspace/BOOT-INF/lib/reactor-core-3.4.18.jar:/workspace/BOOT-INF/lib/spring-boot-autoconfigure-2.7.0.jar:/workspace/BOOT-INF/lib/spring-boot-2.7.0.jar:/workspace/BOOT-INF/lib/jakarta.annotation-api-1.3.5.jar:/workspace/BOOT-INF/lib/spring-data-relational-2.4.0.jar:/workspace/BOOT-INF/lib/spring-data-commons-2.7.0.jar:/workspace/BOOT-INF/lib/spring-tx-5.3.20.jar:/workspace/BOOT-INF/lib/spring-context-5.3.20.jar:/workspace/BOOT-INF/lib/spring-aop-5.3.20.jar:/workspace/BOOT-INF/lib/spring-beans-5.3.20.jar:/workspace/BOOT-INF/lib/spring-expression-5.3.20.jar:/workspace/BOOT-INF/lib/spring-core-5.3.20.jar:/workspace/BOOT-INF/lib/snakeyaml-1.30.jar:/workspace/BOOT-INF/lib/logback-classic-1.2.11.jar:/workspace/BOOT-INF/lib/log4j-to-slf4j-2.17.2.jar:/workspace/BOOT-INF/lib/jul-to-slf4j-1.7.36.jar:/workspace/BOOT-INF/lib/slf4j-api-1.7.36.jar:/workspace/BOOT-INF/lib/reactive-streams-1.0.3.jar:/workspace/BOOT-INF/lib/jackson-datatype-jsr310-2.13.3.jar:/workspace/BOOT-INF/lib/jackson-module-parameter-names-2.13.3.jar:/workspace/BOOT-INF/lib/jackson-annotations-2.13.3.jar:/workspace/BOOT-INF/lib/jackson-core-2.13.3.jar:/workspace/BOOT-INF/lib/jackson-datatype-jdk8-2.13.3.jar:/workspace/BOOT-INF/lib/jackson-databind-2.13.3.jar:/workspace/BOOT-INF/lib/spring-jcl-5.3.20.jar:/workspace/BOOT-INF/lib/netty-codec-http2-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-handler-proxy-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-codec-http-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-resolver-dns-native-macos-4.1.77.Final-osx-x86_64.jar:/workspace/BOOT-INF/lib/netty-resolver-dns-classes-macos-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-resolver-dns-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-transport-native-epoll-4.1.77.Final-linux-x86_64.jar:/workspace/BOOT-INF/lib/logback-core-1.2.11.jar:/workspace/BOOT-INF/lib/log4j-api-2.17.2.jar:/workspace/BOOT-INF/lib/netty-handler-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-codec-dns-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-codec-socks-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-codec-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-transport-classes-epoll-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-transport-native-unix-common-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-transport-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-buffer-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-resolver-4.1.77.Final.jar:/workspace/BOOT-INF/lib/netty-common-4.1.77.Final.jar com.example.springbootnative.SpringBootNativeApplication
    [creator]     Warning: Using a deprecated option --allow-incomplete-classpath from 'META-INF/native-image/org.springframework.aot/spring-aot/native-image.properties' in 'file:///workspace/BOOT-INF/lib/spring-boot-native-0.0.1-SNAPSHOT-aot.jar'. Allowing an incomplete classpath is now the default. Use --link-at-build-time to report linking errors at image build time for a class or package.
    [creator]     ================================================================================
    [creator]     GraalVM Native Image: Generating '/layers/paketo-buildpacks_native-image/native-image/com.example.springbootnative.SpringBootNativeApplication' (static executable)...
    [creator]     ================================================================================
    [creator]     This command is not for general use and should only be run as the result of a call to
    [creator]     ProcessBuilder.start() or Runtime.exec() in a java application
    [creator]     
    [creator]     [1/7] Initializing...                                            (0.0s @ 0.30GB)
    [creator]     Error: Collecting native-compiler info with '/usr/bin/gcc -v' failed
    [creator]     Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
    [creator]     Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
    [creator]     --------------------------------------------------------------------------------
    [creator]          0.9s (3.5% of total time) in 9 GCs | Peak RSS: 0.84GB | CPU load: 3.27
    [creator]     ================================================================================
    [creator]     Failed generating '/layers/paketo-buildpacks_native-image/native-image/com.example.springbootnative.SpringBootNativeApplication' after 21.5s.
    [creator]     Error: Image build request failed with exit status 1
    [creator]     unable to invoke layer creator
    [creator]     unable to contribute native-image layer
    [creator]     error running build
    [creator]     exit status 1
    [creator]     ERROR: failed to build: exit status 1

> Task :bootBuildImage FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4.1/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
9 actionable tasks: 7 executed, 2 up-to-date

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':bootBuildImage'.
> Builder lifecycle 'creator' failed with status code 51

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 26s
10:31:57: Execution finished 'bootBuildImage'.
sdeleuze commented 2 years ago

@dmikusa-pivotal Could you please share an update about the latest status of Apple M1 support for building native images?

dmikusa commented 2 years ago

At present:

  1. You can attempt to build your app as an x86_64 container. This will be really slow. To confirm that you'll get an x86_64 container, run docker run ubuntu:latest uname -a and ensure the output indicates x86_64, not ARM64. If it says x86_64 then your build should work OK.

  2. You can use https://github.com/dmikusa-pivotal/paketo-arm64 to build your own images. This will produce native images which will be very fast.

  3. You can use https://hub.docker.com/r/dashaun/java-native-builder-arm64 as your builder. This is an image published using the tools from # 2 and published by @dashaun. It'll be very fast as well.

All of these options should work though.

The project will at some point publish official builders but there's still a lot of work that needs to happen before we can do that. The options above are a stopgap until we can get all that work done.

I've not seen this error before:

    [creator]     Error: Collecting native-compiler info with '/usr/bin/gcc -v' failed
    [creator]     Error: To prevent native-toolchain checking provide command-line option -H:-CheckToolchain
    [creator]     Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
    [creator]     --------------------------------------------------------------------------------
    [creator]          0.9s (3.5% of total time) in 9 GCs | Peak RSS: 0.84GB | CPU load: 3.27
    [creator]     ================================================================================
    [creator]     Failed generating '/layers/paketo-buildpacks_native-image/native-image/com.example.springbootnative.SpringBootNativeApplication' after 21.5s.
    [creator]     Error: Image build request failed with exit status 1

My initial thought would be to try docker.io/paketobuildpacks/builder:base instead of docker.io/paketobuildpacks/builder:tiny and see if that helps. It's unlikely but I suppose it's possible something changed with tiny or a requirement was added that we don't include in tiny. If using base builder works, I can look into that more.

Also, try...

  1. Using @dashaun image as your builder and see if that works. It should be a lot faster anyway.
  2. Building with pack and see if that makes a difference.
  3. Make sure you do not have pull-policy set to never or if-not-present. Set it to always and rebuild so you get the latest builder/buildpack images.
sdeleuze commented 2 years ago

@dmikusa-pivotal Thanks a lot for the detailed feedback.

@Tonne-TM Are you trying to build a x86_64 image or a ARM64 one?

Tonne-TM commented 2 years ago

@sdeleuze Actually I am actually not sure. I just wanted to try out spring native on my Mac by using gradle :bootBuildImage. Based on the documentation, I would suspect a Linux_64 docker image with native executable. Is this supposed to work on Mac, if not, maybe some kind of check should be added to the Gradle task.

sdeleuze commented 2 years ago

@Tonne-TM See https://github.com/spring-projects-experimental/spring-native/issues/1642#issuecomment-1156442819 where Daniel explained how to check if you are using x86_64 image or a ARM64 image.

Tonne-TM commented 2 years ago 
 docker run ubuntu:latest uname -a                                                                                 125 ↵
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
ed02c6ade914: Pull complete
Digest: sha256:b6b83d3c331794420340093eb706a6f152d9c1fa51b262d9bf34594887c2c7ac
Status: Downloaded newer image for ubuntu:latest
Linux 21f9a0b1d260 5.10.76-linuxkit #1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Seems like it is running ARM64

sdeleuze commented 2 years ago

So not supported in Buildpacks yet, see @dmikusa-pivotal comment if you want to play with the early support, otherwise subscribe to https://github.com/paketo-buildpacks/stacks/issues/51 and please wait the official support.

dmikusa commented 2 years ago

Example if you are comfortable using the community-provided builder with ARM64:

<project>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <image>
                        <builder>dashaun/java-native-builder-arm64</builder>
                    </image>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

https://docs.spring.io/spring-boot/docs/2.7.0/maven-plugin/reference/htmlsingle/#build-image.examples.custom-image-builder

klopfdreh commented 1 year ago

Sorry for the noise at this closed ticket, but I just wanted to confirm that the builder of @dashaun is working for me even with Spring Boot 3 and Java 17.

dashaun commented 1 year ago

Thanks for that note @klopfdreh

For anyone else finding their way here: Everything you love about Spring Native is now baked into Spring Boot 3 which is GA. The latest version of my "community-provided builder with ARM64" can be found in my fork of dmikusa/paketo-arm64 and it delivers Spring Boot 3 native images for ARM64 & AMD64