[GR-39609] Deadlock breaks the native image building process

CH3CHO commented 8 months ago

Describe the issue

Following error is shown when building a native image using Maven plugin.

Image generator watchdog is aborting image generation. To configure the watchdog, use the options -H:DeadlockWatchdogInterval=10 and -H:+DeadlockWatchdogExitOnTimeout

Steps to reproduce the issue

Sorry, the project uses some internal libraries, so I can't share the entire project. Here are the configuration of native-maven-plugin.

<plugin>
  <groupId>org.graalvm.buildtools</groupId>
  <artifactId>native-maven-plugin</artifactId>
  <version>0.9.28</version>
  <configuration>
    <classesDirectory>${project.build.outputDirectory}</classesDirectory>
    <metadataRepository>
      <enabled>true</enabled>
    </metadataRepository>
    <mainClass>com.xxx.testapp.AppSpringBootConfiguration</mainClass>
    <imageName>application</imageName>
    <buildArgs>
      -march=x86-64-v2
      --add-opens java.management/sun.management=ALL-UNNAMED
      --add-opens java.base/java.lang=ALL-UNNAMED
      --add-opens java.base/java.lang.reflect=ALL-UNNAMED
      --add-opens java.base/sun.reflect.annotation=ALL-UNNAMED
      --add-opens java.base/java.math=ALL-UNNAMED
      --add-opens java.base/java.util=ALL-UNNAMED
      --add-opens java.base/sun.util.calendar=ALL-UNNAMED
      --add-opens java.base/sun.util.calendar=ALL-UNNAMED
      --add-opens java.base/java.io=ALL-UNNAMED
      --add-opens java.base/java.net=ALL-UNNAMED
      --add-opens java.xml/com.sun.org.apache.xerces.internal.jaxp.datatype=ALL-UNNAMED
    </buildArgs>
  </configuration>
  <executions>
    <execution>
      <id>add-reachability-metadata</id>
      <goals>
        <goal>add-reachability-metadata</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Describe GraalVM and your environment:

❯ java -version
java version "21.0.2" 2024-01-16 LTS
Java(TM) SE Runtime Environment Oracle GraalVM 21.0.2+13.1 (build 21.0.2+13-LTS-jvmci-23.1-b30)
Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 21.0.2+13.1 (build 21.0.2+13-LTS-jvmci-23.1-b30, mixed mode, sharing)

More details

[INFO] --- native:0.9.28:compile-no-fork (default-cli) @ testapp ---
[INFO] Found GraalVM installation from JAVA_HOME variable.
[INFO] [graalvm reachability metadata repository for ch.qos.logback:logback-classic:1.4.14]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for ch.qos.logback:logback-classic:1.4.14]: Configuration directory is ch.qos.logback/logback-classic/1.4.1
[INFO] [graalvm reachability metadata repository for com.fasterxml.jackson.core:jackson-databind:2.15.3]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for com.fasterxml.jackson.core:jackson-databind:2.15.3]: Configuration directory is com.fasterxml.jackson.core/jackson-databind/2.15.2
[INFO] [graalvm reachability metadata repository for org.apache.tomcat.embed:tomcat-embed-core:10.1.17]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for org.apache.tomcat.embed:tomcat-embed-core:10.1.17]: Configuration directory is org.apache.tomcat.embed/tomcat-embed-core/10.0.20
[INFO] [graalvm reachability metadata repository for io.netty:netty-buffer:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-buffer:4.1.104.Final]: Configuration directory is io.netty/netty-buffer/4.1.80.Final
[INFO] [graalvm reachability metadata repository for io.netty:netty-codec-http:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-codec-http:4.1.104.Final]: Configuration directory is io.netty/netty-codec-http/4.1.80.Final
[INFO] [graalvm reachability metadata repository for io.netty:netty-codec-http2:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-codec-http2:4.1.104.Final]: Configuration directory is io.netty/netty-codec-http2/4.1.80.Final
[INFO] [graalvm reachability metadata repository for io.netty:netty-common:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-common:4.1.104.Final]: Configuration directory is io.netty/netty-common/4.1.80.Final
[INFO] [graalvm reachability metadata repository for io.netty:netty-handler:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-handler:4.1.104.Final]: Configuration directory is io.netty/netty-handler/4.1.80.Final
[INFO] [graalvm reachability metadata repository for io.netty:netty-resolver-dns:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-resolver-dns:4.1.104.Final]: Configuration directory is io.netty/netty-resolver-dns/4.1.80.Final
[INFO] [graalvm reachability metadata repository for io.netty:netty-transport:4.1.104.Final]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for io.netty:netty-transport:4.1.104.Final]: Configuration directory is io.netty/netty-transport/4.1.80.Final
[WARNING] Properties file at 'jar:file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar!/META-INF/native-image/io.grpc.netty.shaded.io.netty/common/native-image.properties' does not match the recommended 'META-INF/native-image/com.xxxx.framework/yyyy-client/native-image.properties' layout.
[WARNING] Properties file at 'jar:file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar!/META-INF/native-image/io.grpc.netty.shaded.io.netty/buffer/native-image.properties' does not match the recommended 'META-INF/native-image/com.xxxx.framework/yyyy-client/native-image.properties' layout.
[WARNING] Properties file at 'jar:file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar!/META-INF/native-image/io.grpc.netty.shaded.io.netty/transport/native-image.properties' does not match the recommended 'META-INF/native-image/com.xxxx.framework/yyyy-client/native-image.properties' layout.
[WARNING] Properties file at 'jar:file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar!/META-INF/native-image/io.grpc.netty.shaded.io.netty/handler/native-image.properties' does not match the recommended 'META-INF/native-image/com.xxxx.framework/yyyy-client/native-image.properties' layout.
[WARNING] Properties file at 'jar:file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar!/META-INF/native-image/io.grpc.netty.shaded.io.netty/codec-http/native-image.properties' does not match the recommended 'META-INF/native-image/com.xxxx.framework/yyyy-client/native-image.properties' layout.
[WARNING] Properties file at 'jar:file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar!/META-INF/native-image/io.grpc.netty.shaded.io.netty/codec-http2/native-image.properties' does not match the recommended 'META-INF/native-image/com.xxxx.framework/yyyy-client/native-image.properties' layout.
[INFO] [graalvm reachability metadata repository for org.jetbrains.kotlin:kotlin-stdlib:1.9.21]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for org.jetbrains.kotlin:kotlin-stdlib:1.9.21]: Configuration directory is org.jetbrains.kotlin/kotlin-stdlib/1.7.10
[INFO] Executing: /home/ch3cho/.sdkman/candidates/java/current/bin/native-image -cp /home/ch3cho/TestProjects/flightapp/target/classes:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot-starter-web/3.2.1/spring-boot-starter-web-3.2.1.jar:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot-starter/3.2.1/spring-boot-starter-3.2.1.jar:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot/3.2.1/spring-boot-3.2.1.jar:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/3.2.1/spring-boot-autoconfigure-3.2.1.jar:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot-starter-logging/3.2.1/spring-boot-starter-logging-3.2.1.jar:/home/ch3cho/.m2/repository/ch/qos/logback/logback-classic/1.4.14/logback-classic-1.4.14.jar:/home/ch3cho/.m2/repository/ch/qos/logback/logback-core/1.4.14/logback-core-1.4.14.jar:/home/ch3cho/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.21.1/log4j-to-slf4j-2.21.1.jar:/home/ch3cho/.m2/repository/org/apache/logging/log4j/log4j-api/2.21.1/log4j-api-2.21.1.jar:/home/ch3cho/.m2/repository/org/slf4j/jul-to-slf4j/2.0.9/jul-to-slf4j-2.0.9.jar:/home/ch3cho/.m2/repository/jakarta/annotation/jakarta.annotation-api/2.1.1/jakarta.annotation-api-2.1.1.jar:/home/ch3cho/.m2/repository/org/yaml/snakeyaml/2.2/snakeyaml-2.2.jar:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot-starter-json/3.2.1/spring-boot-starter-json-3.2.1.jar:/home/ch3cho/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.15.3/jackson-databind-2.15.3.jar:/home/ch3cho/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.15.3/jackson-annotations-2.15.3.jar:/home/ch3cho/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.15.3/jackson-core-2.15.3.jar:/home/ch3cho/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.15.3/jackson-datatype-jdk8-2.15.3.jar:/home/ch3cho/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.15.3/jackson-datatype-jsr310-2.15.3.jar:/home/ch3cho/.m2/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.15.3/jackson-module-parameter-names-2.15.3.jar:/home/ch3cho/.m2/repository/org/springframework/boot/spring-boot-starter-tomcat/3.2.1/spring-boot-starter-tomcat-3.2.1.jar:/home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/10.1.17/tomcat-embed-core-10.1.17.jar:/home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/10.1.17/tomcat-embed-el-10.1.17.jar:/home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/10.1.17/tomcat-embed-websocket-10.1.17.jar:/home/ch3cho/.m2/repository/org/springframework/spring-web/6.1.2/spring-web-6.1.2.jar:/home/ch3cho/.m2/repository/org/springframework/spring-beans/6.1.2/spring-beans-6.1.2.jar:/home/ch3cho/.m2/repository/io/micrometer/micrometer-observation/1.12.1/micrometer-observation-1.12.1.jar:/home/ch3cho/.m2/repository/io/micrometer/micrometer-commons/1.12.1/micrometer-commons-1.12.1.jar:/home/ch3cho/.m2/repository/org/springframework/spring-webmvc/6.1.2/spring-webmvc-6.1.2.jar:/home/ch3cho/.m2/repository/org/springframework/spring-aop/6.1.2/spring-aop-6.1.2.jar:/home/ch3cho/.m2/repository/org/springframework/spring-context/6.1.2/spring-context-6.1.2.jar:/home/ch3cho/.m2/repository/org/springframework/spring-expression/6.1.2/spring-expression-6.1.2.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/fx-spring-boot-starter-monitor/0.0.2/fx-spring-boot-starter-monitor-0.0.2.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/fx-spring-boot/0.0.2/fx-spring-boot-0.0.2.jar:/home/ch3cho/.m2/repository/com/dianping/cat/cat-client/3.5.19-Java21/cat-client-3.5.19-Java21.jar:/home/ch3cho/.m2/repository/com/dianping/cat/cat-exporter/3.5.19-Java21/cat-exporter-3.5.19-Java21.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/framework-foundation/1.8.25-Java21/framework-foundation-1.8.25-Java21.jar:/home/ch3cho/.m2/repository/com/google/guava/guava/31.0.1-jre/guava-31.0.1-jre.jar:/home/ch3cho/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar:/home/ch3cho/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/ch3cho/.m2/repository/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/home/ch3cho/.m2/repository/org/checkerframework/checker-qual/3.12.0/checker-qual-3.12.0.jar:/home/ch3cho/.m2/repository/com/google/errorprone/error_prone_annotations/2.21.1/error_prone_annotations-2.21.1.jar:/home/ch3cho/.m2/repository/com/google/j2objc/j2objc-annotations/1.3/j2objc-annotations-1.3.jar:/home/ch3cho/.m2/repository/io/netty/netty-all/4.1.104.Final/netty-all-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-buffer/4.1.104.Final/netty-buffer-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec/4.1.104.Final/netty-codec-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-dns/4.1.104.Final/netty-codec-dns-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-haproxy/4.1.104.Final/netty-codec-haproxy-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-http/4.1.104.Final/netty-codec-http-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-http2/4.1.104.Final/netty-codec-http2-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-memcache/4.1.104.Final/netty-codec-memcache-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-mqtt/4.1.104.Final/netty-codec-mqtt-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-redis/4.1.104.Final/netty-codec-redis-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-smtp/4.1.104.Final/netty-codec-smtp-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-socks/4.1.104.Final/netty-codec-socks-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-stomp/4.1.104.Final/netty-codec-stomp-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-codec-xml/4.1.104.Final/netty-codec-xml-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-common/4.1.104.Final/netty-common-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-handler/4.1.104.Final/netty-handler-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-native-unix-common/4.1.104.Final/netty-transport-native-unix-common-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-handler-proxy/4.1.104.Final/netty-handler-proxy-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-handler-ssl-ocsp/4.1.104.Final/netty-handler-ssl-ocsp-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-resolver/4.1.104.Final/netty-resolver-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-resolver-dns/4.1.104.Final/netty-resolver-dns-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport/4.1.104.Final/netty-transport-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-rxtx/4.1.104.Final/netty-transport-rxtx-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-sctp/4.1.104.Final/netty-transport-sctp-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-udt/4.1.104.Final/netty-transport-udt-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-classes-epoll/4.1.104.Final/netty-transport-classes-epoll-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-classes-kqueue/4.1.104.Final/netty-transport-classes-kqueue-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-resolver-dns-classes-macos/4.1.104.Final/netty-resolver-dns-classes-macos-4.1.104.Final.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-native-epoll/4.1.104.Final/netty-transport-native-epoll-4.1.104.Final-linux-x86_64.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-native-epoll/4.1.104.Final/netty-transport-native-epoll-4.1.104.Final-linux-aarch_64.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-native-epoll/4.1.104.Final/netty-transport-native-epoll-4.1.104.Final-linux-riscv64.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-native-kqueue/4.1.104.Final/netty-transport-native-kqueue-4.1.104.Final-osx-x86_64.jar:/home/ch3cho/.m2/repository/io/netty/netty-transport-native-kqueue/4.1.104.Final/netty-transport-native-kqueue-4.1.104.Final-osx-aarch_64.jar:/home/ch3cho/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.104.Final/netty-resolver-dns-native-macos-4.1.104.Final-osx-x86_64.jar:/home/ch3cho/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.104.Final/netty-resolver-dns-native-macos-4.1.104.Final-osx-aarch_64.jar:/home/ch3cho/.m2/repository/org/slf4j/jcl-over-slf4j/2.0.9/jcl-over-slf4j-2.0.9.jar:/home/ch3cho/.m2/repository/com/google/code/gson/gson/2.10.1/gson-2.10.1.jar:/home/ch3cho/.m2/repository/org/xerial/snappy/snappy-java/1.1.10.3/snappy-java-1.1.10.3.jar:/home/ch3cho/.m2/repository/com/dianping/cat/kafka/kafka-clients/1.1.1-xxxx.1/kafka-clients-1.1.1-xxxx.1.jar:/home/ch3cho/.m2/repository/org/aspectj/aspectjweaver/1.9.21/aspectjweaver-1.9.21.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar:/home/ch3cho/.m2/repository/org/slf4j/slf4j-api/2.0.9/slf4j-api-2.0.9.jar:/home/ch3cho/.m2/repository/com/xxxx/ops/hickwall-sdk/0.1.16/hickwall-sdk-0.1.16.jar:/home/ch3cho/.m2/repository/org/influxdb/influxdb-java/2.23/influxdb-java-2.23.jar:/home/ch3cho/.m2/repository/com/squareup/retrofit2/retrofit/2.9.0/retrofit-2.9.0.jar:/home/ch3cho/.m2/repository/com/squareup/retrofit2/converter-moshi/2.9.0/converter-moshi-2.9.0.jar:/home/ch3cho/.m2/repository/com/squareup/moshi/moshi/1.8.0/moshi-1.8.0.jar:/home/ch3cho/.m2/repository/org/msgpack/msgpack-core/0.9.3/msgpack-core-0.9.3.jar:/home/ch3cho/.m2/repository/com/squareup/okhttp3/logging-interceptor/4.12.0/logging-interceptor-4.12.0.jar:/home/ch3cho/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib-jdk8/1.9.21/kotlin-stdlib-jdk8-1.9.21.jar:/home/ch3cho/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib/1.9.21/kotlin-stdlib-1.9.21.jar:/home/ch3cho/.m2/repository/org/jetbrains/annotations/13.0/annotations-13.0.jar:/home/ch3cho/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib-jdk7/1.9.21/kotlin-stdlib-jdk7-1.9.21.jar:/home/ch3cho/.m2/repository/com/squareup/okio/okio/1.17.2/okio-1.17.2.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/vi/0.11.65-Java21/vi-0.11.65-Java21.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/vi-core/0.11.65-Java21/vi-core-0.11.65-Java21.jar:/home/ch3cho/.m2/repository/org/javassist/javassist/3.29.2-GA/javassist-3.29.2-GA.jar:/home/ch3cho/.m2/repository/qunar/tc/qtracer/qtracer-standalone-debugger/1.4.14/qtracer-standalone-debugger-1.4.14.jar:/home/ch3cho/.m2/repository/qunar/tc/qtracer/qtracer-instrument-asm/1.4.14/qtracer-instrument-asm-1.4.14.jar:/home/ch3cho/.m2/repository/com/xxxx/framework/fx-spring-boot-starter-native/0.0.2/fx-spring-boot-starter-native-0.0.2.jar:/home/ch3cho/.m2/repository/commons-io/commons-io/2.13.0/commons-io-2.13.0.jar:/home/ch3cho/.m2/repository/org/springframework/spring-core/6.1.2/spring-core-6.1.2.jar --no-fallback -o /home/ch3cho/TestProjects/flightapp/target/application -H:ConfigurationFileDirectories=/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/org.jetbrains.kotlin/kotlin-stdlib/1.7.10,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/com.fasterxml.jackson.core/jackson-databind/2.15.2,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-buffer/4.1.80.Final,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-transport/4.1.80.Final,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/org.apache.tomcat.embed/tomcat-embed-core/10.0.20,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-codec-http/4.1.80.Final,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-handler/4.1.80.Final,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-codec-http2/4.1.80.Final,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-common/4.1.80.Final,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/ch.qos.logback/logback-classic/1.4.1,/home/ch3cho/TestProjects/flightapp/target/graalvm-reachability-metadata/398703dc55b30fdcc52be5738b99125a2dfe365b/io.netty/netty-resolver-dns/4.1.80.Final -march=x86-64-v2 --add-opens java.management/sun.management=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/sun.reflect.annotation=ALL-UNNAMED --add-opens java.base/java.math=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/sun.util.calendar=ALL-UNNAMED --add-opens java.base/sun.util.calendar=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.xml/com.sun.org.apache.xerces.internal.jaxp.datatype=ALL-UNNAMED --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-buffer/4.1.104.Final/netty-buffer-4.1.104.Final.jar\E ^/META-INF/native-image/ --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-codec-http/4.1.104.Final/netty-codec-http-4.1.104.Final.jar\E ^/META-INF/native-image/ --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-codec-http2/4.1.104.Final/netty-codec-http2-4.1.104.Final.jar\E ^/META-INF/native-image/ --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-common/4.1.104.Final/netty-common-4.1.104.Final.jar\E ^/META-INF/native-image/ --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-handler/4.1.104.Final/netty-handler-4.1.104.Final.jar\E ^/META-INF/native-image/ --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-resolver-dns/4.1.104.Final/netty-resolver-dns-4.1.104.Final.jar\E ^/META-INF/native-image/ --exclude-config \Q/home/ch3cho/.m2/repository/io/netty/netty-transport/4.1.104.Final/netty-transport-4.1.104.Final.jar\E ^/META-INF/native-image/ com.xxxx.testapp.AppSpringBootConfiguration
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.grpc.netty.shaded.io.netty/transport/reflection-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-websocket/tomcat-reflection.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-el/tomcat-reflection.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ResourceConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/tomcat-resource.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/tomcat-reflection.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ResourceConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-el/tomcat-resource.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ResourceConfigurationResources=META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-websocket/tomcat-resource.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
========================================================================================================================
GraalVM Native Image: Generating 'application' (executable)...
========================================================================================================================
[1/8] Initializing...                                                                                   (13.9s @ 0.26GB)
 Java version: 21.0.2+13-LTS, vendor version: Oracle GraalVM 21.0.2+13.1
 Graal compiler: optimization level: 2, target machine: x86-64-v2, PGO: ML-inferred
 C compiler: gcc (linux, x86_64, 8.3.0)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 2 user-specific feature(s):
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - org.springframework.aot.nativex.feature.PreComputeFieldFeature
------------------------------------------------------------------------------------------------------------------------
 2 experimental option(s) unlocked:
 - '-H:ResourceConfigurationResources' (origin(s): 'META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/native-image.properties' in 'file:///home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/10.1.17/tomcat-embed-core-10.1.17.jar', 'META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-el/native-image.properties' in 'file:///home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/10.1.17/tomcat-embed-el-10.1.17.jar', 'META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-websocket/native-image.properties' in 'file:///home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/10.1.17/tomcat-embed-websocket-10.1.17.jar')
 - '-H:ReflectionConfigurationResources' (origin(s): 'META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/native-image.properties' in 'file:///home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/10.1.17/tomcat-embed-core-10.1.17.jar', 'META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-el/native-image.properties' in 'file:///home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-el/10.1.17/tomcat-embed-el-10.1.17.jar', 'META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-websocket/native-image.properties' in 'file:///home/ch3cho/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/10.1.17/tomcat-embed-websocket-10.1.17.jar', 'META-INF/native-image/io.grpc.netty.shaded.io.netty/transport/native-image.properties' in 'file:///home/ch3cho/.m2/repository/com/xxxx/framework/yyyy-client/0.0.63-Java21/yyyy-client-0.0.63-Java21.jar')
------------------------------------------------------------------------------------------------------------------------
Build resources:
 - 10.16GB of memory (28.9% of 35.14GB system memory, determined at start)
 - 16 thread(s) (100.0% of 16 available processor(s), determined at start)
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
[2/8] Performing analysis...  [******]                                                                  (74.5s @ 2.03GB)
   24,297 reachable types   (89.6% of   27,111 total)
   41,288 reachable fields  (61.8% of   66,774 total)
  135,173 reachable methods (63.3% of  213,636 total)
    7,810 types,   852 fields, and 8,519 methods registered for reflection
       87 types,    88 fields, and    64 methods registered for JNI access
        4 native libraries: dl, pthread, rt, z
[3/8] Building universe...                                                                              (14.0s @ 2.28GB)
[4/8] Parsing methods...      [*****]                                                                   (23.3s @ 1.99GB)
[5/8] Inlining methods...     [***]                                                                      (2.8s @ 2.99GB)

=== Image generator watchdog detected no activity. This can be a sign of a deadlock during image building. Dumping all stack traces. Current time: Tue Apr 02 16:11:04 GMT+08:00 2024
"main" Id=1 in TIMED_WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.externalHelpQuiesce(ForkJoinPool.java:2274)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.helpQuiescePool(ForkJoinPool.java:2297)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitQuiescence(ForkJoinPool.java:3565)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.complete(CompletionExecutor.java:237)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.runOnExecutor(CompileQueue.java:603)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.compileAll(CompileQueue.java:913)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.finish(CompileQueue.java:433)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:673)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)

"Reference Handler" Id=5 in RUNNABLE
    at java.base@21.0.2/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
    at java.base@21.0.2/java.lang.ref.Reference.processPendingReferences(Reference.java:246)
    at java.base@21.0.2/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:208)

"Finalizer" Id=6 in WAITING on lock=java.lang.ref.NativeReferenceQueue$Lock@127010e3
    at java.base@21.0.2/java.lang.Object.wait0(Native Method)
    at java.base@21.0.2/java.lang.Object.wait(Object.java:366)
    at java.base@21.0.2/java.lang.Object.wait(Object.java:339)
    at java.base@21.0.2/java.lang.ref.NativeReferenceQueue.await(NativeReferenceQueue.java:48)
    at java.base@21.0.2/java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:158)
    at java.base@21.0.2/java.lang.ref.NativeReferenceQueue.remove(NativeReferenceQueue.java:89)
    at java.base@21.0.2/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:173)

"Signal Dispatcher" Id=7 in RUNNABLE

"JVMCI-native CompilerThread0" Id=10 in RUNNABLE

"Common-Cleaner" Id=20 in TIMED_WAITING on lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1888d793
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
    at java.base@21.0.2/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1847)
    at java.base@21.0.2/java.lang.ref.ReferenceQueue.await(ReferenceQueue.java:71)
    at java.base@21.0.2/java.lang.ref.ReferenceQueue.remove0(ReferenceQueue.java:143)
    at java.base@21.0.2/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:218)
    at java.base@21.0.2/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:140)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)
    at java.base@21.0.2/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Notification Thread" Id=21 in RUNNABLE

"native-image pid watcher" Id=22 in TIMED_WAITING on lock=java.util.TaskQueue@23a88580
    at java.base@21.0.2/java.lang.Object.wait0(Native Method)
    at java.base@21.0.2/java.lang.Object.wait(Object.java:366)
    at java.base@21.0.2/java.util.TimerThread.mainLoop(Timer.java:563)
    at java.base@21.0.2/java.util.TimerThread.run(Timer.java:516)

"Thread-1" Id=25 in RUNNABLE
    at java.management@21.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
    at java.management@21.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:518)
    at java.management@21.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:506)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog.threadDump(DeadlockWatchdog.java:125)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog.watchdogThread(DeadlockWatchdog.java:82)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.DeadlockWatchdog$$Lambda/0x00000007c01d0788.run(Unknown Source)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)

"ForkJoinPool.commonPool-worker-1" Id=49 in WAITING on lock=java.util.concurrent.ForkJoinPool@3c7a21cb
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool.commonPool-worker-6" Id=54 in WAITING on lock=java.util.concurrent.ForkJoinPool@3c7a21cb
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool.commonPool-worker-14" Id=62 in TIMED_WAITING on lock=java.util.concurrent.ForkJoinPool@3c7a21cb
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.parkUntil(LockSupport.java:449)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1891)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"Read-Poller" Id=67 in RUNNABLE (running in native)
    at java.base@21.0.2/sun.nio.ch.EPoll.wait(Native Method)
    at java.base@21.0.2/sun.nio.ch.EPollPoller.poll(EPollPoller.java:71)
    at java.base@21.0.2/sun.nio.ch.Poller.poll(Poller.java:363)
    at java.base@21.0.2/sun.nio.ch.Poller.pollLoop(Poller.java:270)
    at java.base@21.0.2/sun.nio.ch.Poller$$Lambda/0x00000007c2051fc0.run(Unknown Source)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)
    at java.base@21.0.2/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Read-Updater" Id=68 in WAITING on lock=java.util.concurrent.LinkedTransferQueue@60ba1c50
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.LinkedTransferQueue$DualNode.await(LinkedTransferQueue.java:458)
    at java.base@21.0.2/java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:613)
    at java.base@21.0.2/java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1252)
    at java.base@21.0.2/sun.nio.ch.Poller.updateLoop(Poller.java:286)
    at java.base@21.0.2/sun.nio.ch.Poller$$Lambda/0x00000007c2052400.run(Unknown Source)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)
    at java.base@21.0.2/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Write-Poller" Id=69 in RUNNABLE (running in native)
    at java.base@21.0.2/sun.nio.ch.EPoll.wait(Native Method)
    at java.base@21.0.2/sun.nio.ch.EPollPoller.poll(EPollPoller.java:71)
    at java.base@21.0.2/sun.nio.ch.Poller.poll(Poller.java:363)
    at java.base@21.0.2/sun.nio.ch.Poller.pollLoop(Poller.java:270)
    at java.base@21.0.2/sun.nio.ch.Poller$$Lambda/0x00000007c2051fc0.run(Unknown Source)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)
    at java.base@21.0.2/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Write-Updater" Id=70 in WAITING on lock=java.util.concurrent.LinkedTransferQueue@725fefac
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.LinkedTransferQueue$DualNode.await(LinkedTransferQueue.java:458)
    at java.base@21.0.2/java.util.concurrent.LinkedTransferQueue.xfer(LinkedTransferQueue.java:613)
    at java.base@21.0.2/java.util.concurrent.LinkedTransferQueue.take(LinkedTransferQueue.java:1252)
    at java.base@21.0.2/sun.nio.ch.Poller.updateLoop(Poller.java:286)
    at java.base@21.0.2/sun.nio.ch.Poller$$Lambda/0x00000007c2052400.run(Unknown Source)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)
    at java.base@21.0.2/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186)

"Timer-0" Id=75 in TIMED_WAITING on lock=java.util.TaskQueue@2cb59a3b
    at java.base@21.0.2/java.lang.Object.wait0(Native Method)
    at java.base@21.0.2/java.lang.Object.wait(Object.java:366)
    at java.base@21.0.2/java.util.TimerThread.mainLoop(Timer.java:563)
    at java.base@21.0.2/java.util.TimerThread.run(Timer.java:516)

"ForkJoinPool-2-worker-4" Id=80 in WAITING on lock=java.util.concurrent.ForkJoinPool@4b568ea8
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-2-worker-15" Id=91 in TIMED_WAITING on lock=java.util.concurrent.ForkJoinPool@4b568ea8
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.parkUntil(LockSupport.java:449)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1891)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"pool-1-thread-1" Id=108 in TIMED_WAITING on lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7bbe530e
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:269)
    at java.base@21.0.2/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1758)
    at java.base@21.0.2/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
    at java.base@21.0.2/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
    at java.base@21.0.2/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
    at java.base@21.0.2/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base@21.0.2/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
    at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)

"ForkJoinPool-3-worker-1" Id=109 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-2" Id=110 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-3" Id=111 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-4" Id=112 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-5" Id=113 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-7" Id=115 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-8" Id=116 in RUNNABLE
    at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.findHash(EconomicMapImpl.java:292)
    at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.find(EconomicMapImpl.java:253)
    at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.get(EconomicMapImpl.java:244)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.InterproceduralPartialEscapeAnalysisPhase$a.a(stripped:241)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:140)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:85)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d$$Lambda/0x00000007c29ad670.accept(Unknown Source)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.nodes.a.c(stripped:452)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:76)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePolicyFactory$a.a(stripped:150)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.d.a(stripped:71)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase$a.d(stripped:582)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase$a.c(stripped:537)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase.d(stripped:474)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase.a(stripped:469)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePriorityInliningPhase.a(stripped:191)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePriorityInliningPhase.run(stripped:76)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:322)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.PhaseSuite.run(PhaseSuite.java:390)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:322)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.emitFrontEnd(GraalCompiler.java:294)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:169)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:137)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.defaultCompileFunction(CompileQueue.java:1310)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$$Lambda/0x00000007c2aae408.compile(Unknown Source)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.doCompile(CompileQueue.java:1258)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$CompileTask.run(CompileQueue.java:306)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:187)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:171)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor$$Lambda/0x00000007c238d648.run(Unknown Source)
    at java.base@21.0.2/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
    at java.base@21.0.2/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-6" Id=114 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-9" Id=117 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-10" Id=118 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-11" Id=119 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-12" Id=120 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-13" Id=121 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-14" Id=122 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

"ForkJoinPool-3-worker-15" Id=123 in WAITING on lock=java.util.concurrent.ForkJoinPool@57728469
    at java.base@21.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@21.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1893)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1809)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

=== Memory statistics (in MB):
=== Used heap size: 4290
=== Free heap size: 1765
=== Maximum heap size: 10404
=== Image generator watchdog is aborting image generation. To configure the watchdog, use the options -H:DeadlockWatchdogInterval=10 and -H:+DeadlockWatchdogExitOnTimeout
Error: Image build request for 'application' (pid: 90095, path: /home/ch3cho/TestProjects/flightapp/target) failed with exit status 30

fernando-valdez commented 8 months ago

This issue seems to be related to this one https://github.com/oracle/graal/issues/4822, so I will ask the same question posted there:

What was your working directory and how many subfolders does it contain? Native Image scans all the folders in the current working directory to find classes that should be on the classpath. If this folder is large, especially on network drives, this can take some time. This behavior is going to change in the future, but it requires significant changes to the image builder.

Is this your case? There is already a fix for the ticket I shared before. But I think is important to see if this is a similar scenario.

CH3CHO commented 8 months ago

This issue seems to be related to this one #4822, so I will ask the same question posted there:

What was your working directory and how many subfolders does it contain? Native Image scans all the folders in the current working directory to find classes that should be on the classpath. If this folder is large, especially on network drives, this can take some time. This behavior is going to change in the future, but it requires significant changes to the image builder.

Is this your case? There is already a fix for the ticket I shared before. But I think is important to see if this is a similar scenario.

It is a single-module maven project, so the structure is very simple. And I use the following command for building: mvn clean package native:compile-no-fork -P native -Dmaven.test.skip=true -U

Project structure:

.
├── mvnw
├── mvnw.cmd
├── native-deadlock.log
├── pom.xml
├── README.md
├── src
│   └── main
│       ├── java
│       │   └── com
│       │       └── xxxx
│       │           └── yyyy
│       │               └── testapp
│       │                   ├── AppSpringBootConfiguration.java
│       │                   ├── HomeController.java
│       │                   └── ServletInitializer.java
│       └── resources
│           ├── application.properties
│           ├── logback.xml
│           └── META-INF
│               └── app.properties
└── tree.txt

fernando-valdez commented 8 months ago

is there a way you can create a minimal version of the project that can produce the same error?

CH3CHO commented 8 months ago

I just found that after excluding an internal runtime dependency which depends on javaassist and objectasm (not sure if they are related), building process can complete successfully. However, since it's an internal library, I'm unable to share it here. But is it anyway to debug the building process, or see where does it get stuck?

And I'm using Oracle GraalVM now, because we need to use G1 GC. Should I try GraalCE to see if the problem is reproducible?

fernando-valdez commented 8 months ago

You can use the debugging tools explained here: https://www.graalvm.org/latest/reference-manual/native-image/debugging-and-diagnostics/DebugInfo/ Feel free to check if the problem persists in CE, if that is not the case, then we can reduce the scope of the issue to only Oracle GraalVM

CH3CHO commented 8 months ago

, then we can reduce the scope of the issue to only Oracle GraalVM

Thanks for your help, Fernando.

I tried with GraalCE, and it worked normally.

❯ java -version
openjdk version "21.0.2" 2024-01-16
OpenJDK Runtime Environment GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30)
OpenJDK 64-Bit Server VM GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30, mixed mode, sharing)

However, the link you shared above seems to be related to "adding debug information to a generated native image". I'm not sure if it is helpful to this issue. And I will try to find if there is any verbose or debug flag in the native-image executable.

CH3CHO commented 8 months ago

Just excluding org.javassist:javassist can make it work. However, upgrading it to the latest version of 3.30.2-GA doesn't.

I also tried to add --diagnostics-mode --verbose --native-image-info arguments to native-image, but it looks like nothing more is output to the console.

In addition, it seems that only one thread in the ForkJoinPool is running during the building process. Does the following stacktrace ring a bell?

"ForkJoinPool-3-worker-8" Id=116 in RUNNABLE
    at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.findHash(EconomicMapImpl.java:292)
    at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.find(EconomicMapImpl.java:253)
    at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.get(EconomicMapImpl.java:244)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.InterproceduralPartialEscapeAnalysisPhase$a.a(stripped:241)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:140)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:85)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d$$Lambda/0x00000007c29ad670.accept(Unknown Source)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.nodes.a.c(stripped:452)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:76)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePolicyFactory$a.a(stripped:150)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.d.a(stripped:71)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase$a.d(stripped:582)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase$a.c(stripped:537)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase.d(stripped:474)
    at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase.a(stripped:469)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePriorityInliningPhase.a(stripped:191)
    at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePriorityInliningPhase.run(stripped:76)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:322)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.PhaseSuite.run(PhaseSuite.java:390)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:322)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.emitFrontEnd(GraalCompiler.java:294)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:169)
    at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:137)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.defaultCompileFunction(CompileQueue.java:1310)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$$Lambda/0x00000007c2aae408.compile(Unknown Source)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.doCompile(CompileQueue.java:1258)
    at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$CompileTask.run(CompileQueue.java:306)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:187)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:171)
    at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor$$Lambda/0x00000007c238d648.run(Unknown Source)
    at java.base@21.0.2/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
    at java.base@21.0.2/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
    at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
    at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

fniephaus commented 8 months ago

What happens if you use -H:-AOTPriorityInline to disable priority inlining?

CH3CHO commented 8 months ago

Luckily (or saddly), after I cleaned my local Maven cache, the whole build process worked as expected.

I will keep tracking this problem. If it doesn't re-occur this week. I will close this issue. Thanks for your help.

CH3CHO commented 7 months ago

-H:-AOTPriorityInline

The previous problem reoccurred today, and adding -H:-AOTPriorityInline fixed it. Thanks, @fniephaus . However, does this option affect the runtime performance of the built native executable?

fniephaus commented 7 months ago

However, does this option affect the runtime performance of the built native executable?

Yes, it can affect the performance. I only suggested it as a workaround and to see whether it really is related to the priority inliner. Another test that shouldn't affect the performance as much is to use -H:-UseIPEA. Could you try whether that fixes your problem as well?

Nonetheless, it would be great if you could provide us with a reproducer so we can take a closer look at what's going wrong. Thanks!

CH3CHO commented 7 months ago

However, does this option affect the runtime performance of the built native executable?

Yes, it can affect the performance. I only suggested it as a workaround and to see whether it really is related to the priority inliner. Another test that shouldn't affect the performance as much is to use -H:-UseIPEA. Could you try whether that fixes your problem as well?

Nonetheless, it would be great if you could provide us with a reproducer so we can take a closer look at what's going wrong. Thanks!

Thanks. I will try -H:-UseIPEA. Since the cause is an internal library, I will see if I can remove all sensitive code and build a reproducer.

CH3CHO commented 7 months ago

-H:-UseIPEA works. I just check the documentation. It seems that UseIPEA is a sub-option of AOTPriorityInline. So are we narrowing the root cause?

CH3CHO commented 7 months ago

Here is a minimal reproducer, deadlock.zip. Please try the following steps to reproduce this issue:

cd deadlock-agent
./mvnw clean install -Dmaven.test.skip
cd ../deadlock-reproducer
./mvnw clean package native:compile-no-fork -P native -Dmaven.test.skip

CH3CHO commented 7 months ago

BTW, removing -march=x86-64-v2 from buildArgs can make it work.

fniephaus commented 7 months ago

It seems that UseIPEA is a sub-option of AOTPriorityInline. So are we narrowing the root cause?

Yes, that is correct.

BTW, removing -march=x86-64-v2 from buildArgs can make it work.

That's weird because it's defaulting to x86-64-v3 which uses more CPU features.

Shouldn't this be:

              <buildArgs>
                <buildArg>-march=x86-64-v2</buildArg>
                <buildArg>--add-opens java.management/sun.management=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/java.lang=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/java.lang.reflect=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/sun.reflect.annotation=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/java.math=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/java.util=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/sun.util.calendar=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/sun.util.calendar=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/java.io=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.base/java.net=ALL-UNNAMED</buildArg>
                <buildArg>--add-opens java.xml/com.sun.org.apache.xerces.internal.jaxp.datatype=ALL-UNNAMED</buildArg>
              </buildArgs>

CH3CHO commented 7 months ago

Oh, finally got the correct way of configuring multiple buildArgs....

Because some of our K8s nodes use a relatively old CPU architecture, we have to use v2, otherwise it won't work. And on my laptop with v3 support, using v3 can result a successful build, but using v2 can't.

fniephaus commented 7 months ago

Because some of our K8s nodes use a relatively old CPU architecture, we have to use v2, otherwise it won't work. And on my laptop with v3 support, using v3 can result a successful build, but using v2 can't.

Also when you configure multiple buildArgs correctly?

CH3CHO commented 7 months ago

Also when you configure multiple buildArgs correctly?

No, it doesn't work, either.

And I tested all the -march options supported on my AMD64 laptop. The result is as follows:

<!-- Goods -->
<!--                <buildArg>-march=native</buildArg>-->
<!--                <buildArg>-march=haswell</buildArg>-->
<!--                <buildArg>-march=skylake</buildArg>-->
<!--                <buildArg>-march=skylake-avx512</buildArg>-->
<!--                <buildArg>-march=x86-64-v4</buildArg>-->
<!--                <buildArg>-march=x86-64-v3</buildArg>-->
<!-- Bads -->
<!--                <buildArg>-march=x86-64-v2</buildArg>-->
<!--                <buildArg>-march=x86-64-v1</buildArg>-->
<!--                <buildArg>-march=x86-64</buildArg>-->
<!--                <buildArg>-march=compatibility</buildArg>-->

-march=list result:

On AMD64, the following machine types are available:

'compatibility'
  CPU features: all of 'x86-64'
'haswell'
  CPU features: all of 'x86-64' + SSE3 + SSSE3 + SSE4_1 + SSE4_2 + POPCNT + LZCNT + AVX + AVX2 + AES + CLMUL + BMI1 + BMI2 + FMA
'native'
  CPU features: CX8 + CMOV + FXSR + HT + MMX + AMD_3DNOW_PREFETCH + SSE + SSE2 + SSE3 + SSSE3 + SSE4A + SSE4_1 + SSE4_2 + POPCNT + LZCNT + TSC + TSCINV_BIT + AVX + AVX2 + AES + CLMUL + BMI1 + BMI2 + ADX + SHA + FMA + VZEROUPPER + FLUSH + FLUSHOPT + RDTSCP + RDPID + F16C
'skylake'
  CPU features: all of 'haswell' + AMD_3DNOW_PREFETCH + ADX + FLUSHOPT
'skylake-avx512'
  CPU features: all of 'skylake' + AVX512F + AVX512DQ + AVX512CD + AVX512BW + AVX512VL + CLWB
'x86-64'
  CPU features: CX8 + CMOV + FXSR + MMX + SSE + SSE2
'x86-64-v1'
  CPU features: all of 'x86-64'
'x86-64-v2'
  CPU features: all of 'x86-64-v1' + SSE3 + SSSE3 + SSE4_1 + SSE4_2 + POPCNT
'x86-64-v3'
  CPU features: all of 'x86-64-v2' + LZCNT + AVX + AVX2 + BMI1 + BMI2 + FMA
'x86-64-v4'
  CPU features: all of 'x86-64-v3' + AVX512F + AVX512DQ + AVX512CD + AVX512BW + AVX512VL

CH3CHO commented 7 months ago

Hi guys, any new findings on this issue? Thanks.

CH3CHO commented 7 months ago

Update: It seems not to be related to the arch option. Adding several dependencies can also result in the same error.

oracle / graal

[GR-39609] Deadlock breaks the native image building process #8685