search

apache / camel-quarkus

Apache Camel Quarkus
https://camel.apache.org
Apache License 2.0
256 stars 189 forks source link

Master OpenShift integration test native compilation fails intermittently #5071

Closed jamesnetherton closed 1 year ago

jamesnetherton commented 1 year ago

Bug description

Native image compilation took over 20 minutes before it eventually failed. Probably due to out of memory.

2023-07-06T15:16:52.0283649Z [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm --user 1001:123 -v /home/runner/work/camel-quarkus/camel-quarkus/integration-tests/master-openshift/target/camel-quarkus-integration-test-master-openshift-3.0.0-SNAPSHOT-native-image-source-jar:/project:z --name build-native-KEdyo quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-17 -J-Dlogging.initial-configurator.min-level=500 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:BuildOutputJSONFile=camel-quarkus-integration-test-master-openshift-3.0.0-SNAPSHOT-runner-build-output-stats.json -H:+AllowFoldMethods -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+ReportExceptionStackTraces -H:-AddAllCharsets --enable-url-protocols=http,https -H:-UseServiceLoaderFeature -H:+StackTrace -J--add-exports=org.graalvm.sdk/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json camel-quarkus-integration-test-master-openshift-3.0.0-SNAPSHOT-runner -jar camel-quarkus-integration-test-master-openshift-3.0.0-SNAPSHOT-runner.jar
2023-07-06T15:17:11.0200983Z ========================================================================================================================
2023-07-06T15:17:11.0222512Z GraalVM Native Image: Generating 'camel-quarkus-integration-test-master-openshift-3.0.0-SNAPSHOT-runner' (executable)...
2023-07-06T15:17:11.0227073Z ========================================================================================================================
2023-07-06T15:17:14.0027901Z Warning: Could not resolve class org.apache.commons.logging.impl.LogFactoryImpl for reflection configuration. Reason: java.lang.ClassNotFoundException: org.apache.commons.logging.impl.LogFactoryImpl.
2023-07-06T15:17:14.0033434Z Warning: Could not resolve class org.apache.commons.logging.impl.Jdk14Logger for reflection configuration. Reason: java.lang.ClassNotFoundException: org.apache.commons.logging.impl.Jdk14Logger.
2023-07-06T15:17:15.5923878Z [1/8] Initializing...                                                                                   (21.1s @ 0.44GB)
2023-07-06T15:17:15.5925051Z  Java version: 17.0.7+7, vendor version: Mandrel-23.0.0.0-Final
2023-07-06T15:17:15.5931054Z  Graal compiler: optimization level: 2, target machine: x86-64-v3
2023-07-06T15:17:15.5931615Z  C compiler: gcc (redhat, x86_64, 8.5.0)
2023-07-06T15:17:15.5933750Z  Garbage collector: Serial GC (max heap size: 80% of RAM)
2023-07-06T15:17:15.5943939Z  3 user-specific feature(s)
2023-07-06T15:17:15.5947572Z  - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
2023-07-06T15:17:15.5956906Z  - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
2023-07-06T15:17:15.5957495Z  - org.eclipse.angus.activation.nativeimage.AngusActivationFeature
2023-07-06T15:20:51.9368017Z WARNING: Resource scanning is taking a long time. This can be caused by class-path or module-path entries that point to large directory structures. Please make sure class-/module-path entries are easily accessible to native-image
2023-07-06T15:20:51.9368674Z Total scanned entries: 17828, current entry: file:///opt/mandrel/lib/jvmci/graal.jar
2023-07-06T15:21:11.3893046Z Total scanned entries: 23544, current entry: jrt:/java.scripting/com/sun/tools/script/shell/init.js
2023-07-06T15:21:30.8612560Z Total scanned entries: 29236, current entry: jrt:/jdk.compiler/com/sun/tools/javac/code/Types$20.class
2023-07-06T15:21:50.3335969Z Total scanned entries: 34925, current entry: jrt:/java.desktop/javax/imageio/plugins/tiff/BaselineTIFFTagSet$PageNumber.class
2023-07-06T15:22:10.1245576Z Total scanned entries: 40616, current entry: file:///project/lib/org.apache.camel.camel-api-4.0.0-RC1.jar
2023-07-06T15:22:30.0892081Z Total scanned entries: 46734, current entry: file:///project/lib/org.apache.camel.camel-main-4.0.0-RC1.jar
2023-07-06T15:22:52.0872716Z Total scanned entries: 53155, current entry: file:///project/lib/io.fabric8.kubernetes-model-core-6.7.2.jar
2023-07-06T15:23:11.5419487Z Total scanned entries: 58871, current entry: file:///project/lib/io.fabric8.kubernetes-model-flowcontrol-6.7.2.jar
2023-07-06T15:23:31.1211950Z Total scanned entries: 64591, current entry: file:///project/lib/io.smallrye.config.smallrye-config-core-3.3.0.jar
2023-07-06T15:23:50.6298226Z Total scanned entries: 70299, current entry: file:///project/lib/io.fabric8.openshift-model-6.7.2.jar
2023-07-06T15:24:10.0702528Z Total scanned entries: 76241, current entry: file:///project/lib/io.fabric8.openshift-model-operator-6.7.2.jar
2023-07-06T15:24:31.7275057Z Total scanned entries: 82420, current entry: file:///project/lib/io.fabric8.openshift-model-operator-6.7.2.jar
2023-07-06T15:24:50.9867271Z Total scanned entries: 88124, current entry: file:///project/lib/io.fabric8.openshift-model-machine-6.7.2.jar
2023-07-06T15:26:28.4439235Z [2/8] Performing analysis...  [*********]                                                              (552.2s @ 2.18GB)
2023-07-06T15:26:28.4468132Z GC warning: 397.0s spent in 252 GCs during the last stage, taking up 71.81% of the time.
2023-07-06T15:26:28.4506467Z             Please ensure more than 3.32GB of memory is available for Native Image
2023-07-06T15:26:28.4506839Z             to reduce GC overhead and improve image build time.
2023-07-06T15:26:28.4557832Z   21,638 (71.93%) of 30,083 types reachable
2023-07-06T15:26:28.4647463Z   40,550 (72.36%) of 56,039 fields reachable
2023-07-06T15:26:28.5208840Z  142,178 (62.17%) of 228,677 methods reachable
2023-07-06T15:26:28.5209205Z    8,969 types, 1,729 fields, and 44,683 methods registered for reflection
2023-07-06T15:26:28.5209567Z       64 types,    70 fields, and    55 methods registered for JNI access
2023-07-06T15:26:28.5210398Z        4 native libraries: dl, pthread, rt, z
2023-07-06T15:38:57.0862911Z 
2023-07-06T15:38:57.0877199Z === Image generator watchdog detected no activity. This can be a sign of a deadlock during image building. Dumping all stack traces. Current time: Thu Jul 06 15:38:57 GMT 2023
jamesnetherton commented 1 year ago

I've not seen this occur since #5128. So I think we're good to close this one.