Karm / mandrel-integration-tests

Integration tests for GraalVM and its Mandrel distribution. Runs Quarkus, Helidon and Micronaut applications and small targeted reproducers. The focus is solely on native-image utility and compilation of Java applications into native executables.
Apache License 2.0
5 stars 3 forks source link

Make FAIL_ON_PERF_REGRESSION more fine grain #205

Closed zakkak closed 9 months ago

zakkak commented 9 months ago

Allow for failing only when time insensitive, time sensitive, or both thresholds fail.

Closes: #204

Karm commented 9 months ago

@zakkak Hmm, OOM. That is weird. Lemme re-run the failed one...

[INFO] [io.quarkus.smallrye.jwt.deployment.SmallRyeJwtProcessor] Adding META-INF/resources/publicKey.pem to native image
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/ts/apps/quarkus-full-microprofile/target/quarkus-native-image-source-jar/quarkus-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/ts/apps/quarkus-full-microprofile/target/quarkus-native-image-source-jar/quarkus-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on native-image 20.0.2 2023-07-18
OpenJDK Runtime Environment Mandrel-23.0.1.2-Final (build 20.0.2+9)
OpenJDK 64-Bit Server VM Mandrel-23.0.1.2-Final (build 20.0.2+9, mixed mode)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/20-23.0.1.2-Final/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dlogging.initial-configurator.min-level=500 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -H:+TrackNodeSourcePosition -H:+DebugCodeInfoUseSourceMappings -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:BuildOutputJSONFile=quarkus-runner-build-output-stats.json -H:Log=registerResource: --trace-object-instantiation=java.util.Random --initialize-at-run-time=io.vertx.ext.auth.impl.jose.JWT -H:+AllowFoldMethods -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+ReportExceptionStackTraces -g -H:DebugInfoSourceSearchPath=app-sources -H:-AddAllCharsets --enable-url-protocols=http,https -H:NativeLinkerOption=-no-pie -H:-UseServiceLoaderFeature -H:+StackTrace -J--add-exports=org.graalvm.sdk/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-runner -jar quarkus-runner.jar
========================================================================================================================
GraalVM Native Image: Generating 'quarkus-runner' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------
[1/8] Initializing...                                                                                   (12.4s @ 0.13GB)
 Java version: 20.0.2+9, vendor version: Mandrel-23.0.1.2-Final
 Graal compiler: optimization level: 2, target machine: x86-64-v3
 C compiler: gcc (linux, x86_64, 9.4.0)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 4 user-specific feature(s)
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
 - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
 - org.eclipse.angus.activation.nativeimage.AngusActivationFeature
07:11:09,257 INFO  [org.jbo.res.res.i18n] RESTEASY002225: Deploying jakarta.ws.rs.core.Application: class com.example.quarkus.QuarkusRestApplication
[2/8] Performing analysis...  [*****]                                                                  (131.9s @ 1.48GB)
  16,803 (89.35%) of 18,806 types reachable
  25,765 (63.05%) of 40,864 fields reachable
  83,605 (58.03%) of 144,065 methods reachable
   5,052 types,   273 fields, and 3,284 methods registered for reflection
      61 types,    59 fields, and    55 methods registered for JNI access
       4 native libraries: dl, pthread, rt, z
[3/8] Building universe...                                                                              (18.3s @ 1.45GB)
[4/8] Parsing methods...      [****]                                                                    (20.8s @ 1.27GB)
[5/8] Inlining methods...     [***]                                                                      (9.8s @ 1.15GB)
[6/8] Compiling methods...    [***********]                                                            (126.8s @ 2.25GB)
[7/8] Layouting methods...    [***]                                                                      (6.7s @ 2.47GB)
Terminating due to java.lang.OutOfMemoryError: Java heap space
The Native Image build process ran out of memory.
Please make sure your build system has more memory available.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  09:30 min
[INFO] Finished at: 2023-09-22T07:20:12Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:3.2.3.Final:build (default) on project quarkus-full-microprofile: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR]     [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: io.quarkus.deployment.pkg.steps.NativeImageBuildStep$ImageGenerationFailureException: Image generation failed. Exit code: 3
[ERROR]     at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.imageGenerationFailed(NativeImageBuildStep.java:457)
[ERROR]     at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:263)
[ERROR]     at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
[ERROR]     at java.base/java.lang.reflect.Method.invoke(Method.java:578)
[ERROR]     at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:864)
[ERROR]     at io.quarkus.builder.BuildContext.run(BuildContext.java:282)
[ERROR]     at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR]     at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
[ERROR]     at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
[ERROR]     at java.base/java.lang.Thread.run(Thread.java:1623)
[ERROR]     at org.jboss.threads.JBossThread.run(JBossThread.java:501)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
mvn dependency:sources -Dquarkus.version=3.2.3.Final
Command: mvn dependency:sources -Dquarkus.version=3.2.3.Final
[INFO] Scanning for projects...
[INFO] 
Karm commented 9 months ago

let's see the CI green, just in case

zakkak commented 9 months ago

The failure is different than the previous and seems unrelated to the PR since I have seen it failing in other PRs as well.