Closed zakkak closed 8 months ago
quarkus-bot[bot]
commented
9 months ago You added a link to a Zulip discussion, please make sure the description of the issue is comprehensive and doesn't require accessing Zulip
This message is automatically generated by a bot.
quarkus-bot[bot]
commented
9 months ago /cc @Karm (mandrel), @galderz (mandrel), @geoand (kubernetes,openshift), @iocanel (kubernetes,openshift)
zakkak
commented
9 months ago Looking at the provided build output one can already see some things::
Note that these differences do not match the ones we observed in https://quarkus.io/blog/mandrel-23-1-image-size-increase/
It looks like in Q 3.6 we are building double the code...
If I understand correctly the way this was tested is by building the exact same code with Quarkus 3.5 and then with Quarkus 3.6, which means that you use both a different set of dependencies and a different builder image. Could you please run another build with Quarkus 3.6 but this time with the jdk-17 builder image? i.e. passing -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-17
This will allow us to see if the main difference is due to the different Mandrel builder image or due to some Quarkus change.
It looks like what you are seeing is also related to https://github.com/quarkusio/quarkus/issues/37142
You might want to give https://github.com/quarkusio/quarkus/pull/37278 a try:
This PR allows for the exclusion of openshift-model-hive, openshift-model-miscellaneous, openshift-model-operator in native mode, which will already downsize the resulting binary considerably.
Indeed running with Quarkus 3.7.1 and Mandrel 23.0.3 gives similar results, indicating that this is not caused by the Mandrel update, but by some dependency change (e.g. "in Quarkus 3.5.3, we are using io.fabric8 6.8.1, and 6.9.2 in 3.6"):
$ mvn clean package -o -DskipTests -Dnative -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-17
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on MANDREL 23.0.3.0 JDK 17.0.10+7
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm -v /e/git/java-xx/target/xx-5.1.1-SNAPSHOT-native-image-source-jar:/project:z --name build-native-KGynW quay.io/quarkus/ub
i-quarkus-mandrel-builder-image:jdk-17 -J-DCoordinatorEnvironmentBean.transactionStatusManagerEnable=false -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dio.quarkus.caffein
e.graalvm.recordStats=true -J-Dlogging.initial-configurator.min-level=500 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetec
tion.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.hibernate.validator.runtime.DisableLoggingFeature,io.quarkus.r
untime.graal.DisableLoggingFeature,io.quarkus.caffeine.runtime.graal.CacheConstructorsFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/
java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:BuildOutputJSONFile=xx-5.1.1-SNAPSHOT-runner-build-output-stats.json --allow-incomplete-classpath -H
:+AllowFoldMethods -J-Djava.awt.headless=true --no-fallback -H:+ReportExceptionStackTraces -J-Xmx20g -H:-AddAllCharsets --enable-url-protocols=http,https --enable-monitoring=heapdump -H:-UseServiceLoaderFeature -H:+StackTrace -J-
-add-exports=org.graalvm.sdk/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config com\.oracle\.database\.jdbc /META-INF/native-image/native-image\.properties --exclude-config com\.oracle\.database\.jdbc /META-INF/native-imag
e/reflect-config\.json --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-
handler/generated/handlers/reflect-config\.json xx-5.1.1-SNAPSHOT-runner -jar xx-5.1.1-SNAPSHOT-runner.jar
Warning: Using a deprecated option --allow-incomplete-classpath from command line. Allowing an incomplete classpath is now the default. Use --link-at-build-time to report linking errors at image build time for a class or package.
========================================================================================================================
GraalVM Native Image: Generating 'xx-5.1.1-SNAPSHOT-runner' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------
Warning: Feature class oracle.nativeimage.NativeImageFeature is annotated with the deprecated annotation @AutomaticFeature. Support for this annotation will be removed in a future version of GraalVM. Applications should register
a feature using the option --features=oracle.nativeimage.NativeImageFeature
...
[1/8] Initializing... (54.9s @ 0.46GB)
Java version: 17.0.10+7, vendor version: Mandrel-23.0.3.0-Final
Graal compiler: optimization level: 2, target machine: x86-64-v3
C compiler: gcc (redhat, x86_64, 8.5.0)
Garbage collector: Serial GC (max heap size: 80% of RAM)
6 user-specific feature(s)
- com.oracle.svm.thirdparty.gson.GsonFeature
- io.quarkus.caffeine.runtime.graal.CacheConstructorsFeature
- io.quarkus.hibernate.validator.runtime.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.hibernate.validator.internal.util.Version] categories
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
- oracle.nativeimage.NativeImageFeature
The bundle named: oracle.net.mesg.Message, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: oracle.net.mesg.Message, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
17:17:51,730 WARN [fre.dom] No XPath support is available. If you need it, add Apache Xalan or Jaxen as dependency.
[2/8] Performing analysis... [*****] (865.3s @ 8.90GB)
44,303 (94.03%) of 47,115 types reachable
91,352 (81.78%) of 111,702 fields reachable
402,379 (84.68%) of 475,176 methods reachable
28,167 types, 57,447 fields, and 240,611 methods registered for reflection
71 types, 93 fields, and 56 methods registered for JNI access
4 native libraries: dl, pthread, rt, z
[3/8] Building universe... (82.7s @ 8.96GB)
GC warning: 47.0s spent in 5 GCs during the last stage, taking up 56.49% of the time.
Please ensure more than 14.26GB of memory is available for Native Image
to reduce GC overhead and improve image build time.
[4/8] Parsing methods... [******] (44.4s @ 10.07GB)
[5/8] Inlining methods... [****] (13.3s @ 8.63GB)
[6/8] Compiling methods... [***************] (240.5s @ 5.06GB)
[7/8] Layouting methods... [******] (44.2s @ 5.89GB)
[8/8] Creating image... [*******] (55.8s @ 6.98GB)
152.88MB (50.89%) for code area: 353,694 compilation units
147.16MB (48.98%) for image heap:1,233,195 objects and 138 resources
410.15kB ( 0.13%) for other data
300.43MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
18.32MB io.fabric8.openshift-model-operator-6.10.0.jar 31.86MB byte[] for code metadata
16.92MB svm.jar (Native Image) 22.61MB java.lang.Class
14.28MB io.fabric8.openshift-model-6.10.0.jar 22.18MB byte[] for java.lang.String
13.53MB java.base 19.25MB byte[] for reflection metadata
9.11MB io.fabric8.kubernetes-model-core-6.10.0.jar 10.97MB java.lang.String
4.70MB io.fabric8.openshift-model-monitoring-6.10.0.jar 10.95MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
4.68MB io.fabric8.openshift-model-hive-6.10.0.jar 8.33MB byte[] for general heap data
4.66MB xx-5.1.1-SNAPSHOT-runner.jar 3.72MB com.oracle.svm.core.hub.DynamicHubCompanion
2.90MB i.f.k.10.0.jar 2.58MB java.lang.Object[]
/project/xx-5.1.1-SNAPSHOT-runner-build-output-stats.json (build_info)
========================================================================================================================
Finished generating 'xx-5.1.1-SNAPSHOT-runner' in 23m 26s.
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm -v /e/git/java-xx/target/xx-5.1.1-SNAPSHOT-native-image-source-jar:/project:z --entrypoint /bin/bash quay.io/quarkus/ubi-q
uarkus-mandrel-builder-image:jdk-17 -c objcopy --strip-debug xx-5.1.1-SNAPSHOT-runner
[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 1527193ms
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:attach-artifact (attach-artifacts) @ xx ---
[INFO] Skip attaching artifacts
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26:21 min
[INFO] Finished at: 2024-02-07T18:38:06+01:00
[INFO] ------------------------------------------------------------------------@vsevel also reported that:
I was able to get some savings with the referenced exclusions (Quarkus 3.7.1 ):
75.9s (11.4% of total time) in 148 GCs | Peak RSS: 12.05GB | CPU load: 4.50 ... Finished generating 'x-5.1.1-SNAPSHOT-runner' in 10m 56s.better than without the exclusion, still not as good compared to 3.5 :
27.2s (8.2% of total time) in 92 GCs | Peak RSS: 10.41GB | CPU load: 4.18 ------------------------------------------------------------------------------------------------------------------------ Produced artifacts: /project/xx-5.1.1-SNAPSHOT-runner (executable) /project/xx-5.1.1-SNAPSHOT-runner-build-output-stats.json (build_info) ======================================================================================================================== Finished generating 'xx-5.1.1-SNAPSHOT-runner' in 5m 23s.I am going to try to see if I can add more exclusions. still I am very much surprised about the impact additional libraries can have on the build.
gsmet
commented
9 months ago FWIW, I know that at some point a Kubernetes Client update caused some problems as the amount of code was massive.
vsevel
commented
9 months ago I am continuing trying to get rid of as many models as possible. it is a pain, because some missing models appear at native runtime only. so it is a lengthy process.
zakkak
commented
9 months ago FWIW, I know that at some point a Kubernetes Client update caused some problems as the amount of code was massive.
Correct, that was https://github.com/quarkusio/quarkus/issues/37142
vsevel
commented
9 months ago with the latest exclusions in 3.7.1 I get on our CI:
49.4s (14.7% of total time) in 171 GCs | Peak RSS: 11.30GB | CPU load: 6.04
Finished generating 'ocpdeploy-5.1.1-SNAPSHOT-runner' in 5m 35s.on a test branch for 3.5 I get:
23.4s (10.0% of total time) in 115 GCs | Peak RSS: 8.32GB | CPU load: 5.77
Finished generating 'ocpdeploy-5.1.1-SNAPSHOT-runner' in 3m 53s.our baseline on 3.2 (master) :
30.2s (13.2% of total time) in 167 GCs | Peak RSS: 5.83GB | CPU load: 5.91
Finished generating 'ocpdeploy-5.1.1-SNAPSHOT-runner' in 3m 48s.the increase of memory is scary. our build machines are going to have a hard time coping with processes 11Gb of RAM.
here is the exclusions I was able to do:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-openshift-client</artifactId>
<exclusions>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-hive</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-miscellaneous</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-operator</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-clusterautoscaling</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-operatorhub</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-machine</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-whereabouts</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-storageversionmigrator</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-tuned</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-console</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-machineconfig</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>openshift-model-installer</artifactId>
</exclusion>
</exclusions>
</dependency>and
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>certmanager-client</artifactId>
<exclusions>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-resource</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-rbac</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-admissionregistration</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-autoscaling</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-coordination</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-discovery</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-events</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-flowcontrol</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-metrics</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-policy</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-scheduling</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-storageclass</artifactId>
</exclusion>
<exclusion>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-model-node</artifactId>
</exclusion>
</exclusions>
</dependency>even with all that cleanup, this shows a signifiant increase in memory and build time. what else can I do?
vsevel
commented
9 months ago 3.5:
2024.02.07 12:58:38 [2/8] Performing analysis... [**********] (89.6s @ 3.25GB)
2024.02.07 12:58:38 25,787 (77.52%) of 33,265 types reachable
2024.02.07 12:58:38 49,039 (70.80%) of 69,265 fields reachable
2024.02.07 12:58:38 168,338 (67.65%) of 248,837 methods reachable
2024.02.07 12:58:38 9,796 types, 2,542 fields, and 51,144 methods registered for reflection
2024.02.07 12:58:38 71 types, 93 fields, and 56 methods registered for JNI access
2024.02.07 12:58:38 4 native libraries: dl, pthread, rt, z
2024.02.07 12:58:48 [3/8] Building universe... (12.8s @ 3.43GB)
2024.02.07 12:58:58 [4/8] Parsing methods... [***] (10.7s @ 3.57GB)
2024.02.07 12:59:06 [5/8] Inlining methods... [****] (6.8s @ 2.94GB)
2024.02.07 13:00:14 [6/8] Compiling methods... [*******] (57.7s @ 5.28GB)
2024.02.07 13:00:19 [7/8] Layouting methods... [****] (14.7s @ 3.91GB)
2024.02.07 13:00:41 [8/8] Creating image... [****] (20.2s @ 5.33GB)
2024.02.07 13:00:41 59.47MB (50.26%) for code area: 122,614 compilation units
2024.02.07 13:00:41 58.45MB (49.40%) for image heap: 644,721 objects and 136 resources
2024.02.07 13:00:41 408.55kB ( 0.34%) for other data
2024.02.07 13:00:41 118.31MB in total
2024.02.07 13:00:41 ------------------------------------------------------------------------------------------------------------------------
2024.02.07 13:00:41 Top 10 origins of code area: Top 10 object types in image heap:
2024.02.07 13:00:41 13.52MB java.base 13.09MB byte[] for code metadata
2024.02.07 13:00:41 5.99MB svm.jar (Native Image) 6.76MB java.lang.Class
2024.02.07 13:00:41 3.84MB ocpdeploy-5.1.1-SNAPSHOT-runner.jar 6.22MB byte[] for java.lang.String
2024.02.07 13:00:41 3.33MB io.fabric8.kubernetes-model-core-6.8.1.jar 5.81MB java.lang.String
2024.02.07 13:00:41 2.18MB org.freemarker.freemarker-2.3.32.jar 4.92MB byte[] for general heap data
2024.02.07 13:00:41 1.93MB c.f.jackson.core.jackson-databind-2.15.3.jar 4.61MB byte[] for reflection metadata
2024.02.07 13:00:41 1.32MB modified-io.vertx.vertx-core-4.4.6.jar 2.25MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
2024.02.07 13:00:41 1.28MB java.net.http 2.16MB com.oracle.svm.core.hub.DynamicHubCompanion
2024.02.07 13:00:41 1.23MB io.fabric8.openshift-model-6.8.1.jar 1.21MB java.lang.String[]
2024.02.07 13:00:41 627.05kB java.naming 934.53kB c.o.svm.core.hub.DynamicHub$ReflectionMetadata
2024.02.07 13:00:41 23.24MB for 264 more packages 9.79MB for 5560 more object types3.7
2024.02.08 17:24:43 [2/8] Performing analysis... [*****] (138.7s @ 5.92GB)
2024.02.08 17:24:43 32,715 reachable types (91.6% of 35,730 total)
2024.02.08 17:24:43 62,938 reachable fields (75.3% of 83,560 total)
2024.02.08 17:24:43 270,083 reachable methods (78.6% of 343,567 total)
2024.02.08 17:24:43 16,500 types, 29,030 fields, and 133,756 methods registered for reflection
2024.02.08 17:24:43 69 types, 91 fields, and 56 methods registered for JNI access
2024.02.08 17:24:43 4 native libraries: dl, pthread, rt, z
2024.02.08 17:24:55 [3/8] Building universe... (22.2s @ 6.12GB)
2024.02.08 17:25:07 [4/8] Parsing methods... [***] (11.4s @ 4.77GB)
2024.02.08 17:25:14 [5/8] Inlining methods... [****] (6.3s @ 7.43GB)
2024.02.08 17:26:36 [6/8] Compiling methods... [*********] (80.0s @ 4.02GB)
2024.02.08 17:27:03 [7/8] Layouting methods... [*****] (27.5s @ 7.24GB)
2024.02.08 17:27:35 [8/8] Creating image... [*****] (31.9s @ 4.55GB)
2024.02.08 17:27:35 107.24MB (50.12%) for code area: 224,347 compilation units
2024.02.08 17:27:35 106.34MB (49.69%) for image heap: 901,740 objects and 200 resources
2024.02.08 17:27:35 417.65kB ( 0.19%) for other data
2024.02.08 17:27:35 213.99MB in total
2024.02.08 17:27:35 ------------------------------------------------------------------------------------------------------------------------
2024.02.08 17:27:35 Top 10 origins of code area: Top 10 object types in image heap:
2024.02.08 17:27:35 14.59MB io.fabric8.openshift-model-6.10.0.jar 33.65MB byte[] for code metadata
2024.02.08 17:27:35 13.77MB java.base 17.27MB byte[] for java.lang.String
2024.02.08 17:27:35 10.05MB svm.jar (Native Image) 11.33MB java.lang.Class
2024.02.08 17:27:35 9.33MB io.fabric8.kubernetes-model-core-6.10.0.jar 10.98MB byte[] for reflection metadata
2024.02.08 17:27:35 4.83MB io.fabric8.openshift-model-monitoring-6.10.0.jar 8.13MB java.lang.String
2024.02.08 17:27:35 4.75MB ocpdeploy-5.1.1-SNAPSHOT-runner.jar 6.12MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
2024.02.08 17:27:35 2.98MB i.f.k.10.0.jar 2.75MB com.oracle.svm.core.hub.DynamicHubCompanion
2024.02.08 17:27:35 2.87MB io.fabric8.kubernetes-model-gatewayapi-6.10.0.jar 1.70MB java.lang.Object[]
2024.02.08 17:27:35 2.24MB org.freemarker.freemarker-2.3.32.jar 1.70MB java.lang.String[]
2024.02.08 17:27:35 2.21MB io.fabric8.certmanager-model-v1alpha2-6.10.0.jar 1.49MB byte[] for general heap data
2024.02.08 17:27:35 37.92MB for 245 more packages 11.22MB for 5683 more object types3.2
2024.02.07 22:55:47 [2/8] Performing analysis... [*********] (90.9s @ 2.90GB)
2024.02.07 22:55:47 26,276 (73.80%) of 35,603 types reachable
2024.02.07 22:55:47 49,003 (70.23%) of 69,779 fields reachable
2024.02.07 22:55:47 167,491 (64.73%) of 258,751 methods reachable
2024.02.07 22:55:47 10,020 types, 2,067 fields, and 48,594 methods registered for reflection
2024.02.07 22:55:47 71 types, 93 fields, and 56 methods registered for JNI access
2024.02.07 22:55:47 4 native libraries: dl, pthread, rt, z
2024.02.07 22:55:58 [3/8] Building universe... (12.5s @ 3.39GB)
2024.02.07 22:56:08 [4/8] Parsing methods... [***] (11.0s @ 2.04GB)
2024.02.07 22:56:13 [5/8] Inlining methods... [****] (4.1s @ 1.91GB)
2024.02.07 22:57:09 [6/8] Compiling methods... [*******] (55.3s @ 3.86GB)
2024.02.07 22:57:25 [7/8] Layouting methods... [****] (14.6s @ 2.77GB)
2024.02.07 22:57:43 [8/8] Creating image... [****] (18.8s @ 2.92GB)
2024.02.07 22:57:43 59.40MB (50.50%) for code area: 120,579 compilation units
2024.02.07 22:57:43 57.81MB (49.16%) for image heap: 645,267 objects and 134 resources
2024.02.07 22:57:43 412.78kB ( 0.34%) for other data
2024.02.07 22:57:43 117.61MB in total
2024.02.07 22:57:43 ------------------------------------------------------------------------------------------------------------------------
2024.02.07 22:57:43 Top 10 origins of code area: Top 10 object types in image heap:
2024.02.07 22:57:43 13.61MB java.base 13.10MB byte[] for code metadata
2024.02.07 22:57:43 6.02MB svm.jar (Native Image) 6.94MB java.lang.Class
2024.02.07 22:57:43 3.76MB ocpdeploy-5.1.1-SNAPSHOT-runner.jar 6.26MB byte[] for java.lang.String
2024.02.07 22:57:43 2.97MB io.fabric8.kubernetes-model-core-6.7.2.jar 5.83MB java.lang.String
2024.02.07 22:57:43 2.18MB org.freemarker.freemarker-2.3.32.jar 4.49MB byte[] for general heap data
2024.02.07 22:57:43 1.93MB c.f.jackson.core.jackson-databind-2.15.2.jar 4.43MB byte[] for reflection metadata
2024.02.07 22:57:43 1.30MB modified-io.vertx.vertx-core-4.4.4.jar 2.21MB com.oracle.svm.core.hub.DynamicHubCompanion
2024.02.07 22:57:43 1.28MB java.net.http 2.13MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
2024.02.07 22:57:43 1.14MB io.fabric8.openshift-model-6.7.2.jar 1.21MB java.lang.String[]
2024.02.07 22:57:43 629.42kB java.naming 944.41kB c.o.svm.core.hub.DynamicHub$ReflectionMetadata
2024.02.07 22:57:43 23.58MB for 270 more packages 9.64MB for 5594 more object typestests on 3.6 and 3.7 were run with a large xmx (20Gb). I reran the tests with the original xmx=7g that we have on master, and it changes the numbers I gave earlier:
3.7
55.1s (15.7% of total time) in 283 GCs | Peak RSS: 7.51GB | CPU load: 6.00
Finished generating 'ocpdeploy-5.1.1-SNAPSHOT-runner' in 5m 50s.3.6
57.8s (16.6% of total time) in 308 GCs | Peak RSS: 7.21GB | CPU load: 5.38
Finished generating 'ocpdeploy-5.1.1-SNAPSHOT-runner' in 5m 46s.baseline 3.2
30.2s (13.2% of total time) in 167 GCs | Peak RSS: 5.83GB | CPU load: 5.91
Finished generating 'ocpdeploy-5.1.1-SNAPSHOT-runner' in 3m 48s.not perfect, but memory looks far better.
manusa
commented
9 months ago even with all that cleanup, this shows a significant increase in memory and build time. what else can I do?
I'm not 100% sure, but I'd say that with those exclusions, the client models shouldn't be accountable for the increase in size.
Take the kubernetes-model-core as an example (which is fairly stable).
The original jar size vs the reported origins of code area:
4907714:2.97MB3665233:3.33MB3631645:9.33MBEven when the original jar file is reducing its size (we did split it up at some point), the resulting size accounted for these types keeps increasing.
zakkak
commented
9 months ago @vsevel looking at the 3.5 vs 3.7 build outputs (I am assuming you are using Mandrel 23.0 in 3.5 and Mandrel 23.1 in 3.7) I see that both the code area and the heap size increased.
For the code area increase (by ~50MB) the key contributors appear to be:
io.fabric8.openshift-model-6.10.0.jar (is ~13MB larger than it used to be in 6.8.1)svm.jar (is ~4MB larger than it used to in Mandrel 23.0)io.fabric8.kubernetes-model-core-6.10.0.jar (is ~6MB larger than it used to be in 6.8.1)io.fabric8.openshift-model-monitoring-6.10.0.jar (brings at least 3MB extra)io.fabric8.kubernetes-model-gatewayapi-6.10.0.jar (brings at least 2MB extra)For the heap area increase (by ~50MB) the key contributors appear to be:
All the above indicate that more code has become reachable and is leading in the build process to be more resource hungry and to take longer. It also results in a larger binary in the end. I think that at this point there is not much you can do on the application side, but we also can't to much without more info.
It would be great if you could share a reproducer with us, but if that's not possible it would also help if you could run a 3.5 and a 3.7 build and a quick run afterwards with the following parameters and share the output files with us.
parameters
-Dquarkus.native.additional-build-args=-R:+DumpHeapAndExit,-H:+DashboardAll,-H:+DashboardJson,-H:DashboardDump=path/to/dashboard.json-Dquarkus.native.enable-reportsinteresting files
target/your-app-native-image-sources/your-app-build-output-stats.json (created at build time)your-app.hprof (created after running the native executable of your app)target/your-app-native-image-sources/reports (this is a directory please include all files)dashboard.json.dump (created at build time)zakkak
commented
9 months ago @vsevel I forgot to mention that in order to reduce the moving parts to the minimum it would be better to build only using Mandrel 23.0 for the time being.
adinn
commented
9 months ago One way to identify more about what is increasing build time RSS would be to generate heap dumps during build and see which object types/how many object instances are being held on to. This might indicate which component of GraalVM Native is blowing up.
Heap dumps might also indicate whether the RSS increase is down to a GraalVM/Mandrel change or -- as looks more likely -- a Quarkus change. If you interpolate between old and new configs by generating heap dumps with the Quarkus 3.7.1 and Mandrel 23.0 then that might further clarify which of the two components is responsible (could still be both).
metacosm
commented
9 months ago I have run some quick tests on a sample operator using QOSDK 6.3.x which targets Quarkus 3.2.10 and QOSDK main with Quarkus 3.7.3:
3.2.10:
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on native-image 21.0.2 2024-01-16
GraalVM Runtime Environment GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30)
Substrate VM GraalVM CE 21.0.2+13.1 (build 21.0.2+13, serial gc)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /Users/claprun/.sdkman/candidates/java/21.0.2-graalce/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dlogging.initial-configurator.min-level=500 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=FR -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:BuildOutputJSONFile=quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner-build-output-stats.json -H:+AllowFoldMethods -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+ReportExceptionStackTraces -H:-AddAllCharsets --enable-url-protocols=http,https -H:-UseServiceLoaderFeature -J--add-exports=org.graalvm.sdk/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner -jar quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner.jar
Warning: The option '-H:+AllowFoldMethods' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.micrometer/micrometer-core/reflect-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:-UseServiceLoaderFeature' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:BuildOutputJSONFile=quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner-build-output-stats.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.netty/netty-transport/reflection-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
WARNING: Unknown module: org.graalvm.sdk specified to --add-exports
========================================================================================================================
GraalVM Native Image: Generating 'quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/8] Initializing... (12.3s @ 0.35GB)
Java version: 21.0.2+13, vendor version: GraalVM CE 21.0.2+13.1
Graal compiler: optimization level: 2, target machine: armv8-a
C compiler: cc (apple, arm64, 15.0.0)
Garbage collector: Serial GC (max heap size: 80% of RAM)
3 user-specific feature(s):
- com.oracle.svm.thirdparty.gson.GsonFeature
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
------------------------------------------------------------------------------------------------------------------------
4 experimental option(s) unlocked:
- '-H:+AllowFoldMethods' (origin(s): command line)
- '-H:BuildOutputJSONFile' (origin(s): command line)
- '-H:-UseServiceLoaderFeature' (origin(s): command line)
- '-H:ReflectionConfigurationResources' (origin(s): 'META-INF/native-image/io.micrometer/micrometer-core/native-image.properties' in 'file:///Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-native-image-source-jar/lib/io.micrometer.micrometer-core-1.11.1.jar', 'META-INF/native-image/io.netty/netty-transport/native-image.properties' in 'file:///Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-native-image-source-jar/lib/io.netty.netty-transport-4.1.100.Final.jar')
------------------------------------------------------------------------------------------------------------------------
Build resources:
- 26.49GB of memory (41.4% of 64.00GB system memory, determined at start)
- 10 thread(s) (100.0% of 10 available processor(s), determined at start)
java.lang.IllegalAccessError: class io.quarkus.runner.Feature (in unnamed module @0x6ab50d1c) cannot access class org.graalvm.nativeimage.impl.RuntimeClassInitializationSupport (in module org.graalvm.nativeimage) because module org.graalvm.nativeimage does not export org.graalvm.nativeimage.impl to unnamed module @0x6ab50d1c
at io.quarkus.runner.Feature.runtimeReinitializedClasses(Unknown Source)
at io.quarkus.runner.Feature.beforeAnalysis(Unknown Source)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$9(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:90)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:592)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)
java.lang.IllegalAccessError: class io.quarkus.runner.Feature (in unnamed module @0x6ab50d1c) cannot access class org.graalvm.nativeimage.impl.RuntimeClassInitializationSupport (in module org.graalvm.nativeimage) because module org.graalvm.nativeimage does not export org.graalvm.nativeimage.impl to unnamed module @0x6ab50d1c
at io.quarkus.runner.Feature.runtimeReinitializedClasses(Unknown Source)
at io.quarkus.runner.Feature.beforeAnalysis(Unknown Source)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$9(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:90)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:592)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)
java.lang.IllegalAccessError: class io.quarkus.runner.Feature (in unnamed module @0x6ab50d1c) cannot access class org.graalvm.nativeimage.impl.RuntimeClassInitializationSupport (in module org.graalvm.nativeimage) because module org.graalvm.nativeimage does not export org.graalvm.nativeimage.impl to unnamed module @0x6ab50d1c
at io.quarkus.runner.Feature.runtimeReinitializedClasses(Unknown Source)
at io.quarkus.runner.Feature.beforeAnalysis(Unknown Source)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$9(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:90)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:592)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)
java.lang.IllegalAccessError: class io.quarkus.runner.Feature (in unnamed module @0x6ab50d1c) cannot access class org.graalvm.nativeimage.impl.RuntimeClassInitializationSupport (in module org.graalvm.nativeimage) because module org.graalvm.nativeimage does not export org.graalvm.nativeimage.impl to unnamed module @0x6ab50d1c
at io.quarkus.runner.Feature.runtimeReinitializedClasses(Unknown Source)
at io.quarkus.runner.Feature.beforeAnalysis(Unknown Source)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$9(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:90)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:773)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:592)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)
[2/8] Performing analysis... [16:04:11,601 WARN [io.net.res.dns.DnsServerAddressStreamProviders] Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS. Check whether you have a dependency on 'io.netty:netty-resolver-dns-native-macos'
*********] (25.8s @ 2.03GB)
17,190 reachable types (69.7% of 24,678 total)
33,054 reachable fields (70.9% of 46,590 total)
122,784 reachable methods (63.3% of 193,960 total)
7,173 types, 871 fields, and 40,747 methods registered for reflection
61 types, 61 fields, and 55 methods registered for JNI access
5 native libraries: -framework CoreServices, -framework Foundation, dl, pthread, z
[3/8] Building universe... (3.9s @ 2.32GB)
[4/8] Parsing methods... [**] (2.7s @ 2.95GB)
[5/8] Inlining methods... [***] (1.2s @ 2.55GB)
[6/8] Compiling methods... [****] (16.7s @ 3.16GB)
[7/8] Layouting methods... [***] (5.4s @ 2.30GB)
[8/8] Creating image... [***] (8.0s @ 3.78GB)
40.02MB (46.83%) for code area: 89,175 compilation units
44.47MB (52.04%) for image heap: 447,396 objects and 56 resources
992.39kB ( 1.13%) for other data
85.46MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
11.78MB java.base 13.50MB byte[] for code metadata
4.76MB svm.jar (Native Image) 6.85MB byte[] for java.lang.String
2.87MB io.fabric8.kubernetes-model-core-6.7.2.jar 4.38MB java.lang.Class
1.80MB c.f.jackson.core.jackson-databind-2.15.2.jar 4.06MB java.lang.String
1.37MB q.3.6-SNAPSHOT-runner.jar 3.52MB byte[] for reflection metadata
1.24MB io.fabric8.openshift-model-6.7.2.jar 1.78MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
1.10MB modified-io.vertx.vertx-core-4.4.6.jar 1.44MB com.oracle.svm.core.hub.DynamicHubCompanion
1.08MB org.bouncycastle.bcprov-jdk18on-1.74.jar 860.73kB byte[] for general heap data
562.08kB io.fabric8.openshift-model-operator-6.7.2.jar 846.55kB java.lang.String[]
561.55kB com.fasterxml.jackson.core.jackson-core-2.15.2.jar 754.49kB java.lang.Object[]
12.49MB for 134 more packages 6.53MB for 3534 more object types
------------------------------------------------------------------------------------------------------------------------
Recommendations:
INIT: Adopt '--strict-image-heap' to prepare for the next GraalVM release.
HEAP: Set max heap for improved and more predictable memory usage.
CPU: Enable more CPU features with '-march=native' for improved performance.
------------------------------------------------------------------------------------------------------------------------
6.5s (8.3% of total time) in 103 GCs | Peak RSS: 6.32GB | CPU load: 5.44
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner (executable)
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner-build-output-stats.json (build_info)
========================================================================================================================
Finished generating 'quarkus-operator-sdk-samples-exposedapp-6.3.6-SNAPSHOT-runner' in 1m 17s.3.7.3:
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GRAALVM 23.1 JDK 21.0.2+13-jvmci-23.1-b30
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /Users/claprun/.sdkman/candidates/java/21.0.2-graalce/bin/native-image -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dlogging.initial-configurator.min-level=500 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=FR -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:+UnlockExperimentalVMOptions -H:BuildOutputJSONFile=quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner-build-output-stats.json -H:-UnlockExperimentalVMOptions --strict-image-heap -H:+UnlockExperimentalVMOptions -H:+AllowFoldMethods -H:-UnlockExperimentalVMOptions -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+UnlockExperimentalVMOptions -H:+ReportExceptionStackTraces -H:-UnlockExperimentalVMOptions -H:-AddAllCharsets --enable-url-protocols=http,https --enable-monitoring=heapdump -H:+UnlockExperimentalVMOptions -H:-UseServiceLoaderFeature -H:-UnlockExperimentalVMOptions -J--add-exports=org.graalvm.nativeimage/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner -jar quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner.jar
========================================================================================================================
GraalVM Native Image: Generating 'quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/8] Initializing... (8.0s @ 0.43GB)
Java version: 21.0.2+13, vendor version: GraalVM CE 21.0.2+13.1
Graal compiler: optimization level: 2, target machine: armv8-a
C compiler: cc (apple, arm64, 15.0.0)
Garbage collector: Serial GC (max heap size: 80% of RAM)
3 user-specific feature(s):
- com.oracle.svm.thirdparty.gson.GsonFeature
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
------------------------------------------------------------------------------------------------------------------------
3 experimental option(s) unlocked:
- '-H:+AllowFoldMethods' (origin(s): command line)
- '-H:BuildOutputJSONFile' (origin(s): command line)
- '-H:-UseServiceLoaderFeature' (origin(s): command line)
------------------------------------------------------------------------------------------------------------------------
Build resources:
- 26.49GB of memory (41.4% of 64.00GB system memory, determined at start)
- 10 thread(s) (100.0% of 10 available processor(s), determined at start)
[2/8] Performing analysis... [16:21:48,585 WARN [io.net.res.dns.DnsServerAddressStreamProviders] Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS. Check whether you have a dependency on 'io.netty:netty-resolver-dns-native-macos'
*****] (191.1s @ 8.51GB)
34,517 reachable types (94.8% of 36,416 total)
73,068 reachable fields (84.1% of 86,892 total)
339,522 reachable methods (86.5% of 392,604 total)
24,339 types, 51,922 fields, and 216,678 methods registered for reflection
61 types, 61 fields, and 55 methods registered for JNI access
5 native libraries: -framework CoreServices, -framework Foundation, dl, pthread, z
[3/8] Building universe... (7.2s @ 11.65GB)
[4/8] Parsing methods... [***] (6.5s @ 7.95GB)
[5/8] Inlining methods... [***] (2.8s @ 9.26GB)
[6/8] Compiling methods... [******] (39.5s @ 7.39GB)
[7/8] Layouting methods... [****] (14.9s @ 5.84GB)
[8/8] Creating image... [*****] (23.8s @ 5.06GB)
120.43MB (46.23%) for code area: 303,936 compilation units
137.73MB (52.88%) for image heap: 999,594 objects and 57 resources
2.33MB ( 0.89%) for other data
260.49MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
17.00MB io.fabric8.openshift-model-operator-6.10.0.jar 40.09MB byte[] for code metadata
14.62MB svm.jar (Native Image) 24.68MB byte[] for java.lang.String
13.48MB io.fabric8.openshift-model-6.10.0.jar 19.74MB java.lang.Class
12.10MB java.base 17.11MB byte[] for reflection metadata
8.61MB io.fabric8.kubernetes-model-core-6.10.0.jar 9.87MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
4.47MB io.fabric8.openshift-model-monitoring-6.10.0.jar 8.92MB java.lang.String
4.44MB io.fabric8.openshift-model-hive-6.10.0.jar 2.90MB com.oracle.svm.core.hub.DynamicHubCompanion
2.78MB i.f.k.10.0.jar 2.38MB java.lang.Object[]
2.65MB io.fabric8.kubernetes-model-gatewayapi-6.10.0.jar 1.85MB java.lang.String[]
2.59MB io.fabric8.openshift-model-operatorhub-6.10.0.jar 1.48MB byte[] for general heap data
36.07MB for 137 more packages 8.72MB for 3626 more object types
------------------------------------------------------------------------------------------------------------------------
Recommendations:
HEAP: Set max heap for improved and more predictable memory usage.
CPU: Enable more CPU features with '-march=native' for improved performance.
------------------------------------------------------------------------------------------------------------------------
22.9s (7.8% of total time) in 95 GCs | Peak RSS: 16.90GB | CPU load: 6.61
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner (executable)
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner-build-output-stats.json
========================================================================================================================
Finished generating 'quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner' in 4m 55s.So while it's not completely comparing apples to apples, things do appear significantly worse in 3.7.3, with the openshift model dominating the code areas, as opposed to java.base in 3.2. Types registered for reflection are significantly worse in 3.7.3 as well.
metacosm
commented
9 months ago With 3.5.0:
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GRAALVM 23.1 JDK 21.2
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /Users/claprun/.sdkman/candidates/java/21.0.2-graalce/bin/native-image -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dlogging.initial-configurator.min-level=500 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=FR -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:+UnlockExperimentalVMOptions -H:BuildOutputJSONFile=quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner-build-output-stats.json -H:-UnlockExperimentalVMOptions --strict-image-heap -H:+UnlockExperimentalVMOptions -H:+AllowFoldMethods -H:-UnlockExperimentalVMOptions -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+UnlockExperimentalVMOptions -H:+ReportExceptionStackTraces -H:-UnlockExperimentalVMOptions -H:-AddAllCharsets --enable-url-protocols=http,https --enable-monitoring=heapdump -H:+UnlockExperimentalVMOptions -H:-UseServiceLoaderFeature -H:-UnlockExperimentalVMOptions -J--add-exports=org.graalvm.nativeimage/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner -jar quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner.jar
[INFO] [io.quarkiverse.operatorsdk.bundle.deployment.BundleProcessor] Generating CSV for 'quarkus-operator-sdk-samples-exposedapp' controller -> /Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/bundle/quarkus-operator-sdk-samples-exposedapp/manifests/quarkus-operator-sdk-samples-exposedapp.clusterserviceversion.yaml
[INFO] [io.quarkiverse.operatorsdk.bundle.deployment.BundleProcessor] Generating annotations for 'quarkus-operator-sdk-samples-exposedapp' controller -> /Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/bundle/quarkus-operator-sdk-samples-exposedapp/metadata/annotations.yaml
[INFO] [io.quarkiverse.operatorsdk.bundle.deployment.BundleProcessor] Generating bundle for 'quarkus-operator-sdk-samples-exposedapp' controller -> /Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/bundle/quarkus-operator-sdk-samples-exposedapp/bundle.Dockerfile
[INFO] [io.quarkiverse.operatorsdk.bundle.deployment.BundleProcessor] Generating Custom Resource Definition for 'quarkus-operator-sdk-samples-exposedapp' controller -> /Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/bundle/quarkus-operator-sdk-samples-exposedapp/manifests/exposedapps.halkyon.io-v1.crd.yml
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.micrometer/micrometer-core/reflect-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.netty/netty-transport/reflection-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
========================================================================================================================
GraalVM Native Image: Generating 'quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/8] Initializing... (8.0s @ 0.29GB)
Java version: 21.0.2+13, vendor version: GraalVM CE 21.0.2+13.1
Graal compiler: optimization level: 2, target machine: armv8-a
C compiler: cc (apple, arm64, 15.0.0)
Garbage collector: Serial GC (max heap size: 80% of RAM)
3 user-specific feature(s):
- com.oracle.svm.thirdparty.gson.GsonFeature
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
------------------------------------------------------------------------------------------------------------------------
4 experimental option(s) unlocked:
- '-H:+AllowFoldMethods' (origin(s): command line)
- '-H:BuildOutputJSONFile' (origin(s): command line)
- '-H:-UseServiceLoaderFeature' (origin(s): command line)
- '-H:ReflectionConfigurationResources' (origin(s): 'META-INF/native-image/io.micrometer/micrometer-core/native-image.properties' in 'file:///Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/lib/io.micrometer.micrometer-core-1.11.1.jar', 'META-INF/native-image/io.netty/netty-transport/native-image.properties' in 'file:///Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/lib/io.netty.netty-transport-4.1.100.Final.jar')
------------------------------------------------------------------------------------------------------------------------
Build resources:
- 13.88GB of memory (21.7% of 64.00GB system memory, determined at start)
- 10 thread(s) (100.0% of 10 available processor(s), determined at start)
[2/8] Performing analysis... [17:00:23,192 WARN [io.net.res.dns.DnsServerAddressStreamProviders] Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS. Check whether you have a dependency on 'io.netty:netty-resolver-dns-native-macos'
**********] (25.0s @ 3.12GB)
17,235 reachable types (74.2% of 23,226 total)
33,982 reachable fields (71.3% of 47,683 total)
126,631 reachable methods (67.5% of 187,468 total)
7,120 types, 1,206 fields, and 43,751 methods registered for reflection
61 types, 61 fields, and 55 methods registered for JNI access
5 native libraries: -framework CoreServices, -framework Foundation, dl, pthread, z
[3/8] Building universe... (4.9s @ 4.50GB)
[4/8] Parsing methods... [**] (2.6s @ 3.96GB)
[5/8] Inlining methods... [***] (1.2s @ 3.60GB)
[6/8] Compiling methods... [****] (16.9s @ 2.99GB)
[7/8] Layouting methods... [***] (5.6s @ 4.28GB)
[8/8] Creating image... [***] (9.0s @ 2.15GB)
41.52MB (46.91%) for code area: 93,175 compilation units
46.00MB (51.97%) for image heap: 458,097 objects and 57 resources
1014.56kB ( 1.12%) for other data
88.51MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
12.07MB java.base 13.96MB byte[] for code metadata
4.92MB svm.jar (Native Image) 6.96MB byte[] for java.lang.String
3.17MB io.fabric8.kubernetes-model-core-6.8.1.jar 4.55MB java.lang.Class
1.80MB c.f.jackson.core.jackson-databind-2.15.2.jar 4.16MB java.lang.String
1.39MB q.6.4-SNAPSHOT-runner.jar 3.75MB byte[] for reflection metadata
1.34MB io.fabric8.openshift-model-6.8.1.jar 1.93MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
1.15MB org.bouncycastle.bcprov-jdk18on-1.76.jar 1.45MB com.oracle.svm.core.hub.DynamicHubCompanion
1.11MB modified-io.vertx.vertx-core-4.4.5.jar 1.15MB byte[] for general heap data
630.71kB io.fabric8.openshift-model-operator-6.8.1.jar 865.65kB java.lang.String[]
561.52kB com.fasterxml.jackson.core.jackson-core-2.15.2.jar 765.25kB java.lang.Object[]
12.98MB for 137 more packages 6.51MB for 3606 more object types
------------------------------------------------------------------------------------------------------------------------
Recommendations:
HEAP: Set max heap for improved and more predictable memory usage.
CPU: Enable more CPU features with '-march=native' for improved performance.
------------------------------------------------------------------------------------------------------------------------
5.9s (7.9% of total time) in 62 GCs | Peak RSS: 7.52GB | CPU load: 5.32
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner (executable)
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner-build-output-stats.json (build_info)
========================================================================================================================
Finished generating 'quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner' in 1m 14s.With 3.6.0:
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GRAALVM 23.1 JDK 21.0.2+13-jvmci-23.1-b30
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /Users/claprun/.sdkman/candidates/java/21.0.2-graalce/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dlogging.initial-configurator.min-level=500 -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Duser.language=en -J-Duser.country=FR -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:+UnlockExperimentalVMOptions -H:BuildOutputJSONFile=quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner-build-output-stats.json -H:-UnlockExperimentalVMOptions --strict-image-heap -H:+UnlockExperimentalVMOptions -H:+AllowFoldMethods -H:-UnlockExperimentalVMOptions -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+UnlockExperimentalVMOptions -H:+ReportExceptionStackTraces -H:-UnlockExperimentalVMOptions -H:-AddAllCharsets --enable-url-protocols=http,https --enable-monitoring=heapdump -H:+UnlockExperimentalVMOptions -H:-UseServiceLoaderFeature -H:-UnlockExperimentalVMOptions -J--add-exports=org.graalvm.nativeimage/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner -jar quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner.jar
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.micrometer/micrometer-core/reflect-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.netty/netty-transport/reflection-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
========================================================================================================================
GraalVM Native Image: Generating 'quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/8] Initializing... (9.1s @ 0.33GB)
Java version: 21.0.2+13, vendor version: GraalVM CE 21.0.2+13.1
Graal compiler: optimization level: 2, target machine: armv8-a
C compiler: cc (apple, arm64, 15.0.0)
Garbage collector: Serial GC (max heap size: 80% of RAM)
3 user-specific feature(s):
- com.oracle.svm.thirdparty.gson.GsonFeature
- io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
- io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
------------------------------------------------------------------------------------------------------------------------
4 experimental option(s) unlocked:
- '-H:+AllowFoldMethods' (origin(s): command line)
- '-H:BuildOutputJSONFile' (origin(s): command line)
- '-H:-UseServiceLoaderFeature' (origin(s): command line)
- '-H:ReflectionConfigurationResources' (origin(s): 'META-INF/native-image/io.micrometer/micrometer-core/native-image.properties' in 'file:///Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/lib/io.micrometer.micrometer-core-1.11.5.jar', 'META-INF/native-image/io.netty/netty-transport/native-image.properties' in 'file:///Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/lib/io.netty.netty-transport-4.1.100.Final.jar')
------------------------------------------------------------------------------------------------------------------------
Build resources:
- 11.40GB of memory (17.8% of 64.00GB system memory, determined at start)
- 10 thread(s) (100.0% of 10 available processor(s), determined at start)
[2/8] Performing analysis... [17:08:25,416 WARN [io.net.res.dns.DnsServerAddressStreamProviders] Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS. Check whether you have a dependency on 'io.netty:netty-resolver-dns-native-macos'
*****] (156.8s @ 6.09GB)
34,253 reachable types (94.7% of 36,152 total)
72,364 reachable fields (84.1% of 86,072 total)
336,541 reachable methods (86.5% of 389,146 total)
24,179 types, 51,594 fields, and 215,026 methods registered for reflection
61 types, 61 fields, and 55 methods registered for JNI access
5 native libraries: -framework CoreServices, -framework Foundation, dl, pthread, z
[3/8] Building universe... (10.9s @ 6.70GB)
[4/8] Parsing methods... [**] (4.5s @ 8.24GB)
[5/8] Inlining methods... [***] (2.2s @ 7.43GB)
[6/8] Compiling methods... [******] (37.5s @ 4.05GB)
[7/8] Layouting methods... [****] (13.6s @ 6.02GB)
[8/8] Creating image... [*****] (21.7s @ 7.56GB)
119.17MB (46.16%) for code area: 301,358 compilation units
136.66MB (52.94%) for image heap: 992,513 objects and 57 resources
2.32MB ( 0.90%) for other data
258.14MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
17.00MB io.fabric8.openshift-model-operator-6.9.2.jar 39.68MB byte[] for code metadata
14.52MB svm.jar (Native Image) 24.52MB byte[] for java.lang.String
13.48MB io.fabric8.openshift-model-6.9.2.jar 19.63MB java.lang.Class
12.08MB java.base 16.97MB byte[] for reflection metadata
8.49MB io.fabric8.kubernetes-model-core-6.9.2.jar 9.80MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
4.47MB io.fabric8.openshift-model-monitoring-6.9.2.jar 8.85MB java.lang.String
4.44MB io.fabric8.openshift-model-hive-6.9.2.jar 2.87MB com.oracle.svm.core.hub.DynamicHubCompanion
2.78MB i.f.k.9.2.jar 2.37MB java.lang.Object[]
2.59MB io.fabric8.openshift-model-operatorhub-6.9.2.jar 1.84MB java.lang.String[]
2.32MB io.fabric8.kubernetes-model-flowcontrol-6.9.2.jar 1.47MB byte[] for general heap data
35.39MB for 137 more packages 8.65MB for 3624 more object types
------------------------------------------------------------------------------------------------------------------------
Recommendations:
HEAP: Set max heap for improved and more predictable memory usage.
CPU: Enable more CPU features with '-march=native' for improved performance.
------------------------------------------------------------------------------------------------------------------------
24.7s (9.6% of total time) in 126 GCs | Peak RSS: 12.17GB | CPU load: 6.19
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner (executable)
/Volumes/CaseSensitiveDev/quarkus-operator-sdk/samples/exposedapp/target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner-build-output-stats.json (build_info)
========================================================================================================================
Finished generating 'quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner' in 4m 17s.
galderz
commented
9 months ago Thanks for the info Chris. Indeed the operator presents the same symptoms as seen in Vincent's application. We'll look into. To replicate:
git clone https://github.com/quarkiverse/quarkus-operator-sdk
cd quarkus-operator-sdk
mvn install -DskipTests
cd samples/exposedapp
mvn package -DskipTests -Dnative -Dnative.surefire.skipp.s. @metacosm Why doesn't the sample operator app honor -DskipTests? I had to run with mvn debug to discover that -skipTests can only be overriden with -Dnative.surefire.skip?
metacosm
commented
9 months ago p.s. @metacosm Why doesn't the sample operator app honor
-DskipTests? I had to run with mvn debug to discover that -skipTestscan only be overriden with-Dnative.surefire.skip?
That looks like a bug, will look into it
metacosm
commented
9 months ago More information, I ran native builds with reports and here's the difference of packages between 3.5 and 3.7:
--- reports-3.5/used_packages_quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner_20240215_095352.txt 2024-02-15 09:53:52
+++ reports-3.7/used_packages_quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner_20240215_102428.txt 2024-02-15 10:24:28
@@ -113,6 +113,8 @@
com.sun.management
com.sun.management.internal
com.sun.naming.internal
+com.sun.org.apache.xerces.internal.jaxp.datatype
+com.sun.org.apache.xerces.internal.util
com.sun.proxy.jdk.proxy1
com.sun.proxy.jdk.proxy2
com.sun.proxy.jdk.proxy3
@@ -149,10 +151,11 @@
io.fabric8.kubernetes.api.model.events.v1
io.fabric8.kubernetes.api.model.events.v1beta1
io.fabric8.kubernetes.api.model.extensions
-io.fabric8.kubernetes.api.model.flowcontrol.v1alpha1
+io.fabric8.kubernetes.api.model.flowcontrol.v1
io.fabric8.kubernetes.api.model.flowcontrol.v1beta1
io.fabric8.kubernetes.api.model.flowcontrol.v1beta2
io.fabric8.kubernetes.api.model.flowcontrol.v1beta3
+io.fabric8.kubernetes.api.model.gatewayapi.v1
io.fabric8.kubernetes.api.model.gatewayapi.v1alpha2
io.fabric8.kubernetes.api.model.gatewayapi.v1beta1
io.fabric8.kubernetes.api.model.internal.apiserver.v1alpha1
@@ -171,6 +174,7 @@
io.fabric8.kubernetes.api.model.scheduling.v1
io.fabric8.kubernetes.api.model.scheduling.v1beta1
io.fabric8.kubernetes.api.model.storage
+io.fabric8.kubernetes.api.model.storage.v1alpha1
io.fabric8.kubernetes.api.model.storage.v1beta1
io.fabric8.kubernetes.api.model.version
io.fabric8.kubernetes.client
@@ -207,11 +211,14 @@
io.fabric8.openshift.api.model.console.v1
io.fabric8.openshift.api.model.console.v1alpha1
io.fabric8.openshift.api.model.hive.agent.v1
+io.fabric8.openshift.api.model.hive.alibabacloud.v1
io.fabric8.openshift.api.model.hive.aws.v1
io.fabric8.openshift.api.model.hive.azure.v1
io.fabric8.openshift.api.model.hive.baremetal.v1
io.fabric8.openshift.api.model.hive.gcp.v1
io.fabric8.openshift.api.model.hive.ibmcloud.v1
+io.fabric8.openshift.api.model.hive.metricsconfig.v1
+io.fabric8.openshift.api.model.hive.none.v1
io.fabric8.openshift.api.model.hive.openstack.v1
io.fabric8.openshift.api.model.hive.ovirt.v1
io.fabric8.openshift.api.model.hive.v1
@@ -239,6 +246,7 @@
io.fabric8.openshift.api.model.miscellaneous.cncf.cni.v1
io.fabric8.openshift.api.model.miscellaneous.imageregistry.operator.v1
io.fabric8.openshift.api.model.miscellaneous.metal3.v1alpha1
+io.fabric8.openshift.api.model.miscellaneous.metal3.v1beta1
io.fabric8.openshift.api.model.miscellaneous.network.operator.v1
io.fabric8.openshift.api.model.monitoring.v1
io.fabric8.openshift.api.model.monitoring.v1alpha1
@@ -325,6 +333,7 @@
io.netty.util.internal
io.netty.util.internal.logging
io.netty.util.internal.shaded.org.jctools.queues
+io.netty.util.internal.shaded.org.jctools.queues.atomic
io.netty.util.internal.shaded.org.jctools.util
io.quarkiverse.operatorsdk.common
io.quarkiverse.operatorsdk.runtime
@@ -368,6 +377,7 @@
io.quarkus.runtime.util
io.quarkus.smallrye.context.runtime
io.quarkus.smallrye.health.runtime
+io.quarkus.vertx
io.quarkus.vertx.core.runtime
io.quarkus.vertx.core.runtime.config
io.quarkus.vertx.core.runtime.context
@@ -387,10 +397,10 @@
io.smallrye.common.classloader
io.smallrye.common.constraint
io.smallrye.common.expression
-io.smallrye.common.function
io.smallrye.common.ref
io.smallrye.common.vertx
io.smallrye.config
+io.smallrye.config._private
io.smallrye.config.common
io.smallrye.config.common.utils
io.smallrye.config.inject
@@ -437,6 +447,7 @@
io.vertx.core.impl.verticle
io.vertx.core.json
io.vertx.core.json.impl
+io.vertx.core.json.jackson
io.vertx.core.logging
io.vertx.core.metrics
io.vertx.core.net
@@ -454,6 +465,7 @@
io.vertx.core.spi.transport
io.vertx.core.streams
io.vertx.core.streams.impl
+io.vertx.core.tracing
io.vertx.ext.web
io.vertx.ext.web.client
io.vertx.ext.web.handler
@@ -523,6 +535,8 @@
javax.security.auth.callback
javax.security.auth.x500
javax.tools
+javax.xml.datatype
+javax.xml.namespace
jdk.internal.access
jdk.internal.event
jdk.internal.icu.impl
@@ -550,6 +564,7 @@
jdk.vm.ci.aarch64
jdk.vm.ci.code
jdk.vm.ci.meta
+jdk.xml.internal
org.HdrHistogram
org.bouncycastle.asn1
org.bouncycastle.asn1.cms
@@ -564,6 +579,7 @@
org.bouncycastle.crypto.constraints
org.bouncycastle.crypto.digests
org.bouncycastle.crypto.engines
+org.bouncycastle.crypto.modes
org.bouncycastle.crypto.params
org.bouncycastle.jcajce.provider.symmetric.util
org.bouncycastle.jcajce.util
@@ -661,6 +677,7 @@
org.snakeyaml.engine.v2.tokens
org.wildfly.common
org.wildfly.common._private
+org.wildfly.common.cpu
org.wildfly.common.lock
org.wildfly.common.math
org.wildfly.common.net
@@ -688,6 +705,7 @@
sun.management
sun.misc
sun.net
+sun.net.dns
sun.net.ext
sun.net.spi
sun.net.utilLooks like for some reason, we're now getting io.quarkus.vertx and XML support, which wasn't previously the case. That said, that doesn't explain why the OpenShift model jars are now so much larger.
metacosm
commented
9 months ago Upon further investigation it looks like Graal now looks at Builder classes when it previously wasn't the case so, for example, for the AdmissionReview class, the following methods are now considered by Graal, when they previously were not:
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_0777984050acf10da1493a6170fecab4d6751fcb():AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_0fd11b86103ae5deac94f17cd7ebc9ab0f80db23(AdmissionReview):AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_1b146ff3d728cebdeee6ce3b7a16914bc1fa167a(AdmissionReviewFluent, AdmissionReview):AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_60c83db85be99307ac1cfa5e02331b73895992f2(AdmissionReviewFluent):AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_7d86b4847991d0ea7bc237011e756b6bbf79ade5(AdmissionReview):AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_8407036dc50873034984857788b3f548e6463c6d():AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_b76e0fbe8c4fc902f7f35602d471fc03f7a1b049(AdmissionReviewFluent, AdmissionReview):AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewBuilder_constructor_d1c32c1d0681c1b2f19acb7ac28dac86d0b61674(AdmissionReviewFluent):AdmissionReviewBuilder
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewFluent$ResponseNested_constructor_259ca1fdf585b79b107e52e1c9e93f64803f3dd1(AdmissionReviewFluent, AdmissionResponse):AdmissionReviewFluent$ResponseNested
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewFluent$ResponseNested_constructor_9ede732b5de68aeabe21301372523139b271aeb0(AdmissionReviewFluent, AdmissionResponse):AdmissionReviewFluent$ResponseNested
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewFluent_constructor_8e18a7129d7cfce282cc44631a106586b9f50160():AdmissionReviewFluent
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewFluent_constructor_a3286f0b1cb894d801b61242fb3ee01a918d9ac7():AdmissionReviewFluent
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewFluent_constructor_b36cf7bb0e633398005c903e018b111dd2f934d5(AdmissionReview):AdmissionReviewFluent
com.oracle.svm.core.code.FactoryMethodHolder.AdmissionReviewFluent_constructor_c44ba2db71b9766138f88aac8a2d5f191200375d(AdmissionReview):AdmissionReviewFluentSo with around as many methods added for each type, that certainly explains the size difference. The question, now, is why are the builders considered when they previously weren't.
galderz
commented
9 months ago Thanks @metacosm for your comments. We have replicated the issue now on our side but the amount of data to get through is considerable. The only thing I've been able to observe so far is that the blow out is already present in the Quarkus operator, which makes things easier on our side. We'll reply when we have a good understanding of the causes.
metacosm
commented
9 months ago I'm looking into the reason why the builders are now considered. I think that we'll be in good shape if we can figure this out and address this particular issue first.
manusa
commented
9 months ago For some extra insights. I'm working on a demo project to showcase how to work on OpenShift with just the vanilla kubernetes-client-extension: https://github.com/marcnuri-demo/quarkus-kubernetes-client-models/blob/c4123613e94646018d2d4e6ade36267dde056d8a/
Compiling a native image in 3.7.3, produces the following output:
159.55MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
14.53MB io.fabric8.openshift-model-6.10.0.jar 25.02MB byte[] for code metadata
12.94MB java.base 13.04MB byte[] for java.lang.String
9.29MB io.fabric8.kubernetes-model-core-6.10.0.jar 8.99MB byte[] for reflection metadata
8.82MB svm.jar (Native Image) 8.19MB java.lang.Class
2.97MB i.f.k.10.0.jar 5.88MB java.lang.String
2.85MB io.fabric8.kubernetes-model-gatewayapi-6.10.0.jar 5.15MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
2.48MB io.fabric8.kubernetes-model-flowcontrol-6.10.0.jar 1.84MB com.oracle.svm.core.hub.DynamicHubCompanion
1.97MB c.f.jackson.core.jackson-databind-2.16.1.jar 1.36MB java.lang.Object[]
1.76MB io.fabric8.kubernetes-model-autoscaling-6.10.0.jar 1.24MB byte[] for general heap data
1.75MB i.f.kubernetes-model-apiextensions-6.10.0.jar 1.21MB java.lang.String[]
19.24MB for 117 more packages 7.27MB for 3416 more object typesWhile the same project in 3.5.3:
76.36MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
12.90MB java.base 11.76MB byte[] for code metadata
4.06MB svm.jar (Native Image) 5.93MB byte[] for java.lang.String
3.43MB io.fabric8.kubernetes-model-core-6.8.1.jar 3.95MB java.lang.Class
1.93MB c.f.jackson.core.jackson-databind-2.15.3.jar 3.65MB java.lang.String
1.26MB io.fabric8.openshift-model-6.8.1.jar 2.58MB byte[] for reflection metadata
1.17MB modified-io.vertx.vertx-core-4.4.6.jar 1.27MB com.oracle.svm.core.hub.DynamicHubCompanion
823.45kB q.0.0-SNAPSHOT-runner.jar 1.21MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
582.35kB com.fasterxml.jackson.core.jackson-core-2.15.3.jar 1.12MB byte[] for general heap data
575.69kB org.yaml.snakeyaml-2.1.jar 754.34kB java.lang.String[]
510.70kB org.snakeyaml.snakeyaml-engine-2.6.jar 672.46kB java.lang.Object[]
9.09MB for 117 more packages 6.23MB for 3391 more object typesSo the increase in size is also extremely visible here. Since this project is much smaller, maybe it's easier to use it to analyze why now classes such as builders are now being considered.
metacosm
commented
9 months ago I've pinpointed the culprit: we added an edit method to all model objects in https://github.com/fabric8io/kubernetes-client/commit/a3efaa3d773de02f42ca16255d66155b8bc97118. Prior to that commit, a native build would report:
more target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/reports/used_methods_quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner_20240215_164459.txt |
grep Builder | wc -l
6398With that commit, we get:
more target/quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-native-image-source-jar/reports/used_methods_quarkus-operator-sdk-samples-exposedapp-6.6.4-SNAPSHOT-runner_20240215_170845.txt |
grep Builder | wc -l
44403Looking at the changelog and the symptoms, it made sense that this change was the culprit but I wanted to have the numbers to back that claim, first… 😄 Now, the question is how to fix it.
galderz
commented
9 months ago Ok, I guess that commits binds the types to their builders and then combined with:
Collection<ClassInfo> visitableBuilderImpls = fullIndex.getAllKnownImplementors(VISITABLE_BUILDER);
...
populateReflectionRegistrationLists(visitableBuilderImpls, watchedClasses, ignoreJsonDeserialization,
withoutFieldsRegistration,
withFieldsRegistration);All those builders found are registered for reflection.
Builders create types. By having types create builders that creates a bidirectional dependency which I'm not sure it's really needed. You can still have a builder that can be created out of a type, but done outside of the type and that breaks the cycle. Something for @manusa et al to chew on.
A workaround at the Quarkus level might be to only register those builders for reflection that are considered reachable by the points-to analysis. The concept is explained here and some parts of Quarkus use this, so there are examples in the code base, but I'm not 100% sure it would solve the issue. It would depend on how different the set of reachable builders is vs the set of builder implementations in the jandex index.
maxandersen
commented
9 months ago great find @metacosm - so the cause is that the read view of class hierarchy via this edit method got connected to all the builders in one fell swoop.
That explains it but definitely very counter to our past efforts of reducing/decoupling the client.
Could a visitor pattern solve this instead?
And yes as @galderz mention only register those that are accessible via the points to analysis could help here !
vsevel
commented
9 months ago I was working on providing the resources asked by @zakkak in https://github.com/quarkusio/quarkus/issues/38683#issuecomment-1935486283 I would not be able to make it public (only through a support case). but it feels that you made a lot of progress in the analysis. do you still need something from me?
metacosm
commented
9 months ago @vsevel you probably don't need to anymore… though it might be a good idea to keep the work you did around just in case we need it later on! 😉
galderz
commented
9 months ago To follow up on ideas for a solution here:
An even better solution would be to avoid using reflection in the Kubernetes client.
Hibernate was in a not so dissimilar position 4 years ago, when pushing the boundaries of number of entities created was causing build times to increase in a big way (see Zulip chat). The refactoring https://github.com/fabric8io/kubernetes-client/commit/a3efaa3d773de02f42ca16255d66155b8bc97118 alone is not an issue. Combining that with the fact that builders get registered for reflection is what causes the blowout in build times...etc. You can test the effects of reflection registration with this small project I created at the time: reflection-bigbang
metacosm
commented
9 months ago To be honest, I'm not sure why we register the builders for reflection. We do need to register the other classes of the model because we need to be able to serialize / deserialize them. Of course, it would be better to only register what is actually needed but that's just not feasible because when we have no way of knowing which classes are actually used when a given model package is imported, so all of them need to be registered.
gsmet
commented
9 months ago It would be surprising if we needed to register the builders for reflection.
I would start by not doing that and see how it goes.
metacosm
commented
9 months ago Yeah, that's what I'm planning on exploring tomorrow.
Sanne
commented
9 months ago We do need to register the other classes of the model because we need to be able to serialize / deserialize them.
Just curious, is that actual Java serialization? It would really be nice to find a way to avoid registering such large models, especially as there's not even a guaranteed that they will ever be used.
In other extensions we worked hard to avoid reflection - it's not a strict rule as we're pragmatic about it (having some won't harm), but we do try to find alternative solutions when the number is significant.
manusa
commented
9 months ago Great job, @metacosm :raised_hands: :raised_hands: :raised_hands:
I can't really think of a case where builders would be accessed reflectively. Maybe @shawkins has more ideas about this.
especially as there's not even a guaranteed that they will ever be used
The problem is the way in which the Fabric8 client is designed, especially when dealing with the API server or resources that users might store locally.
For example, a user might try to deserialize a YAML file which contains a variety of resources by invoking kubernetesClient.load(yamlListInputStream). The deserializer tries to figure out the target types and deserialize it into object instances of these types.
Similar situations happen when interacting with the kube API server and retrieving objects, especially deeply nested types.
This doesn't play well with GraalVM and the way it traverses the code to discover the needed types at runtime.
If removing the builders from the list of reflectively accessed classes fixes the issue, I think we might be able to get out of the woods for now.
metacosm
commented
9 months ago Hmm, looks like this might not be as trivial as removing the builders from reflection since it seems they were added to solve https://github.com/quarkusio/quarkus/issues/27469. Will dig deeper.
metacosm
commented
9 months ago I have opened a PR in the fabric8 client to reduce reflective access but the issue is that the Builders are part of the type of resources that are being registered for reflection (they implement an interface that includes the associated builder as a generic parameter) so I suspect that they're being pulled transitively.
metacosm
commented
9 months ago I have also opened https://github.com/quarkusio/quarkus/pull/38823 to remove the explicit registration of builders but this doesn't help. I'm going to be on PTO all of next week, though…
zakkak
commented
8 months ago I suspect that they're being pulled transitively.
That's right. In Quarkus main JacksonProcessor is registering 7896 types because they belong in the type hierarchy of io.fabric8.kubernetes.api.model.ValidationSchema which is annotated with @JsonDeserialize
Note that these registrations are unconditional and register for reflection all the type methods and fields as well, which leads to the explosion we observe.
To confirm this I disabled such registrations with:
that saved ~50MB on its own with the reproducer in https://github.com/quarkusio/quarkus/issues/38683#issuecomment-1946275370
going one step further and applying https://github.com/quarkusio/quarkus/pull/38823 it saved another ~10MB. Which still results in a binary ~17MB bigger than in 3.5.
manusa
commented
8 months ago That's right. In Quarkus main JacksonProcessor is registering 7896 types because they belong in the type hierarchy of io.fabric8.kubernetes.api.model.ValidationSchema which is annotated with @JsonDeserialize
This is a great find. For me it was difficult to understand why now just by adding an edit method was pulling the builder types while before it wasn't even if the types aren't used.
This clearly explains why we're pulling everything in, regardless of that being needed at runtime or not. So #38823 shouldn't be applicable and might actually bring some corner-case scenarios.
AFAIK I think we can safely remove the ValidationSchema types, I'm not really aware of anyone actually using those (especially from a reflective perspective).
maxandersen
commented
8 months ago @zakkak any chance you havea similar output so we can compare the sizes on each of the model jars to the other runs?
zakkak
commented
8 months ago @zakkak any chance you havea similar output so we can compare the sizes on each of the model jars to the other runs?
[8/8] Creating image... [**] (4.0s @ 7.18GB)
36.26MB (50.59%) for code area: 73,146 compilation units
35.02MB (48.86%) for image heap: 398,582 objects and 10 resources
402.88kB ( 0.55%) for other data
71.68MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
12.83MB java.base 8.02MB byte[] for code metadata
4.02MB svm.jar (Native Image) 3.90MB java.lang.Class
3.28MB io.fabric8.kubernetes-model-core-6.8.1.jar 3.69MB byte[] for java.lang.String
1.91MB c.f.jackson.core.jackson-databind-2.15.3.jar 3.62MB java.lang.String
1.20MB io.fabric8.openshift-model-6.8.1.jar 3.32MB byte[] for general heap data
1.16MB modified-io.vertx.vertx-core-4.4.6.jar 2.58MB byte[] for reflection metadata
802.50kB q.0.0-SNAPSHOT-runner.jar 1.26MB com.oracle.svm.core.hub.DynamicHubCompanion
562.71kB com.fasterxml.jackson.core.jackson-core-2.15.3.jar 1.21MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
556.74kB org.yaml.snakeyaml-2.1.jar 749.27kB java.lang.String[]
495.56kB org.snakeyaml.snakeyaml-engine-2.6.jar 596.81kB java.util.HashMap$Node
8.92MB for 116 more packages 5.76MB for 3308 more object typesmain[8/8] Creating image... [***] (6.9s @ 4.76GB)
78.78MB (52.68%) for code area: 174,296 compilation units
70.37MB (47.06%) for image heap: 651,043 objects and 10 resources
404.81kB ( 0.26%) for other data
149.54MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
14.27MB io.fabric8.openshift-model-6.10.0.jar 16.53MB byte[] for code metadata
12.87MB java.base 9.47MB byte[] for java.lang.String
9.10MB io.fabric8.kubernetes-model-core-6.10.0.jar 8.98MB byte[] for reflection metadata
8.79MB svm.jar (Native Image) 8.14MB java.lang.Class
2.90MB i.f.k.10.0.jar 5.84MB java.lang.String
2.80MB io.fabric8.kubernetes-model-gatewayapi-6.10.0.jar 5.14MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
2.43MB io.fabric8.kubernetes-model-flowcontrol-6.10.0.jar 4.77MB byte[] for general heap data
1.95MB c.f.jackson.core.jackson-databind-2.16.1.jar 1.82MB com.oracle.svm.core.hub.DynamicHubCompanion
1.72MB i.f.kubernetes-model-apiextensions-6.10.0.jar 1.29MB java.lang.Object[]
1.71MB io.fabric8.kubernetes-model-autoscaling-6.10.0.jar 1.20MB java.lang.String[]
18.90MB for 116 more packages 6.53MB for 3331 more object typesmain patched with diff from https://github.com/quarkusio/quarkus/issues/38683#issuecomment-1952655934[8/8] Creating image... [**] (4.5s @ 5.79GB)
52.73MB (54.21%) for code area: 100,164 compilation units
44.14MB (45.38%) for image heap: 463,734 objects and 10 resources
402.77kB ( 0.40%) for other data
97.27MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 origins of code area: Top 10 object types in image heap:
12.86MB java.base 10.91MB byte[] for code metadata
6.71MB io.fabric8.openshift-model-6.10.0.jar 5.47MB byte[] for java.lang.String
5.46MB svm.jar (Native Image) 4.61MB java.lang.Class
4.30MB io.fabric8.kubernetes-model-core-6.10.0.jar 4.29MB java.lang.String
1.95MB c.f.jackson.core.jackson-databind-2.16.1.jar 3.76MB byte[] for general heap data
1.44MB i.f.k.10.0.jar 3.42MB byte[] for reflection metadata
1.31MB io.fabric8.kubernetes-model-gatewayapi-6.10.0.jar 1.59MB c.oracle.svm.core.reflect.SubstrateMethodAccessor
1.24MB io.fabric8.kubernetes-model-flowcontrol-6.10.0.jar 1.50MB com.oracle.svm.core.hub.DynamicHubCompanion
1.18MB modified-io.vertx.vertx-core-4.5.3.jar 880.38kB java.lang.String[]
997.95kB q.0.0-SNAPSHOT-runner.jar 679.41kB java.lang.Object[]
14.54MB for 116 more packages 6.06MB for 3318 more object types
Describe the bug
This was originally reported in https://quarkusio.zulipchat.com/#narrow/stream/187030-users/topic/performance.20decrease.20with.20graalvm.2023.2E1.2E2.20-quarkus.203.2E5.20to.203.2E6 by @vsevel
With Quarkus 3.5 and Mandrel 23.0.3
With Quarkus 3.6 and Mandrel 23.1.2
Expected behavior
Build times and binary size are expected to be comparable between 3.5, 3.6 and 3.7 given no other changes happened in the application code.
Actual behavior
The user observes significant increases in build execution times, memory usage, and binary size.
How to Reproduce?
No response
Output of
uname -aorverNo response
Output of
java -versionNo response
Mandrel or GraalVM version (if different from Java)
No response
Quarkus version or git rev
No response
Build tool (ie. output of
mvnw --versionorgradlew --version)No response
Additional information
No response