search

graalvm / mandrel

Mandrel is a downstream distribution of the GraalVM community edition. Mandrel's main goal is to provide a native-image release specifically to support Quarkus.
Other
389 stars 15 forks source link

`-H:+ParseOnce` singificantly increases the memory footprint of native-image in Quarkus' integration-tests/main #316

Closed zakkak closed 5 months ago

zakkak commented 2 years ago

Description

As described in https://github.com/graalvm/mandrel/issues/304#issuecomment-952070568 -H:+ParseOnce significantly increases the memory footprint of native-image in Quarkus' integration-tests/main.

We need to better understand why this is happening while other integration tests are not affected to such an extend.

My plan is to:

How To Reproduce

git clone https://github.com/quarkusio/quarkus.git
cd quarkus
./mvnw -Dnative \
  -Dquarkus.native.additional-build-args="-H:+ParseOnce,-J-Xlog:gc*" \
  -Dquarkus.native.native-image-xmx=5g \
  -pl integration-tests/main package
github-actions[bot] commented 2 years ago

This issue appears to be stale because it has been open 30 days with no activity. This issue will be closed in 7 days unless Stale label is removed, a new comment is made, or not-Stale label is added.

zakkak commented 2 years ago

After gathering the data it's clear that main is the test with the most reachable Classes, Fields, and Methods, and as expected has the highest Peak RSS. Another interesting observation is that the top "offenders" (in terms of Peak RSS) all have a high number of classes/fields/methods registered for JNI access, so there seems to be some correlation between those JNI accesses and Peak RSS as well when ParseOnce is enabled (with awt and jaxb being the exceptions)

I am attaching the results sorted by Peak RSS. The measurements where gathered using GraalVM CE 22.0-dev build without libgraal from source based on e00d0635a390f4a4345f4ce333d16e736362c9da and Quarkus https://github.com/quarkusio/quarkus/commit/af3db6daf5

Benchmark Reach. Classes Reach. Fields Reach. Methods Refl. Classes Refl. Fields Refl. Methods JNI Classes JNI Fields JNI Methods -ParseOnce ParseOnce Heap Classes Heap Objects Compilation Units
main 25,130 42,548 131,695 994 953 3,690 119 840 1,095 8.99 9.45 17,525 761,557 91,419
liquibase 17,824 29,501 104,189 915 188 5,875 119 840 1,095 8.01 9.39 12,588 640,144 72,731
spring-web 16,727 29,622 91,652 578 289 1,761 119 840 1,095 7.6 9.1 11,634 482,136 64,140
hibernate-orm-panache 19,567 33,188 108,222 688 208 3,209 119 840 1,095 7.93 8.89 13,393 684,018 75,783
tika 17,838 30,300 94,380 443 22 752 119 840 1,095 8.09 8.87 12,557 555,447 63,501
jpa 15,487 25,942 78,824 541 44 1,015 69 91 54 7.7 8.84 10,485 386,700 52,498
hibernate-search-orm-elasticsearch-aws 17,512 27,069 84,509 623 149 1,529 68 89 54 6.89 8.78 11,729 449,096 55,883
liquibase-mongodb 16,488 29,874 96,929 1,015 500 5,953 120 842 1,095 7.72 8.75 11,682 644,305 68,841
hibernate-validator 18,280 30,147 95,131 629 57 1,175 119 840 1,095 7.21 8.7 12,456 470,226 65,389
hibernate-search-orm-elasticsearch 17,792 27,419 85,396 615 90 1,298 68 89 54 7.65 8.61 11,937 453,074 56,251
reactive-messaging-kafka 14,970 22,755 74,938 561 180 1,351 68 89 54 6.78 8.16 10,932 376,645 49,640
kafka-avro 16,356 26,883 85,292 642 893 2,348 68 89 54 7.56 8.07 11,656 453,478 57,386
keycloak-authorization 16,008 27,216 86,314 721 1,079 3,934 68 89 54 6.77 7.8 11,246 558,969 58,226
kafka 15,872 26,405 83,651 650 1,106 2,523 68 89 54 7.48 7.79 11,431 448,826 56,114
jpa-db2 13,387 20,577 66,368 529 24 912 68 89 54 6.99 7.77 8,954 335,329 43,711
kubernetes-client 13,661 22,251 81,274 848 43 7,946 68 89 54 7.09 7.75 9,378 705,436 52,777
hibernate-orm-rest-data-panache 15,698 23,183 78,281 588 47 1,273 68 89 54 7.03 7.73 10,583 411,763 52,083
micrometer-prometheus 15,853 23,862 79,521 587 26 1,178 68 89 54 7.4 7.72 10,814 409,563 53,042
jgit 12,391 21,607 62,274 512 1,025 1,003 68 89 55 6.91 7.66 8,729 309,221 41,201
openshift-client 13,320 21,638 79,411 848 32 7,991 68 89 54 7.22 7.63 9,156 698,926 51,487
oidc-client-wiremock 12,388 18,800 62,881 460 73 1,004 68 89 54 6.75 7.61 8,682 318,392 42,333
spring-data-rest 15,728 23,195 78,421 596 48 1,372 68 89 54 7.33 7.61 10,605 412,219 52,240
jpa-mysql 16,435 27,669 82,916 552 999 1,164 75 112 55 7.72 7.6 11,300 422,590 55,913
jpa-h2 13,518 20,810 66,411 530 122 1,001 68 89 54 6.89 7.54 9,070 342,594 43,740
jpa-oracle 14,002 20,965 67,446 537 44 1,004 68 89 54 7.17 7.52 9,327 354,639 44,366
kafka-streams 14,236 21,778 72,098 556 168 1,284 68 89 54 7.09 7.52 10,282 358,313 48,041
hibernate-reactive-panache 16,820 24,343 81,610 576 149 2,240 68 89 54 7.13 7.5 11,661 523,774 53,603
cache 15,684 23,361 76,322 610 38 1,257 68 89 54 6.98 7.47 10,641 404,399 50,341
opentelemetry 12,246 17,933 60,864 484 66 1,138 68 89 54 6.69 7.45 8,377 314,034 40,803
reactive-messaging-amqp 12,997 18,729 64,320 454 36 801 68 89 54 6.69 7.45 9,200 306,849 42,594
rest-client-reactive-multipart 12,255 18,451 60,806 427 22 777 68 89 54 6.52 7.45 8,414 294,854 40,522
vault-app 15,938 24,306 78,926 658 220 1,075 68 89 54 7.08 7.45 10,901 405,766 52,379
jaxb 12,729 23,532 70,866 473 19 1,088 119 840 1,095 6.91 7.41 8,761 374,910 50,280
jpa-mariadb 14,940 24,189 73,629 539 24 922 68 89 54 6.91 7.38 10,137 376,686 48,442
jpa-postgresql-withxml 14,369 21,419 70,450 576 99 1,242 68 89 54 7.23 7.38 9,621 373,746 46,962
jpa-mssql 15,194 26,355 79,351 585 27 2,051 68 89 54 7.16 7.36 10,313 446,458 53,398
hibernate-orm-envers 13,699 20,559 66,978 556 20 954 68 89 54 7.09 7.33 9,180 341,553 44,308
mongodb-panache 14,400 21,253 72,949 571 215 2,704 68 89 54 6.74 7.32 10,385 522,034 48,179
vertx-http 12,324 19,891 63,345 474 539 1,026 68 89 54 6.84 7.32 8,545 320,382 42,810
flyway 14,824 21,870 72,320 575 33 1,004 68 89 54 7.05 7.3 10,111 374,341 47,963
smallrye-jwt-token-propagation 12,909 19,709 65,184 475 119 1,050 68 89 54 6.73 7.29 8,946 322,477 43,736
qute 15,248 22,747 76,179 599 37 1,129 68 89 54 6.66 7.28 10,351 390,062 50,662
grpc-plain-text-gzip 13,653 22,661 70,175 481 49 1,069 68 89 54 6.78 7.26 9,338 337,352 45,820
consul-config 12,074 19,502 62,219 442 19 783 68 89 54 6.91 7.25 8,370 296,551 41,757
jpa-derby 13,935 20,942 68,460 554 122 1,315 68 89 54 6.76 7.25 9,287 355,249 45,442
kafka-avro-apicurio2 13,962 21,705 70,805 534 164 1,275 68 89 54 6.7 7.25 10,110 367,605 47,137
smallrye-config 13,479 21,095 68,847 520 45 1,456 68 89 55 7.04 7.25 9,327 365,879 46,350
jpa-without-entity 10,935 15,669 51,838 496 22 809 68 89 54 6.52 7.24 7,437 272,516 34,017
spring-di 11,255 18,475 56,912 445 26 847 68 89 54 6.66 7.24 7,735 288,762 37,846
jpa-postgresql 15,018 23,841 73,254 540 25 948 68 89 54 7.16 7.2 10,024 374,836 48,222
artemis-core 12,036 19,451 60,079 457 36 776 68 89 54 6.92 7.19 8,232 302,112 39,981
elasticsearch-rest-high-level-client 12,905 18,568 62,938 441 47 878 68 89 54 6.77 7.19 8,964 317,352 42,450
neo4j 13,225 19,062 65,643 463 45 849 68 89 54 6.94 7.19 9,204 310,801 44,174
oidc 11,490 17,520 58,073 433 49 726 68 89 54 5.79 7.18 8,021 284,425 39,178
amazon-lambda 13,437 21,595 69,113 478 80 1,114 68 89 54 6.63 7.17 9,328 344,622 46,144
bouncycastle 13,547 21,550 65,561 446 40 788 68 89 54 7.09 7.17 9,672 327,489 43,812
elytron-security 11,797 19,180 60,163 468 23 841 68 89 54 6.75 7.17 8,124 301,873 40,458
grpc-tls 12,387 20,439 63,842 460 39 1,001 68 89 54 6.82 7.17 8,483 311,546 41,898
kafka-oauth-keycloak 14,069 21,316 70,892 545 149 1,243 68 89 54 6.86 7.17 10,140 351,664 47,294
spring-cloud-config-client 12,357 19,980 63,537 447 19 788 68 89 54 6.83 7.16 8,595 302,302 42,532
grpc-mutual-auth 12,400 20,456 63,884 462 39 1,003 68 89 54 6.42 7.15 8,494 311,961 41,990
hibernate-reactive-postgresql 15,061 22,237 71,837 556 58 1,086 68 89 54 7.02 7.12 10,415 367,962 47,293
artemis-jms 12,180 19,666 60,796 452 36 771 68 89 54 6.88 7.1 8,309 303,817 40,284
reactive-messaging-rabbitmq 13,031 18,893 63,581 461 50 802 68 89 54 6.57 7.1 9,203 310,529 42,141
grpc-plain-text-mutiny 12,518 18,959 64,573 453 44 1,093 68 89 54 6.53 7.09 8,573 313,864 42,762
kafka-ssl 14,722 23,119 74,884 550 164 1,351 71 91 59 7.37 7.09 10,620 374,039 49,794
smallrye-graphql 14,672 23,011 74,231 508 157 1,588 68 89 54 7.2 7.09 10,405 382,079 49,100
elytron-undertow 13,020 20,862 65,838 456 23 791 68 89 54 6.57 7.08 8,992 314,977 43,633
mongodb-rest-data-panache 12,309 17,919 63,043 444 36 925 68 89 54 6.53 7.08 8,572 322,818 42,398
oidc-code-flow 11,974 18,074 60,750 447 73 908 68 89 54 6.42 7.07 8,355 302,532 40,907
kafka-sasl 14,281 21,823 72,454 557 183 1,322 68 89 54 7.15 7.06 10,316 360,127 48,336
mongodb-client 12,061 17,846 59,048 438 57 799 68 89 54 6.69 7.06 8,507 299,999 39,279
oidc-token-propagation 11,906 17,988 60,546 449 48 788 68 89 54 6.58 7.06 8,299 296,656 40,900
kafka-snappy 14,069 21,316 70,890 545 149 1,242 68 89 54 6.95 7.04 10,140 351,588 47,292
oidc-client-reactive 12,491 20,600 62,804 474 61 899 68 89 54 6.8 7.02 8,620 309,420 41,703
reactive-messaging-http 13,052 20,841 65,897 456 33 784 68 89 54 6.65 7.01 9,176 312,091 43,982
oidc-tenancy 13,685 22,425 69,770 480 72 900 68 89 54 6.87 7 9,512 331,453 46,760
elytron-resteasy 10,840 15,819 55,091 435 18 766 68 89 54 6.34 6.99 7,489 278,009 37,378
micrometer-mp-metrics 11,514 16,772 57,937 453 24 981 68 89 54 6.35 6.99 7,958 292,293 39,032
reactive-mysql-client 11,438 18,561 57,667 447 33 775 68 89 54 6.57 6.99 7,899 293,173 38,699
reactive-pg-client 11,501 18,614 57,653 447 33 775 68 89 54 6.59 6.99 7,954 292,380 38,546
hibernate-reactive-mysql 14,200 20,963 67,402 541 15 898 68 89 54 6.67 6.98 9,699 347,955 43,952
elytron-security-jdbc 11,382 18,444 57,460 449 25 793 68 89 54 6.53 6.97 7,861 287,446 38,402
hibernate-reactive-db2 14,133 20,962 67,225 541 15 898 68 89 54 6.73 6.96 9,650 346,129 43,767
reactive-db2-client 11,385 18,564 57,547 446 34 780 68 89 54 6.59 6.96 7,857 292,016 38,507
redis-client 11,469 18,794 58,093 461 40 812 68 89 54 6.64 6.96 7,984 296,122 39,104
oidc-wiremock 11,974 18,062 60,624 450 66 873 68 89 54 6.25 6.94 8,359 297,729 40,809
devtools-registry-client 11,012 16,104 56,049 491 110 1,276 68 89 54 6.59 6.91 7,573 293,158 37,716
rest-client 11,009 16,521 55,920 434 17 751 68 89 54 6.57 6.91 7,624 279,114 37,765
smallrye-opentracing 13,415 20,130 66,458 501 56 1,165 68 89 55 6.88 6.91 9,219 346,530 44,365
quartz 11,180 17,209 57,252 459 29 1,192 68 89 54 5.93 6.9 7,716 309,946 38,420
spring-boot-properties 10,679 17,259 53,894 440 22 769 68 89 54 6.37 6.86 7,342 275,092 35,949
grpc-proto-v2 12,390 20,449 63,870 460 46 1,011 68 89 54 6.9 6.84 8,485 311,936 41,914
bouncycastle-fips 12,152 19,509 62,362 456 82 837 68 89 54 6.64 6.83 8,336 333,312 42,136
infinispan-client 12,100 19,657 60,876 458 27 809 68 89 54 6.72 6.83 8,311 302,941 40,325
elasticsearch-rest-client 11,588 17,088 58,028 439 37 850 68 89 54 6.35 6.81 7,964 293,607 39,090
oidc-client 13,673 22,403 69,683 477 65 892 68 89 54 6.77 6.81 9,503 330,627 46,683
rest-client-reactive-stork 13,346 21,822 66,242 520 593 1,242 68 89 54 6.92 6.81 9,204 345,063 44,255
resteasy-jackson 11,405 16,745 57,596 444 107 1,205 68 89 54 6.44 6.81 7,858 302,632 38,528
spring-data-jpa 16,245 23,702 82,050 657 215 2,144 68 89 54 7.45 6.8 11,076 461,841 55,560
vertx-graphql 10,684 17,243 53,905 439 22 753 68 89 54 6.33 6.8 7,343 274,576 35,973
infinispan-cache-jpa 13,851 20,949 68,770 566 70 1,160 68 89 54 6.81 6.79 9,299 362,117 45,710
jaxp 10,963 17,681 55,354 439 18 807 68 89 54 6.42 6.78 7,536 281,591 37,075
bouncycastle-jsse 11,735 18,681 59,485 454 71 820 68 89 54 6.58 6.77 8,092 321,179 40,113
narayana-jta 9,923 14,821 49,044 430 19 740 68 89 54 6.43 6.76 6,793 256,260 32,511
jsch 10,810 17,812 54,817 484 169 984 68 89 54 6.53 6.74 7,432 284,173 36,580
logging-gelf 10,749 17,372 54,203 436 19 743 68 89 54 6.58 6.73 7,377 274,762 36,122
rest-client-reactive 10,918 16,340 55,493 430 8 725 68 89 54 6.59 6.73 7,551 275,876 37,420
vertx 10,816 16,346 53,674 386 13 680 68 89 54 6.41 6.72 7,708 259,124 35,060
logging-json 9,636 14,349 47,765 421 19 726 68 89 54 5.89 6.71 6,607 250,640 31,737
smallrye-metrics 10,914 15,744 52,910 444 25 839 68 89 54 6.41 6.69 7,672 280,728 35,549
elytron-resteasy-reactive 11,165 17,957 56,354 447 24 787 68 89 54 6.6 6.67 7,708 284,431 37,674
logging-min-level-set 9,620 14,328 47,710 425 24 739 68 89 54 5.96 6.64 6,606 251,630 31,715
native-config-profile 9,664 14,450 48,303 421 19 746 68 89 54 6.1 6.62 6,660 251,773 32,311
elytron-security-oauth2 10,741 15,534 52,733 446 42 814 68 89 54 6.26 6.6 7,480 272,667 35,207
mailer 11,234 16,745 54,839 467 29 801 68 89 54 6.35 6.56 7,925 282,234 36,488
vertx-web-jackson 9,937 14,705 51,152 395 11 697 68 89 54 6.31 6.5 6,865 259,146 34,607
smallrye-graphql-client 11,858 17,726 59,627 439 675 1,553 68 89 54 6.69 6.48 8,432 339,627 39,513
elytron-security-ldap 10,819 15,691 52,943 442 27 775 68 89 54 6.51 6.43 7,535 271,161 35,316
jackson 10,638 15,642 53,998 444 45 828 68 89 54 6.36 6.37 7,337 271,040 36,245
jsonb 10,213 14,830 49,910 429 21 758 68 89 54 6.22 6.37 7,087 260,343 33,192
vertx-web 9,862 14,461 50,422 390 10 686 68 89 54 6.32 6.24 6,801 249,020 31,928
awt 6,780 10,966 37,661 342 0 591 117 836 1,094 5.66 6.22 4,632 212,244 27,379
test-extension 9,381 14,235 46,775 392 105 847 68 89 54 5.84 6.21 6,423 241,359 30,948
grpc-interceptors 11,490 17,595 58,027 435 27 1,011 68 89 54 6.58 6.14 7,861 282,964 37,824
amazon-lambda-http 7,859 10,983 39,023 372 118 934 66 85 53 6.22 6.08 5,378 250,104 26,564
logging-min-level-unset 9,611 14,320 47,658 422 21 735 68 89 54 6.11 6.07 6,597 250,930 31,669
picocli-native 6,587 9,493 33,737 529 590 2,671 66 85 53 5.88 5.5 4,547 287,893 22,986

It's worth noting that after the merge of https://github.com/quarkusio/quarkus/pull/20850 Peak RSS seams to have reduced significantly for main (dropped by ~1GB), as well as the classes/fields/methods registered for JNI access.

Compiling main of Quarkus https://github.com/quarkusio/quarkus/commit/07e923aed3 (which includes https://github.com/quarkusio/quarkus/pull/20850) gives:

Benchmark Reach. Classes Reach. Fields Reach. Methods Refl. Classes Refl. Fields Refl. Methods JNI Classes JNI Fields JNI Methods Peak RSS -ParseOnce PeakRSS +ParseOnce Heap Classes Heap Objects Compilation Units
main 26,262 53,493 208,112 906 867 3587 69 89 57 8.57 8.43 16425 705470 79303
zakkak commented 2 years ago

Pasting the results (sorted by Peak RSS with ParseOnce) from Quarkus https://github.com/quarkusio/quarkus/commit/963e380f19aa087f1973ff9b5fd8d496aae9a5f4 (i.e. after https://github.com/quarkusio/quarkus/pull/20850).

As expected, the pattern that the tests with the most reachable Classes, Fields, and Methods have the highest Peak RSS seems to persist. What's interesting though is that if we sort the data by the Peak RSS difference between the -ParseOnce and the +ParseOnce runs we see that there is no clear pattern. For instance main, despite the high number of fields, methods and classes, doesn't come high in terms of Peak RSS diff.

Note that the above also negates my earlier comment:

... other integration tests are not affected to such an extend.

There are tests that are much more affected but their Peak RSS was low enough to not push the limits of the GH runner.

Benchmark Reach. Classes Reach. Fields Reach. Methods Refl. Classes Refl. Fields Refl. Methods JNI Classes JNI Fields JNI Methods -ParseOnce ParseOnce Heap Classes Heap Objects Compilation Units
hibernate-search-orm-elasticsearch 17,493 26,526 84,215 623 149 1,529 68 89 54 7.22 8.82 11,730 449,078 55,887
main 23,822 38,035 118,890 906 867 3,587 69 89 57 8.51 8.64 16,517 707,874 80,577
kafka-avro 16,370 26,424 85,073 642 893 2,348 68 89 54 7.67 8.21 11,657 453,454 57,389
jpa-mysql 16,432 27,313 82,758 552 999 1,164 75 112 55 7.58 7.89 11,305 422,851 55,973
jpa 15,469 25,602 78,578 541 44 1,015 69 91 54 7.48 7.87 10,486 386,727 52,502
keycloak-authorization 16,010 26,863 86,131 721 1,079 3,934 68 89 54 7.16 7.85 11,248 559,057 58,254
kafka 15,889 25,951 83,487 650 1,106 2,523 68 89 54 7.7 7.83 11,432 448,783 56,117
bouncycastle 13,530 21,273 65,300 446 40 788 68 89 54 6.75 7.7 9,673 327,563 43,815
kubernetes-client 14,102 22,624 83,131 848 43 7,946 68 89 54 7.26 7.69 9,679 712,103 54,151
spring-data-rest 15,731 22,824 78,202 596 48 1,372 68 89 54 7.17 7.69 10,606 412,297 52,244
liquibase-mongodb 15,466 26,272 85,809 1,013 500 5,951 69 91 54 7.49 7.65 10,898 608,973 59,204
qute 15,238 22,363 75,950 599 37 1,129 68 89 54 7.09 7.64 10,354 390,180 50,685
smallrye-graphql 14,691 22,544 74,152 508 157 1,588 68 89 54 7.34 7.62 10,432 382,661 49,171
openshift-client 13,313 21,314 79,188 848 32 7,991 68 89 54 6.75 7.59 9,157 698,965 51,490
grpc-plain-text-gzip 13,640 22,364 69,852 482 49 1,070 68 89 54 6.79 7.56 9,347 337,515 45,856
liquibase 16,833 25,923 93,136 913 188 5,873 68 89 54 7.5 7.56 11,807 604,998 63,111
hibernate-orm-panache 18,461 29,088 96,080 686 208 3,207 69 89 57 7.64 7.55 12,547 646,220 65,359
hibernate-orm-rest-data-panache 15,701 22,813 78,054 588 47 1,273 68 89 54 7.18 7.55 10,584 411,837 52,087
jpa-mssql 15,176 25,971 79,076 585 27 2,051 68 89 54 7.3 7.55 10,314 446,478 53,402
hibernate-reactive-panache 16,803 23,742 81,355 576 149 2,240 68 89 54 6.72 7.52 11,654 527,508 53,592
hibernate-validator 17,156 26,099 82,970 627 57 1,173 69 89 57 7.14 7.52 11,610 432,477 54,965
cache 15,666 22,934 76,059 610 38 1,257 68 89 54 7.15 7.5 10,642 404,436 50,344
jpa-postgresql-withxml 14,998 23,503 72,951 576 99 1,242 68 89 54 7.06 7.5 9,622 373,765 46,966
spring-web 15,634 25,599 79,520 576 289 1,759 69 89 57 7.31 7.49 10,788 444,545 53,688
jaxb 11,725 19,851 59,438 481 26 1,102 69 89 57 6.64 7.45 8,016 340,251 40,212
flyway 14,805 21,465 72,067 575 33 1,004 68 89 54 6.7 7.44 10,112 374,356 47,968
hibernate-reactive-db2 14,116 20,542 66,988 541 15 898 68 89 54 6.91 7.42 9,651 346,143 43,770
jpa-mariadb 14,919 23,835 73,347 539 24 922 68 89 54 7.03 7.42 10,138 376,670 48,446
reactive-messaging-kafka 14,992 22,249 74,810 571 182 1,365 68 89 54 7.02 7.42 10,952 377,671 49,703
kafka-oauth-keycloak 14,239 21,302 71,890 545 149 1,243 68 89 54 7.01 7.41 10,141 351,696 47,294
micrometer-prometheus 15,862 23,401 79,390 587 26 1,178 68 89 54 7.02 7.4 10,830 409,712 53,085
reactive-messaging-rabbitmq-dyn 13,026 18,474 63,353 462 50 803 68 89 54 6.72 7.4 9,204 310,588 42,145
jpa-db2 13,369 20,248 66,111 529 24 912 68 89 54 7.04 7.32 8,955 335,344 43,715
jpa-derby 13,919 20,607 68,202 554 122 1,315 68 89 54 6.97 7.29 9,288 355,262 45,446
kafka-snappy 14,284 21,347 72,247 545 149 1,242 68 89 54 6.59 7.29 10,141 351,666 47,292
kafka-streams 14,738 22,506 74,692 556 168 1,284 68 89 54 7.07 7.29 10,283 358,324 48,044
kafka-avro-apicurio2 13,965 21,246 70,588 534 164 1,275 68 89 54 7.05 7.27 10,111 367,678 47,140
hibernate-orm-envers 13,679 20,215 66,734 557 20 955 68 89 54 6.91 7.26 9,181 341,746 44,329
jpa-without-entity 10,922 15,372 51,608 496 22 809 68 89 54 6.48 7.24 7,438 272,535 34,020
kafka-sasl 14,072 20,847 70,684 557 183 1,322 68 89 54 7.12 7.24 10,317 360,143 48,339
mongodb-rest-data-panache 12,306 17,636 62,833 444 36 925 68 89 54 5.45 7.24 8,573 322,826 42,401
amazon-lambda 13,428 21,227 68,821 478 80 1,114 68 89 54 6.67 7.23 9,329 344,578 46,147
oidc-wiremock 11,983 17,790 60,500 450 66 873 68 89 54 6.62 7.21 8,361 297,827 40,837
kafka-ssl 14,072 20,847 70,682 550 164 1,351 71 91 59 6.99 7.2 10,621 374,061 49,797
elytron-undertow 13,012 20,512 65,587 456 23 791 68 89 54 6.8 7.19 8,993 315,011 43,636
smallrye-config 13,485 20,802 68,628 520 45 1,456 68 89 55 6.78 7.18 9,328 365,949 46,353
smallrye-jwt-token-propagation 12,917 19,401 65,024 475 119 1,050 68 89 54 6.76 7.18 8,947 322,476 43,749
jpa-h2 13,501 20,468 66,156 530 122 1,001 68 89 54 6.71 7.15 9,071 342,659 43,744
reactive-messaging-amqp 12,995 18,321 64,117 454 36 801 68 89 54 6.72 7.15 9,201 306,934 42,598
reactive-messaging-http 13,049 20,479 65,687 456 33 784 68 89 54 6.81 7.15 9,177 312,064 43,985
jpa-oracle 14,351 21,038 70,194 537 44 1,003 68 89 54 7.04 7.14 9,328 354,637 44,369
hibernate-reactive-mysql 14,183 20,542 67,159 541 15 898 68 89 54 6.88 7.1 9,700 348,024 43,954
vertx-http 12,312 19,595 63,082 474 539 1,026 68 89 54 6.75 7.1 8,546 320,481 42,813
elytron-resteasy 11,151 17,686 56,109 435 18 766 68 89 54 6.38 7.07 7,490 277,976 37,381
rest-client-reactive-multipart 12,782 19,331 63,692 427 557 955 68 89 54 6.65 7.07 8,820 319,051 42,800
awt 11,220 19,199 67,688 446 44 792 183 1,568 2,102 6.92 7.05 7,853 320,169 50,060
jaxp 10,945 17,406 55,087 439 18 807 68 89 54 6.46 7.04 7,537 281,615 37,078
grpc-proto-v2 12,389 20,179 63,675 461 46 1,012 68 89 54 6.92 7.02 8,494 312,104 41,950
reactive-db2-client 11,372 18,276 57,302 446 34 780 68 89 54 6.53 7 7,858 291,962 38,510
reactive-mysql-client 11,425 18,271 57,416 447 33 775 68 89 54 6.66 7 7,900 293,172 38,702
hibernate-reactive-postgresql 15,046 21,782 71,608 556 58 1,086 68 89 54 7.02 6.99 10,416 367,932 47,296
jpa-postgresql 13,983 20,633 67,161 540 25 948 68 89 54 7.35 6.99 10,025 374,850 48,226
oidc-tenancy 11,983 17,798 60,617 480 72 900 68 89 54 6.73 6.99 9,514 331,564 46,798
spring-di 11,241 18,209 56,653 445 26 847 68 89 54 6.6 6.99 7,736 288,780 37,849
mongodb-client 12,043 17,504 58,822 438 57 799 68 89 54 6.66 6.96 8,508 300,089 39,282
infinispan-client 12,083 19,350 60,490 458 27 809 68 89 54 6.83 6.95 8,312 302,889 40,328
elasticsearch-rest-high-level-client 12,915 18,187 62,715 441 47 878 68 89 54 6.71 6.94 8,965 317,362 42,453
smallrye-graphql-client 11,848 17,335 59,433 439 675 1,553 68 89 54 6.69 6.94 8,441 340,053 39,546
bouncycastle-fips 12,136 19,238 62,105 456 82 837 68 89 54 6.71 6.91 8,337 333,325 42,139
bouncycastle-jsse 11,719 18,402 59,219 454 71 820 68 89 54 6.59 6.91 8,093 321,259 40,116
elytron-security 11,783 18,899 59,922 468 23 841 68 89 54 6.53 6.91 8,125 301,942 40,462
oidc-token-propagation 13,679 22,110 69,592 449 48 788 68 89 54 6.68 6.91 8,301 296,799 40,928
reactive-messaging-rabbitmq 13,039 18,496 63,430 463 52 802 68 89 54 6.56 6.91 9,215 310,861 42,228
redis-client 11,456 18,485 57,849 461 40 812 68 89 54 6.79 6.9 7,985 296,076 39,107
grpc-plain-text-mutiny 12,537 18,695 64,417 454 44 1,094 68 89 54 6.66 6.89 8,581 314,015 42,795
grpc-mutual-auth 12,399 20,181 63,689 463 39 1,004 68 89 54 6.69 6.88 8,503 312,084 42,025
infinispan-cache-jpa 13,831 20,587 68,513 566 70 1,160 68 89 54 6.92 6.88 9,300 362,121 45,714
micrometer-mp-metrics 11,523 16,426 57,734 453 24 981 68 89 54 6.58 6.88 7,959 292,352 39,035
oidc-client-wiremock 12,397 18,484 62,767 460 73 1,004 68 89 54 6.73 6.88 8,684 318,529 42,369
devtools-registry-client 11,017 15,839 55,836 491 110 1,276 68 89 54 6.11 6.86 7,574 293,126 37,719
oidc-client-reactive 12,477 20,286 62,612 474 62 901 68 89 54 6.74 6.85 8,621 309,517 41,717
opentelemetry 12,319 17,658 60,986 484 62 1,139 68 89 54 6.58 6.84 8,427 315,867 41,003
mongodb-panache 14,396 20,748 72,756 571 215 2,704 68 89 54 7 6.83 10,386 522,091 48,183
oidc-code-flow 11,901 17,714 60,397 447 74 910 68 89 54 6.26 6.83 8,357 302,868 40,937
reactive-pg-client 11,488 18,321 57,413 447 33 775 68 89 54 6.61 6.83 7,955 292,403 38,548
rest-client 13,327 21,433 65,975 434 17 751 68 89 54 6.48 6.83 7,668 280,087 37,937
spring-cloud-config-client 12,352 19,692 63,333 447 19 788 68 89 54 6.8 6.83 8,596 302,377 42,535
elasticsearch-rest-client 11,595 16,819 57,814 439 37 850 68 89 54 6.57 6.82 7,965 293,625 39,093
oidc-client 11,508 17,301 58,061 477 65 892 68 89 54 6.73 6.79 9,505 330,735 46,719
no-awt 10,795 17,238 53,914 436 19 744 69 89 57 6.44 6.78 7,418 275,637 36,075
narayana-jta 9,908 14,555 48,811 430 19 740 68 89 54 6.39 6.76 6,794 256,269 32,514
resteasy-jackson 11,412 16,405 57,383 444 107 1,205 68 89 54 6.48 6.76 7,859 302,655 38,531
spring-boot-properties 10,661 16,988 53,629 440 22 769 68 89 54 6.36 6.76 7,343 275,105 35,952
elytron-resteasy-reactive 10,834 15,560 54,870 447 24 787 68 89 54 6.58 6.75 7,709 284,445 37,677
logging-gelf 10,731 17,097 53,942 436 19 743 68 89 54 6.47 6.75 7,378 274,783 36,125
logging-min-level-set 9,603 14,067 47,477 425 24 739 68 89 54 6.05 6.75 6,607 251,648 31,718
quartz 11,164 16,916 57,015 459 29 1,192 68 89 54 6.64 6.75 7,717 310,021 38,424
vertx-graphql 10,658 16,961 53,559 439 22 753 68 89 54 6.54 6.75 7,344 274,599 35,976
smallrye-metrics 10,901 15,408 52,676 444 25 839 68 89 54 6.37 6.67 7,673 280,709 35,552
elytron-security-jdbc 11,367 18,171 57,223 449 25 793 68 89 54 6.57 6.66 7,862 287,516 38,405
grpc-tls 12,386 20,167 63,647 461 39 1,002 68 89 54 6.76 6.66 8,492 311,723 41,934
logging-min-level-unset 9,594 14,056 47,417 422 21 735 68 89 54 5.97 6.66 6,598 250,953 31,673
rest-client-reactive 11,052 16,313 55,922 430 8 725 68 89 54 6.43 6.66 7,567 276,150 37,462
smallrye-opentracing 13,422 19,820 66,256 501 56 1,165 68 89 55 7.08 6.63 9,220 346,551 44,368
jsonb 10,200 14,515 49,672 429 21 758 68 89 54 6.23 6.61 7,088 260,349 33,195
rest-client-reactive-stork 10,932 16,088 55,377 520 593 1,242 68 89 54 6.77 6.55 9,205 345,229 44,258
elytron-security-ldap 10,806 15,398 52,731 442 27 775 68 89 54 6.26 6.54 7,536 271,243 35,319
elytron-security-oauth2 10,727 15,231 52,511 446 42 814 68 89 54 6.47 6.52 7,481 272,748 35,210
native-config-profile 9,649 14,186 48,065 421 19 746 68 89 54 6.08 6.5 6,661 251,741 32,314
jackson 10,629 15,368 53,757 444 45 828 68 89 54 6.45 6.44 7,338 271,083 36,248
vertx 10,799 15,805 53,427 386 13 680 68 89 54 6.25 6.44 7,709 259,154 35,063
vertx-web-jackson 9,928 14,442 50,902 395 11 697 68 89 54 6.15 6.41 6,866 259,131 34,610
mailer 11,227 16,398 54,640 467 29 801 68 89 54 6.41 6.38 7,928 282,306 36,510
logging-json 9,619 14,085 47,524 421 19 726 68 89 54 6.25 6.37 6,608 250,660 31,740
oidc 13,667 22,091 69,512 433 49 726 68 89 54 6.62 6.35 8,043 284,749 39,292
vertx-web 9,853 14,200 50,166 390 10 686 68 89 54 6.03 6.27 6,802 249,037 31,931
amazon-lambda-http 7,854 10,744 38,894 370 118 932 66 85 53 6.02 6.01 5,372 239,599 26,467
test-extension 9,368 13,996 46,534 392 105 847 68 89 54 6.01 5.99 6,424 241,369 30,950
spring-data-jpa 16,233 23,278 81,797 657 215 2,144 68 89 54 7.27 5.96 11,077 461,813 55,564
grpc-interceptors 11,491 17,275 57,844 436 27 1,012 68 89 54 6.43 5.83 7,869 283,125 37,853
picocli-native 6,583 9,288 33,710 529 590 2,671 66 85 53 5.92 5.59 4,548 287,917 22,990
zakkak commented 2 years ago

Depending on the case using ParseOnce results in increasing Peak RSS by up to ~30% while there are cases where Peak RSS is reduced by ~18%, but in most cases we see an increase (as expected) ranging from 1% to 15%.

jerboaa commented 2 years ago

Very interesting results. Do you have numbers comparing mandrel vs Graal VM CE? It would certainly be useful to monitor those results continuously. Especially for Quarkus as we see a significant change pre/post AWT extension integration.

zakkak commented 2 years ago

Do you have numbers comparing mandrel vs Graal VM CE?

No, I might schedule some runs for the weekend.

It would certainly be useful to monitor those results continuously. Especially for Quarkus as we see a significant change pre/post AWT extension integration.

I agree, the integration-tests are already being built regularly, so what I think we need to do is to gather the data and raise some alarm in case of big differences between runs.

zakkak commented 2 years ago

Update 2022-08-17

After running similar tests with GraalVM CE 22.2.0 I am still observing similar behavior.

To make it easier to parse I have created a plot form the data in https://github.com/graalvm/mandrel/issues/316#issuecomment-990738929

image

and a plot with the new data using GraalVM CE 22.2.0

image

Comparing the two plots we see that there is a significant reduction in Peak RSS in 22.2.0 and the gap between +ParseOnce and -ParseOnce seems to be shrinking. However, the difference between the two still ranges between -12% and 19% with the majority of the integration tests ending up needing more Peak RSS, see next plot:

image

jerboaa commented 2 years ago

Nice work! So it seems -H:-ParseOnce still makes sense.

zakkak commented 1 year ago

Update 2022-12-15

After running similar tests with Mandrel 22.3.0 I am still observing similar behavior.

image

image

Note: Some tests didn't compile in one configuration but compiled in the other. Not sure why that is (they should all compile) but the trend is still clear.

zakkak commented 5 months ago

Closing this as ParseOnce has been the default for quite some time in Quarkus and it's also now deprecated (and has no longer any effect).