search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.64k stars 2.64k forks source link

Native build always get stuck at 'parsing method' #25672

Closed deathwaiting closed 1 year ago

deathwaiting commented 2 years ago

Describe the bug

I using quarkus 8.3.Final , gradle Native build always get stucked at the step [4/7] Parsing methods... [******] while grinding the CPU 

Building native image source jar: /home/runner/work/qu-queue-service/qu-queue-service/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar/qu-queue-service-1.0.0-SNAPSHOT-runner.jar
Building native image from /home/runner/work/qu-queue-service/qu-queue-service/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar/qu-queue-service-1.0.0-SNAPSHOT-runner.jar
Using docker to run the native image builder
Checking image status quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java17
22.0.0.2-Final-java17: Pulling from quarkus/ubi-quarkus-mandrel
54e56e6f8572: Pulling fs layer
4f8ddd7f5a75: Pulling fs layer
20939a5b3d59: Pulling fs layer
4f8ddd7f5a75: Verifying Checksum
4f8ddd7f5a75: Download complete
54e56e6f8572: Verifying Checksum
54e56e6f8572: Download complete
54e56e6f8572: Pull complete
20939a5b3d59: Verifying Checksum
20939a5b3d59: Download complete
4f8ddd7f5a75: Pull complete
20939a5b3d59: Pull complete
Digest: sha256:7751b408ac408d6f91a95c864a2b8d85129987c8d5c1fc5356e9940c8e330837
Status: Downloaded newer image for quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java17
quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java17
Running Quarkus native-image plugin on native-image 22.0.0.2-Final Mandrel Distribution (Java Version 17.0.2+8)
docker run --env LANG=C --rm --user 1001:121 -v /home/runner/work/qu-queue-service/qu-queue-service/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar:/project:z --name build-native-clyaI quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java17 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-DCoordinatorEnvironmentBean.transactionStatusManagerEnable=false -J-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Dfile.encoding=UTF-8 -H:-ParseOnce -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy\$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -J-Djava.awt.headless=true -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -J-Xmx3G -H:-AddAllCharsets -H:EnableURLProtocols=http,https -H:-UseServiceLoaderFeature -H:+StackTrace qu-queue-service-1.0.0-SNAPSHOT-runner -jar qu-queue-service-1.0.0-SNAPSHOT-runner.jar
========================================================================================================================
GraalVM Native Image: Generating 'qu-queue-service-1.0.0-SNAPSHOT-runner'...
========================================================================================================================
[1/7] Initializing...                                                                                   (13.1s @ 0.22GB)
 Version info: 'GraalVM 22.0.0.2-Final Java 17 Mandrel Distribution'
 8 user-provided feature(s)
  - io.quarkus.caffeine.runtime.graal.CacheConstructorsAutofeature
  - io.quarkus.hibernate.orm.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.jdbc.postgresql.runtime.graal.SQLXMLFeature
  - io.quarkus.runner.AutoFeature
  - io.quarkus.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.runtime.graal.ResourcesFeature
  - org.hibernate.graalvm.internal.GraalVMStaticAutofeature
  - org.hibernate.graalvm.internal.QueryParsingSupport
[2/7] Performing analysis...  [**********]                                                             (191.2s @ 2.47GB)
  26,471 (96.76%) of 27,356 classes reachable
  45,531 (68.17%) of 66,791 fields reachable
 143,995 (80.30%) of 179,318 methods reachable
   1,537 classes, 1,795 fields, and 10,535 methods registered for reflection
      65 classes,    77 fields, and    55 methods registered for JNI access
[3/7] Building universe...                                                                               (7.2s @ 2.60GB)
[4/7] Parsing methods...      [******]

Expected behavior

Native build should be completed, or a clarification error should be shown

Actual behavior

The native build get stuck at the the step, for as long as 6h

How to Reproduce?

No response

Output of uname -a or ver

Linux ahmed-legion 5.16.20-2-MANJARO #1 SMP PREEMPT Thu Apr 14 06:44:39 UTC 2022 x86_64 GNU/Linux

Output of java -version

openjdk version "17.0.2" 2022-01-18 OpenJDK Runtime Environment Temurin-17.0.2+8 (build 17.0.2+8) OpenJDK 64-Bit Server VM Temurin-17.0.2+8 (build 17.0.2+8, mixed mode, sharing)

GraalVM version (if different from Java)

using builder image quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java17

Quarkus version or git rev

2.8.3.Final

Build tool (ie. output of mvnw --version or gradlew --version)

------------------------------------------------------------ Gradle 7.3-rc-5 ------------------------------------------------------------ Build time: 2021-11-05 18:43:37 UTC Revision: 426f267f7b768154fd63b27e055ba56b357e6e4e Kotlin: 1.5.31 Groovy: 3.0.9 Ant: Apache Ant(TM) version 1.10.11 compiled on July 10 2021 JVM: 17.0.2 (Eclipse Adoptium 17.0.2+8) OS: Linux 5.16.20-2-MANJARO amd64

Additional information

build.gradle.kts

plugins {
    java
    id("io.quarkus")
}

repositories {
    mavenLocal()
    mavenCentral()
}

val quarkusPlatformGroupId: String by project
val quarkusPlatformArtifactId: String by project
val quarkusPlatformVersion: String by project

dependencies {
    implementation("io.quarkus:quarkus-container-image-jib")
    implementation(enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}"))
    implementation("io.quarkus:quarkus-resteasy-reactive-jackson")
    implementation("io.quarkus:quarkus-jaxrs-client-reactive")
    implementation("io.quarkus:quarkus-hibernate-reactive-panache")
    implementation("io.quarkus:quarkus-smallrye-openapi")
//    implementation("io.quarkus:quarkus-smallrye-jwt")
    implementation("io.quarkus:quarkus-liquibase")
    implementation("io.quarkus:quarkus-mailer")
    implementation("io.quarkus:quarkus-reactive-pg-client")
    implementation("io.quarkus:quarkus-smallrye-metrics")
    implementation("io.quarkus:quarkus-arc")
    implementation("io.quarkus:quarkus-oidc")
    implementation("org.hibernate.validator:hibernate-validator")
    implementation("io.quarkus:quarkus-jdbc-postgresql")
    implementation("io.quarkus:quarkus-keycloak-admin-client")
    implementation("io.quarkus:quarkus-cache")
    implementation("org.mapstruct:mapstruct:1.4.1.Final")

    compileOnly("org.projectlombok:lombok:1.18.16")
    compileOnly("javax.validation:validation-api:2.0.1.Final")
    testCompileOnly("org.projectlombok:lombok:1.18.20")
    annotationProcessor("org.projectlombok:lombok:1.18.20")
    testAnnotationProcessor("org.projectlombok:lombok:1.18.20")
    annotationProcessor("org.mapstruct:mapstruct-processor:1.4.1.Final")

    testImplementation("io.quarkus:quarkus-junit5")
    testImplementation("io.quarkus:quarkus-junit5-mockito")
    testImplementation("io.quarkus:quarkus-test-security-oidc")
    testImplementation("io.rest-assured:rest-assured")
    testImplementation("org.testcontainers:postgresql:1.15.1")
    testImplementation("org.jdbi:jdbi3-core:3.1.0")
    testImplementation("com.tngtech.keycloakmock:mock:0.6.0");
    testImplementation("io.quarkus:quarkus-test-oidc-server")
    testImplementation("uk.co.datumedge:hamcrest-json:0.2")
    testImplementation("org.skyscreamer:jsonassert:1.5.0")
}

group = "com.qu"
version = "1.0.0-SNAPSHOT"

java {
    sourceCompatibility = JavaVersion.VERSION_17
    targetCompatibility = JavaVersion.VERSION_17
}

tasks.withType<JavaCompile> {
    options.encoding = "UTF-8"
    options.compilerArgs.add("-parameters")
}
quarkus-bot[bot] commented 2 years ago

/cc @Karm, @evanchooly, @galderz, @geoand, @zakkak

Karm commented 2 years ago

Hello @deathwaiting,

Could you create a simple repo that reproduces the issue, please? It could be your project, sanitized of all important business logic and sensitive information and configs.

Right off the bat, seeing lombok in the deps makes me worry. What happens if you remove it from the code and deps?

deathwaiting commented 2 years ago

@Karm

This is a sample repo.

also what was tried so far:

So, it seems the problem happens if I add panache entities, which is strange as I did native build before with the same entities in previous versions of quarkus/java 11. The problem is also, the logs gives no clue of what can be the issue.

galderz commented 2 years ago

@deathwaiting Thanks for uploading the sample repo. I'll give it a go.

galderz commented 2 years ago

@deathwaiting If you leave it long enough it says:

GC warning: 197.5s spent in 316 GCs during the last stage, taking up 98.48% of the time.
            Please ensure more than 3.68GB of memory is available for Native Image
            to reduce GC overhead and improve image build time.

You just need to give native image more memory really. So, comment this line in application.properties:

#quarkus.native.native-image-xmx=3G

And with GraalVM 22.0 you get:

./gradlew build -Dquarkus.package.type=native --info -x test
...
========================================================================================================================
GraalVM Native Image: Generating 'qu-queue-service-1.0.0-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                    (7.6s @ 0.22GB)
 Version info: 'GraalVM 22.2.0-dev Java 17 CE'
 C compiler: cc (apple, arm64, 13.1.6)
 Garbage collector: Serial GC
 8 user-provided feature(s)
  - io.quarkus.caffeine.runtime.graal.CacheConstructorsAutofeature
  - io.quarkus.hibernate.orm.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.jdbc.postgresql.runtime.graal.SQLXMLFeature
  - io.quarkus.runner.AutoFeature
  - io.quarkus.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.runtime.graal.ResourcesFeature
  - org.hibernate.graalvm.internal.GraalVMStaticAutofeature
  - org.hibernate.graalvm.internal.QueryParsingSupport
[2/7] Performing analysis...  [**********]                                                              (27.0s @ 5.00GB)
  25,006 (92.13%) of 27,141 classes reachable
  42,120 (64.41%) of 65,397 fields reachable
 137,658 (60.32%) of 228,224 methods reachable
   1,504 classes, 1,665 fields, and 11,492 methods registered for reflection
      66 classes,    78 fields, and    56 methods registered for JNI access
[3/7] Building universe...                                                                               (3.0s @ 2.55GB)
[4/7] Parsing methods...      [***]                                                                      (5.6s @ 3.20GB)
[5/7] Inlining methods...     [*****]                                                                    (4.9s @ 3.59GB)
[6/7] Compiling methods...    [****]                                                                    (18.6s @ 4.81GB)
[7/7] Creating image...                                                                                  (5.8s @ 5.61GB)
  54.42MB (43.23%) for code area:   92,759 compilation units
  60.55MB (48.10%) for image heap:  17,686 classes and 639,325 objects
  10.92MB ( 8.67%) for other data
 125.89MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area:                               Top 10 object types in image heap:
   2.85MB com.oracle.svm.core.reflect                         11.83MB byte[] for code metadata
   2.85MB jdk.proxy4                                          11.01MB byte[] for general heap data
   2.05MB liquibase.pro.packaged                               6.27MB java.lang.Class
   1.80MB sun.security.ssl                                     5.52MB byte[] for java.lang.String
   1.18MB java.util                                            5.41MB java.lang.String
 832.89KB io.quarkus.runtime.generated                         3.78MB java.lang.reflect.Method
 745.53KB com.sun.crypto.provider                              2.10MB com.oracle.svm.core.hub.DynamicHubCompanion
 614.95KB com.oracle.svm.core.code                             1.38MB byte[] for reflection metadata
 548.89KB org.hibernate.hql.internal.antlr                     1.09MB java.lang.String[]
 533.13KB java.lang                                          854.53KB java.util.HashMap$Node
      ... 1135 additional packages                                ... 5438 additional object types
                                           (use GraalVM Dashboard to see all)
------------------------------------------------------------------------------------------------------------------------
                       10.7s (13.5% of total time) in 63 GCs | Peak RSS: 8.51GB | CPU load: 5.83
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /Users/g/tmp/quarkus-native-build-stuck/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar/qu-queue-service-1.0.0-SNAPSHOT-runner (executable)
 /Users/g/tmp/quarkus-native-build-stuck/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar/qu-queue-service-1.0.0-SNAPSHOT-runner.build_artifacts.txt
========================================================================================================================
Finished generating 'qu-queue-service-1.0.0-SNAPSHOT-runner' in 1m 18s.

Notice the peak RSS numbers. You need a fair bit of heap to do native compilation.

geoand commented 2 years ago

@galderz I assume we can close this?

galderz commented 2 years ago

@geoand I'd wait a bit and give @deathwaiting opportunity to explain the need for constrained resources.

deathwaiting commented 2 years ago

@galderz Thanks for your help!

I removed the memory limit options from my main project and was able to make a native build using github CI actions.

however, I couldn't do it on my local machine, as it consumed too much memory and almost crashed the system.

image

And in the end, the build failed.

 ./gradlew build -Dquarkus.package.type=native -Dquarkus.native.container-build=true -Dquarkus.native.container-runtime=podman -x test
The Agroal dependency is present but no JDBC datasources have been defined.
Quarkus detected the need for Mutiny reactive programming support, however the quarkus-resteasy-mutiny extension was not present. Reactive REST endpoints in your application that return Uni or Multi will not function as you expect until you add this extension. Endpoints that need Mutiny are: com.qu.controller.OrganizationController{io.smallrye.mutiny.Multi<com.qu.dto.AdminInvitationDTO> getAdminInvitations()}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.String> getAdminRegistrationPage(java.lang.String token)}, com.qu.controller.Pages{io.smallrye.mutiny.Uni<java.lang.String> getTurnRequestPage(java.lang.Long id)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.util.Set<com.qu.services.queue.event.model.QueueEventHandlerInfo<?>>> getAllEventHandlers()}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.util.List<com.qu.dto.QueueTypeDto>> getQueueTypes()}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueListResponse> getQueueList(java.lang.Integer pageCount, java.lang.Integer pageIndex)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueDetailsDto> getQueue(java.lang.Long id)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnView> getTurnByUid(@NotNull java.lang.Long quId, @NotNull java.lang.String uuid)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueRequestDto> getRequestByUid(@NotNull java.lang.Long quId, @NotNull java.lang.String uuid)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> dequeue(@NotNull java.lang.Long id)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.Long> createOrganization(com.qu.dto.OrganizationCreateDTO orgDto)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<com.qu.dto.AdminInvitationCreateResponse> inviteAdmin(com.qu.dto.AdminInvitationCreateRequest invitation)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.String> getAdminRegistrationPage(java.lang.String token, java.lang.String password, java.lang.String passwordRepeat)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Long> createQueueType(com.qu.dto.QueueTypeDto queueType)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Long> createQueue(com.qu.dto.QueueDto queue)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> setQueueStatus(@NotNull java.lang.Long id, @NotNull com.qu.commons.enums.QueueActionType action)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> addTurn(@NotNull java.lang.Long id, @NotNull com.qu.dto.QueueTurnCreateDto turn)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueRequestAnswerDto> makeTurnRequest(@NotNull java.lang.Long id, @NotNull com.qu.dto.QueueRequestCreateDto request)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> acceptTurnRequest(@NotNull java.lang.Long id, @NotNull java.lang.Long requestId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> denyTurnRequest(@NotNull java.lang.Long quId, @NotNull java.lang.Long requestId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> skipTurn(@NotNull java.lang.Long id, java.lang.String skipReason)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> endTurn(@NotNull java.lang.Long queueId, @NotNull java.lang.Long turnId)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.Void> cancelAdminInvitations(java.lang.String invitationId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> deleteQueueType(java.util.List<java.lang.Long> id)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> cancelRequestByCustomer(@NotNull java.lang.Long quId, @NotNull java.lang.String uuid)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> cancelTurn(@NotNull java.lang.Long queueId, @NotNull java.lang.Long turnId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> cancelTurnByCustomer(@NotNull java.lang.Long queueId, @NotNull java.lang.String turnUid)}

> Task :quarkusBuild
building quarkus jar

Trying to pull quay.io/quarkus/ubi-quarkus-mandrel:22.0.0.2-Final-java17...
Getting image source signatures
Copying blob sha256:20939a5b3d59d0f08da07a066229900bd30794c15fa00a87e6366019fb30c2a1
Copying blob sha256:20939a5b3d59d0f08da07a066229900bd30794c15fa00a87e6366019fb30c2a1
Copying blob sha256:4f8ddd7f5a755f537dd9d5f553c8c78171dcf3018c5fc96676a07380d3e14e20
Copying blob sha256:54e56e6f85721741ee7bf0336de8ad3bf138a56769a6d0097b600a0e361be58d
Copying config sha256:7e9ff147de36d3c835bbc2ad5134dc04ae0764a8cdc17c27b9d7e5c2aedb3888
Writing manifest to image destination
Storing signatures
7e9ff147de36d3c835bbc2ad5134dc04ae0764a8cdc17c27b9d7e5c2aedb3888
========================================================================================================================
GraalVM Native Image: Generating 'qu-queue-service-1.0.0-SNAPSHOT-runner'...
========================================================================================================================
[1/7] Initializing...                                                                                    (8.4s @ 0.25GB)
 Version info: 'GraalVM 22.0.0.2-Final Java 17 Mandrel Distribution'
 8 user-provided feature(s)
  - io.quarkus.caffeine.runtime.graal.CacheConstructorsAutofeature
  - io.quarkus.hibernate.orm.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.jdbc.postgresql.runtime.graal.SQLXMLFeature
  - io.quarkus.runner.AutoFeature
  - io.quarkus.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.runtime.graal.ResourcesFeature
  - org.hibernate.graalvm.internal.GraalVMStaticAutofeature
  - org.hibernate.graalvm.internal.QueryParsingSupport
[2/7] Performing analysis...  [**********]                                                             (100.1s @ 3.94GB)
  26,471 (96.76%) of 27,356 classes reachable
  45,530 (68.17%) of 66,790 fields reachable
 143,995 (80.30%) of 179,318 methods reachable
   1,537 classes, 1,795 fields, and 10,535 methods registered for reflection
      65 classes,    77 fields, and    55 methods registered for JNI access
[3/7] Building universe...                                                                              (14.6s @ 2.35GB)
Error: Image build request failed with exit status 137

> Task :quarkusBuild FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':quarkusBuild'.
> io.quarkus.builder.BuildException: Build failure: Build failed due to errors
        [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: io.quarkus.deployment.pkg.steps.NativeImageBuildStep$ImageGenerationFailureException: Image generation failed. Exit code was 137 which indicates an out of memory error. Consider increasing the Xmx value for native image generation by setting the "quarkus.native.native-image-xmx" property
        at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.imageGenerationFailed(NativeImageBuildStep.java:391)
        at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:237)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:925)
        at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
        at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
        at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
        at java.base/java.lang.Thread.run(Thread.java:833)
        at org.jboss.threads.JBossThread.run(JBossThread.java:501)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 6m 41s
5 actionable tasks: 1 executed, 4 up-to-date

That's why I set the memory limit option , but previously, 4GB was enough to make a successful build. In fact I was surprised that github CLI managed to do the build, as AFAIK a server instance running the job have only like 6GB

Karm commented 2 years ago

I'll try to run it in a matrix with different versions on a fleet of weaker systems and collect the stats.

galderz commented 2 years ago

@deathwaiting What is your local system? Have you tried building outside native outside of a container? E.g. set JAVA_HOME to GraalVM or Mandrel. If you're on anything other than Linux, running a container runtime involves some kind of VM, whose overall memory you can control...etc.

deathwaiting commented 2 years ago

@galderz 

Kernel Version: 5.16.20-2-MANJARO (64-bit)
Graphics Platform: X11
Processors: 16 × AMD Ryzen 7 4800H with Radeon Graphics
Memory: 15.1 GiB of RAM

I am using podman with runC, so it should be running in native mode with no VM.

Running the native build without a container succeeded and had better performance, and at least it didn't take too much memory to cause a crash

quarkus_memory_2

./gradlew build -Dquarkus.package.type=native -x test
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details

> Task :quarkusBuild
building quarkus jar

The Agroal dependency is present but no JDBC datasources have been defined.
Quarkus detected the need for Mutiny reactive programming support, however the quarkus-resteasy-mutiny extension was not present. Reactive REST endpoints in your application that return Uni or Multi will not function as you expect until you add this extension. Endpoints that need Mutiny are: com.qu.controller.OrganizationController{io.smallrye.mutiny.Multi<com.qu.dto.AdminInvitationDTO> getAdminInvitations()}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.String> getAdminRegistrationPage(java.lang.String token)}, com.qu.controller.Pages{io.smallrye.mutiny.Uni<java.lang.String> getTurnRequestPage(java.lang.Long id)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.util.Set<com.qu.services.queue.event.model.QueueEventHandlerInfo<?>>> getAllEventHandlers()}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.util.List<com.qu.dto.QueueTypeDto>> getQueueTypes()}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueListResponse> getQueueList(java.lang.Integer pageCount, java.lang.Integer pageIndex)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueDetailsDto> getQueue(java.lang.Long id)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnView> getTurnByUid(@NotNull java.lang.Long quId, @NotNull java.lang.String uuid)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueRequestDto> getRequestByUid(@NotNull java.lang.Long quId, @NotNull java.lang.String uuid)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> dequeue(@NotNull java.lang.Long id)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.Long> createOrganization(com.qu.dto.OrganizationCreateDTO orgDto)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<com.qu.dto.AdminInvitationCreateResponse> inviteAdmin(com.qu.dto.AdminInvitationCreateRequest invitation)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.String> getAdminRegistrationPage(java.lang.String token, java.lang.String password, java.lang.String passwordRepeat)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Long> createQueueType(com.qu.dto.QueueTypeDto queueType)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Long> createQueue(com.qu.dto.QueueDto queue)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> setQueueStatus(@NotNull java.lang.Long id, @NotNull com.qu.commons.enums.QueueActionType action)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> addTurn(@NotNull java.lang.Long id, @NotNull com.qu.dto.QueueTurnCreateDto turn)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueRequestAnswerDto> makeTurnRequest(@NotNull java.lang.Long id, @NotNull com.qu.dto.QueueRequestCreateDto request)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> acceptTurnRequest(@NotNull java.lang.Long id, @NotNull java.lang.Long requestId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> denyTurnRequest(@NotNull java.lang.Long quId, @NotNull java.lang.Long requestId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<com.qu.dto.QueueTurnDto> skipTurn(@NotNull java.lang.Long id, java.lang.String skipReason)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> endTurn(@NotNull java.lang.Long queueId, @NotNull java.lang.Long turnId)}, com.qu.controller.OrganizationController{io.smallrye.mutiny.Uni<java.lang.Void> cancelAdminInvitations(java.lang.String invitationId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> deleteQueueType(java.util.List<java.lang.Long> id)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> cancelRequestByCustomer(@NotNull java.lang.Long quId, @NotNull java.lang.String uuid)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> cancelTurn(@NotNull java.lang.Long queueId, @NotNull java.lang.Long turnId)}, com.qu.controller.QueueController{io.smallrye.mutiny.Uni<java.lang.Void> cancelTurnByCustomer(@NotNull java.lang.Long queueId, @NotNull java.lang.String turnUid)}
========================================================================================================================
GraalVM Native Image: Generating 'qu-queue-service-1.0.0-SNAPSHOT-runner'...
========================================================================================================================
[1/7] Initializing...                                                                                    (8.5s @ 0.28GB)
 Version info: 'GraalVM 22.0.0.2-Final Java 17 Mandrel Distribution'
 8 user-provided feature(s)
  - io.quarkus.caffeine.runtime.graal.CacheConstructorsAutofeature
  - io.quarkus.hibernate.orm.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.jdbc.postgresql.runtime.graal.SQLXMLFeature
  - io.quarkus.runner.AutoFeature
  - io.quarkus.runtime.graal.DisableLoggingAutoFeature
  - io.quarkus.runtime.graal.ResourcesFeature
  - org.hibernate.graalvm.internal.GraalVMStaticAutofeature
  - org.hibernate.graalvm.internal.QueryParsingSupport
[2/7] Performing analysis...  [**********]                                                              (65.2s @ 3.65GB)
  26,471 (96.76%) of 27,356 classes reachable
  45,530 (68.17%) of 66,790 fields reachable
 143,995 (80.30%) of 179,318 methods reachable
   1,537 classes, 1,795 fields, and 10,535 methods registered for reflection
      65 classes,    77 fields, and    55 methods registered for JNI access
[3/7] Building universe...                                                                               (7.8s @ 2.71GB)
[4/7] Parsing methods...      [***]                                                                      (7.7s @ 2.40GB)
[5/7] Inlining methods...     [*****]                                                                   (11.7s @ 2.96GB)
[6/7] Compiling methods...    [******]                                                                  (42.5s @ 5.28GB)
[7/7] Creating image...                                                                                 (13.6s @ 4.20GB)
  58.32MB (37.17%) for code area:   99,955 compilation units
  83.55MB (53.25%) for image heap:  19,013 classes and 1,147,847 objects
  15.03MB ( 9.58%) for other data
 156.91MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area:                               Top 10 object types in image heap:
   2.99MB com.oracle.svm.core.reflect                         24.22MB byte[] for general heap data
   2.82MB org.keycloak.admin.client.resource                   6.64MB java.util.LinkedHashMap
   2.46MB jdk.proxy4                                           6.48MB java.lang.Class
   1.97MB liquibase.pro.packaged                               6.14MB java.lang.String
   1.74MB sun.security.ssl                                     5.49MB byte[] for java.lang.String
   1.17MB java.util                                            4.25MB java.lang.reflect.Method
 762.82KB io.quarkus.runtime.generated                         3.78MB s.r.a.AnnotatedTypeFactory$AnnotatedTypeBaseImpl
 730.68KB com.sun.crypto.provider                              1.56MB java.lang.reflect.Parameter
 623.86KB liquibase.command.core                               1.12MB java.lang.Object[]
 578.39KB org.hibernate.hql.internal.antlr                     1.02MB java.lang.String[]
      ... 1228 additional packages                                ... 5848 additional object types
                                           (use GraalVM Dashboard to see all)
------------------------------------------------------------------------------------------------------------------------
                       11.9s (7.2% of total time) in 102 GCs | Peak RSS: 8.59GB | CPU load: 10.79
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /run/media/ahmed/Work/java/workspaces/qu-queue-service/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar/qu-queue-service-1.0.0-SNAPSHOT-runner (executable)
 /run/media/ahmed/Work/java/workspaces/qu-queue-service/build/qu-queue-service-1.0.0-SNAPSHOT-native-image-source-jar/qu-queue-service-1.0.0-SNAPSHOT-runner.build_artifacts.txt
========================================================================================================================
Finished generating 'qu-queue-service-1.0.0-SNAPSHOT-runner' in 2m 45s.
Base image 'quay.io/quarkus/quarkus-micro-image:1.0' does not use a specific image digest - build may not be reproducible

BUILD SUCCESSFUL in 5m 33s
5 actionable tasks: 1 executed, 4 up-to-date
geoand commented 1 year ago

Is there any update on this? Is it still an issue?

Karm commented 1 year ago

With these modifications to the original reproducer:

$ git diff
diff --git a/gradle.properties b/gradle.properties
index 9b25299..4ef6191 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -1,7 +1,7 @@
         #Gradle properties
 #Fri May 13 17:59:43 EET 2022
-quarkusPluginVersion=2.8.3.Final
+quarkusPluginVersion=2.13.4.Final
 quarkusPlatformArtifactId=quarkus-universe-bom
 quarkusPluginId=io.quarkus
 quarkusPlatformGroupId=io.quarkus
-quarkusPlatformVersion=2.8.3.Final
+quarkusPlatformVersion=2.13.4.Final
diff --git a/gradlew b/gradlew
old mode 100644
new mode 100755
diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties
index 68f7675..b08af3f 100644
--- a/src/main/resources/application.properties
+++ b/src/main/resources/application.properties
@@ -75,6 +75,6 @@ quarkus.swagger-ui.oauth-realm=qu
 quarkus.swagger-ui.oauth-app-name=backend-service

 #Image build
-quarkus.container-image.image=docker.io/ahmedgalalfathy/private:qu-backend
+#quarkus.container-image.image=docker.io/ahmedgalalfathy/private:qu-backend
 quarkus.container-image.push=false
-quarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel:21.3.2.0-Final-java17
\ No newline at end of file
+quarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel:22.3-java17

The build completes for me just fine, the memory consumption is nothing outlandish or unexpected:

$ ./gradlew build -Dquarkus.package.type=native -Dquarkus.native.container-build=true -Dquarkus.native.container-runtime=docker -x test
Starting a Gradle Daemon (subsequent builds will be faster)
The Agroal dependency is present but no JDBC datasources have been defined.
22.3-java17: Pulling from quarkus/ubi-quarkus-mandrel
...
Digest: sha256:5f24fe7559dd5ed8f5765ea44de42395c7ae13f8ea48f75dd4ca3a67c3a91486
Status: Downloaded newer image for quay.io/quarkus/ubi-quarkus-mandrel:22.3-java17
quay.io/quarkus/ubi-quarkus-mandrel:22.3-java17
========================================================================================================================
GraalVM Native Image: Generating 'qu-queue-service-1.0.0-SNAPSHOT-runner' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                   (12.3s @ 0.41GB)
 Version info: 'GraalVM 22.3.0.1-Final Java 17 Mandrel Distribution'
 Java version info: '17.0.5+8'
 C compiler: gcc (redhat, x86_64, 8.5.0)
 Garbage collector: Serial GC
 9 user-specific feature(s)
 - io.quarkus.caffeine.runtime.graal.CacheConstructorsFeature
 - io.quarkus.hibernate.orm.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.hibernate.Version, org.hibernate.annotations.common.Version, org.hibernate.dialect.Dialect] categories
 - io.quarkus.jdbc.postgresql.runtime.graal.SQLXMLFeature
 - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
 - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.jboss.threads] categories
 - io.quarkus.runtime.graal.ResourcesFeature: Register each line in META-INF/quarkus-native-resources.txt as a resource on Substrate VM
 - org.graalvm.home.HomeFinderFeature: Finds GraalVM paths and its version number
 - org.hibernate.graalvm.internal.GraalVMStaticFeature: Hibernate ORM's static reflection registrations for GraalVM
 - org.hibernate.graalvm.internal.QueryParsingSupport: Hibernate ORM's support for HQL Parser in GraalVM
[2/7] Performing analysis...  [*******]                                                                 (73.8s @ 2.45GB)
  24,627 (92.11%) of 26,737 classes reachable
  41,336 (64.53%) of 64,054 fields reachable
 126,032 (58.37%) of 215,918 methods reachable
   1,376 classes, 1,634 fields, and 11,049 methods registered for reflection
      64 classes,    70 fields, and    55 methods registered for JNI access
       4 native libraries: dl, pthread, rt, z
[3/7] Building universe...                                                                              (15.3s @ 2.20GB)
[4/7] Parsing methods...      [***]                                                                      (9.2s @ 1.51GB)
[5/7] Inlining methods...     [***]                                                                      (4.0s @ 1.62GB)
[6/7] Compiling methods...    [*******]                                                                 (54.6s @ 2.09GB)
[7/7] Creating image...                                                                                  (8.8s @ 1.95GB)
  50.14MB (48.42%) for code area:    84,407 compilation units
  53.08MB (51.26%) for image heap:  552,567 objects and 326 resources
 344.74KB ( 0.33%) for other data
 103.56MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area:                               Top 10 object types in image heap:
   2.68MB org.keycloak.admin.client.resource                  11.21MB byte[] for code metadata
   1.64MB sun.security.ssl                                     6.16MB byte[] for embedded resources
   1.60MB jdk.proxy4                                           6.01MB java.lang.Class
   1.10MB java.util                                            5.09MB java.lang.String
 771.50KB io.quarkus.runtime.generated                         4.71MB byte[] for java.lang.String
 729.31KB java.lang.invoke                                     4.13MB byte[] for general heap data
 718.47KB com.sun.crypto.provider                              2.07MB com.oracle.svm.core.hub.DynamicHubCompanion
 587.36KB org.hibernate.hql.internal.antlr                     1.70MB byte[] for reflection metadata
 552.63KB liquibase.sqlgenerator.core                          1.02MB java.lang.String[]
 537.21KB com.oracle.svm.core.code                           872.20KB java.util.HashMap$Node
  38.69MB for 1166 more packages                               9.84MB for 5340 more object types
------------------------------------------------------------------------------------------------------------------------
                       24.0s (12.4% of total time) in 256 GCs | Peak RSS: 3.93GB | CPU load: 7.98
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /project/qu-queue-service-1.0.0-SNAPSHOT-runner (executable)
 /project/qu-queue-service-1.0.0-SNAPSHOT-runner-build-output-stats.json (json)
 /project/qu-queue-service-1.0.0-SNAPSHOT-runner-timing-stats.json (raw)
 /project/qu-queue-service-1.0.0-SNAPSHOT-runner.build_artifacts.txt (txt)
========================================================================================================================
Finished generating 'qu-queue-service-1.0.0-SNAPSHOT-runner' in 3m 12s.

BUILD SUCCESSFUL in 15m 6s
6 actionable tasks: 3 executed, 3 up-to-date