search

Karm / mandrel-integration-tests

Integration tests for GraalVM and its Mandrel distribution. Runs Quarkus, Helidon and Micronaut applications and small targeted reproducers. The focus is solely on native-image utility and compilation of Java applications into native executables.
Apache License 2.0
5 stars 3 forks source link

Quarkus renaming REST extensions #251

Closed Karm closed 3 months ago

Karm commented 3 months ago

fixes #250

Karm commented 3 months ago 
Err:9 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal/main amd64 gdb amd64 10.2-0ubuntu1~20.04~1
  503  Service Unavailable [IP: 185.125.190.80 80]
E: Failed to fetch http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu/pool/main/g/gdb/gdb_10.2-0ubuntu1~20.04~1_amd64.deb  503  Service Unavailable [IP: 185.125.190.80 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@jerboaa GHA glitching...

jerboaa commented 3 months ago 
Err:9 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal/main amd64 gdb amd64 10.2-0ubuntu1~20.04~1
  503  Service Unavailable [IP: 185.125.190.80 80]
E: Failed to fetch http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu/pool/main/g/gdb/gdb_10.2-0ubuntu1~20.04~1_amd64.deb  503  Service Unavailable [IP: 185.125.190.80 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@jerboaa GHA glitching...

Yeah. :-(

Question: What does quarkus main, i.e. 999-SNAPSHOT, amount to in terms of quarkus version? Or in other words, has the main branch been tested with this?

Karm commented 3 months ago 
Err:9 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal/main amd64 gdb amd64 10.2-0ubuntu1~20.04~1
  503  Service Unavailable [IP: 185.125.190.80 80]
E: Failed to fetch http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu/pool/main/g/gdb/gdb_10.2-0ubuntu1~20.04~1_amd64.deb  503  Service Unavailable [IP: 185.125.190.80 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@jerboaa GHA glitching...

Yeah. :-(

Question: What does quarkus main, i.e. 999-SNAPSHOT, amount to in terms of quarkus version? Or in other words, has the main branch been tested with this?

999-SNAPSHOT, amount to in terms of quarkus version?

it could mean anything, it's just the main branch build, says nothing about its age. In my case on my laptop with this PR, that is: 9d6f6335c6 built as $ time ./mvnw -Dquickly -Prelocations

Karm commented 3 months ago 
 Error:  Failures: 
Error:    PerfCheckTest.testQuarkusFullMicroProfile:506 build-and-run.log log should not contain error or warning lines that are not whitelisted. See /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/ts/testsuite/target/archived-logs/org.graalvm.tests.integration.PerfCheckTest/testQuarkusFullMicroProfile/build-and-run.log and check these offending lines: 
    at io.quarkus.security.runtime.SecurityIdentityAssociation_ClientProxy.arc$delegate(Unknown Source)
    at io.quarkus.security.runtime.SecurityIdentityAssociation_ClientProxy.getIdentity(Unknown Source)
2024-03-20 21:43:55,439 ERROR [org.jbo.thr.errors] (executor-thread-2) Thread Thread[#56,executor-thread-2,5,main] threw an uncaught exception: jakarta.enterprise.context.ContextNotActiveException: RequestScoped context was not active when trying to obtain a bean instance for a client proxy of CLASS bean [class=io.quarkus.security.runtime.SecurityIdentityAssociation, id=Qby7zgDOyRL0j3YxdYwwuXFQIeE]
    at io.quarkus.security.runtime.SecurityIdentityProxy_ClientProxy.isAnonymous(Unknown Source) ==> expected: <true> but was: <false>
New failure. Investigating....
jerboaa commented 3 months ago

999-SNAPSHOT, amount to in terms of quarkus version?

it could mean anything, it's just the main branch build, says nothing about its age. In my case on my laptop with this PR, that is: 9d6f6335c6 built as $ time ./mvnw -Dquickly -Prelocations

So does it apply the new 3.9 patch does it? Otherwise it wouldn't build, right?

Karm commented 3 months ago

999-SNAPSHOT, amount to in terms of quarkus version?

it could mean anything, it's just the main branch build, says nothing about its age. In my case on my laptop with this PR, that is: 9d6f6335c6 built as $ time ./mvnw -Dquickly -Prelocations

So does it apply the new 3.9 patch does it? Otherwise it wouldn't build, right?

So does it apply the new 3.9 patch does it? Otherwise it wouldn't build, right?

I used -Prelocations so as it builds even without the patch. That way I collected all failures in the log where the TS spotted WARNING about extension relocation. Then I went app by app adding a patch...

jerboaa commented 3 months ago

999-SNAPSHOT, amount to in terms of quarkus version?

it could mean anything, it's just the main branch build, says nothing about its age. In my case on my laptop with this PR, that is: 9d6f6335c6 built as $ time ./mvnw -Dquickly -Prelocations

So does it apply the new 3.9 patch does it? Otherwise it wouldn't build, right?

So does it apply the new 3.9 patch does it? Otherwise it wouldn't build, right?

I used -Prelocations so as it builds even without the patch. That way I collected all failures in the log where the TS spotted WARNING about extension relocation. Then I went app by app adding a patch...

But that's not what the CI is using, right? Please check it builds with quarkus main (without -Prelocations) and passes the test. Thanks!

Karm commented 3 months ago

But that's not what the CI is using, right?

It indeed ain't, it would not have been failing the way it did otherwise. I merely used it to make the update easier.

As you can see, I added Q 3.9.0.CR2 to the CI here, so it is going to be easily visible that it works / doesn't work.

I am trying to sort out the failure on GHA now, what is cached glitch and what is actual failure.

Pls do not merge until it's all green.

jerboaa commented 3 months ago

As you can see, I added Q 3.9.0.CR2 to the CI here, so it is going to be easily visible that it works / doesn't work.

But that doesn't test main branch and how 999-SNAPSHOT versions are being handled (in terms of the 3.9.x patch). Hence my ask to test with the main branch as well.

Karm commented 3 months ago

@jerboaa

As you can see, I added Q 3.9.0.CR2 to the CI here, so it is going to be easily visible that it works / doesn't work.

But that doesn't test main branch and how 999-SNAPSHOT versions are being handled (in terms of the 3.9.x patch). Hence my ask to test with the main branch as well.

I can confirm that Q main, 3650cb478c, using these previously faulty sections:

$  mvn  clean verify -Ptestsuite -DexcludeTags=all -DincludeTags=perfcheck,reproducers 
-Dtest=JFRTest,PerfCheckTest -Dquarkus.version=999-SNAPSHOT 
-Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 -Dquarkus.native.container-runtime=podman  | tee log.log

Q built without -Prelocations.

There is a somewhat noticeable drop in perf with JFR smoke in container on my laptop:

JFRTest.jfrSmokeContainerTest:140->jfrSmoke:737 
Application JFR_SMOKE_BUILDER_IMAGE in mode diff_native took 49 ms more to finish, which is over 30 ms threshold by 63%.

when running with Q main. Gonna verify and drop a line to Robert if I can confirm it.

In the latest commit, i whitelisted what looks like a Q 3.8.3 specific error message in the log. Gonna investigate separately.

Karm commented 3 months ago

Forgot about DebugSymbolsTest.debugSymbolsQuarkusContainer:, fixing...

Karm commented 3 months ago

So it's not just Q 3.8.3, it's intermittently Q 3.9 too :-(

Error:  Failures: 
Error:    PerfCheckTest.testQuarkusFullMicroProfile:506 build-and-run.log log should not contain error or warning lines that are not whitelisted. See /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/ts/testsuite/target/archived-logs/org.graalvm.tests.integration.PerfCheckTest/testQuarkusFullMicroProfile/build-and-run.log and check these offending lines: 
    at io.quarkus.security.runtime.SecurityIdentityProxy_ClientProxy.arc$delegate(Unknown Source)
2024-03-21 23:43:10,519 ERROR [org.jbo.thr.errors] (executor-thread-1) Thread Thread[#20,executor-thread-1,5,main] threw an uncaught exception: jakarta.enterprise.context.ContextNotActiveException: RequestScoped context was not active when trying to obtain a bean instance for a client proxy of CLASS bean [class=io.quarkus.security.runtime.SecurityIdentityProxy, id=U3fuB4yO9MSr82V2xU36xFn98dk]
2024-03-21 23:43:23,967 ERROR [org.jbo.thr.errors] (executor-thread-1) Thread Thread[#20,executor-thread-1,5,main] threw an uncaught exception: jakarta.enterprise.context.ContextNotActiveException: RequestScoped context was not active when trying to obtain a bean instance for a client proxy of CLASS bean [class=io.quarkus.security.runtime.SecurityIdentityProxy, id=U3fuB4yO9MSr82V2xU36xFn98dk]
2024-03-21 23:43:41,969 ERROR [org.jbo.thr.errors] (executor-thread-3) Thread Thread[#35,executor-thread-3,5,main] threw an uncaught exception: jakarta.enterprise.context.ContextNotActiveException: RequestScoped context was not active when trying to obtain a bean instance for a client proxy of CLASS bean [class=io.quarkus.security.runtime.SecurityIdentityProxy, id=U3fuB4yO9MSr82V2xU36xFn98dk]
    at io.quarkus.security.runtime.SecurityIdentityProxy_ClientProxy.isAnonymous(Unknown Source) ==> expected: <true> but was: <false>
[INFO]

I incline towards whitelisting so as we have CI clean and investigating separately.

I haven't been successful reproducing it locally so far.

$  mvn  clean verify -Ptestsuite -DexcludeTags=all -DincludeTags=perfcheck,reproducers -Dtest=PerfCheckTest#testQuarkusFullMicroProfile -Dquarkus.version=999-SNAPSHOT -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 -Dquarkus.native.container-runtime=podman  | tee log.log 
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO]
jerboaa commented 3 months ago

I incline towards whitelisting so as we have CI clean and investigating separately.

+1 and please file an issue to investigate it. Thanks!

Karm commented 3 months ago

@jerboaa IMHO ready to merge.

Ad io.quarkus.security.runtime.SecurityIdentityAssociation: I am crafting a Q reproducer project and I can deterministically reproduce it. It requires the combination of MP JWT + REST Client + OpenTelemetry in a single app context. It is unrelated to native. As soon as I have the minimal reproducer gift wrapped, I paste it on Quarkus issues. It might be so that /me trying to make it simpler for the test app, mushing together client+server app in a single one does something OT did not expect.

Karm commented 3 months ago

@jerboaa Follow up Q issue: https://github.com/quarkusio/quarkus/issues/39667

(it's possible that it's my fault, but let the Q folks decide....)