search

Karm / mandrel-integration-tests

Integration tests for GraalVM and its Mandrel distribution. Runs Quarkus, Helidon and Micronaut applications and small targeted reproducers. The focus is solely on native-image utility and compilation of Java applications into native executables.
Apache License 2.0
5 stars 3 forks source link

JFR perf tests in container #181

Closed Karm closed 10 months ago

Karm commented 10 months ago

@roberttoyonaga This PR adds another run to your tests, this time in containers, i.e. it

One thing that it not tested at all is the contents of the generated recordings, examining flight-native.jfr.

I'd add that in another PR. Suggestions are welcome :-D

roberttoyonaga commented 10 months ago

Hi Karm, this looks good to me!

One thing that it not tested at all is the contents of the generated recordings, examining flight-native.jfr.

I think the graalVM unittests have a good example of how to quickly read the event content from the .jfr snapshot: https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.test/src/com/oracle/svm/test/jfr/AbstractJfrTest.java#L101-L111

I think checking that the snapshot contains a total number of jdk.ThreadPark events greater-than-or-equal-to completed requests reported by hyperfoil would be a useful test. That should also confirm that the JFR snapshot is well-formed. Also, adding a check that the "blocker" class field contained in the event data corresponds to org.acme.getting.started.GreetingService would help verify that the Type and Symbol constant pools are working/formed correctly. I'd be happy to help with this test too.

Having such JFR correctness checks as a part of this test would be a great sanity check to verify the test is actually doing what we expect. But, its probably not critical to check everything because the GraalVM unit tests check correctness already.

Karm commented 10 months ago

@roberttoyonaga I have the event counting almost done, gonna update this PR for a review.

Karm commented 10 months ago

I am still testing it. Pending some simplification... Seems legit locally though.

Karm commented 10 months ago

JFRTest.jfrPerfTest:176->jfrPerfTestRun:210 build-and-run.log log should not contain error or warning lines that are not whitelisted. See /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/ts/testsuite/target/archived-logs/org.graalvm.tests.integration.JFRTest/jfrPerfTest/build-and-run.log and check these offending lines: 2023-08-31 16:46:00,988 WARN [io.net.boo.ServerBootstrap] (vert.x-acceptor-thread-0) Failed to register an accepted channel: [id: 0xee9f8975, L:/127.0.0.1:8080 ! R:/127.0.0.1:52004]: java.lang.IllegalStateException ==> expected: but was:

roberttoyonaga commented 10 months ago

JFRTest.jfrPerfTest:176->jfrPerfTestRun:210 build-and-run.log log should not contain error or warning lines that are not whitelisted. See /home/runner/work/mandrel-integration-tests/mandrel-integration-tests/ts/testsuite/target/archived-logs/org.graalvm.tests.integration.JFRTest/jfrPerfTest/build-and-run.log and check these offending lines: 2023-08-31 16:46:00,988 WARN [io.net.boo.ServerBootstrap] (vert.x-acceptor-thread-0) Failed to register an accepted channel: [id: 0xee9f8975, L:/127.0.0.1:8080 ! R:/127.0.0.1:52004]: java.lang.IllegalStateException ==> expected: but was:

I'm not too sure where this is coming from. I tried to reproduce it on my machine including the recent commits, but was unable to (rhel mandrel 23.0.1.2-Final for JDK20, Quarkus 3.2.3). Found a similar problem on SO (but maybe not relevant).

Karm commented 10 months ago

@roberttoyonaga It intermittently comes from Quarkus: JDK 20 based run.

 905 Command: ./target/jfr-plaintext-runner
 906 __  ____  __  _____   ___  __ ____  ______
 907  --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 908  -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
 909 --\___\_\____/_/ |_/_/|_/_/|_|\____/___/
 910 2023-08-31 16:46:00,955 INFO  [io.quarkus] (main) jfr-native-image-performance 1.0.0-SNAPSHOT native (powered by Quarkus 3.2.3.Final) started in 0.123s. Listening on: http://0.0.0.0:8080
 911 2023-08-31 16:46:00,961 INFO  [io.quarkus] (main) Profile prod activated.
 912 2023-08-31 16:46:00,961 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy-reactive, smallrye-context-propagation, vertx]
 913 2023-08-31 16:46:00,988 WARN  [io.net.boo.ServerBootstrap] (vert.x-acceptor-thread-0) Failed to register an accepted channel: [id: 0xee9f8975, L:/127.0.0.1:8080 ! R:/127.0.0.1:52004]: java.lang.IllegalState     Exception
 914   at io.vertx.core.net.impl.VertxEventLoopGroup.next(VertxEventLoopGroup.java:37)
 915   at io.vertx.core.net.impl.VertxEventLoopGroup.register(VertxEventLoopGroup.java:53)
 916   at io.netty.bootstrap.ServerBootstrap$ServerBootstrapAcceptor.channelRead(ServerBootstrap.java:215)
 917   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
 918   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
 919   at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
 920   at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
 921   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
 922   at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
 923   at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
 924   at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:97)
 925   at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
 926   at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
 927   at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
 928   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:553)
 929   at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
 930   at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 931   at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 932   at java.base@20.0.2/java.lang.Thread.runWith(Thread.java:1636)
 933   at java.base@20.0.2/java.lang.Thread.run(Thread.java:1623)
 934   at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:807)
 935   at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:210)
 936

I will whitelist it here...

Karm commented 10 months ago

@roberttoyonaga @jerboaa It works now. I'll merge it then.

Thx for the review.