search

parttimenerd / jfreventcollector

Collection of information on JFR events
https://sap.github.io/SapMachine/jfrevents/
18 stars 1 forks source link

Java-level JDK JFR events are not collected from Native Image #7

Open roberttoyonaga opened 9 months ago

roberttoyonaga commented 9 months ago

Such events that are missing include: ActiveRecording, ActiveSetting, ContainerCPUThrottling, ContainerCPUUsage, ContainerConfiguration, ContainerIOUsage, ContainerMemoryUsage, ThreadSleep, VirtualThreadEnd, VirtualThreadPinned, VirtualThreadStart.

These events are tricky because they are not listed in the source code (unlike the VM-level events found in JfrEvent.java). Instead, Native Image gets support for these events "for free" at build time. Another complexity is that we don't get all Java-level JDK events "for free", only some of them. We can support "mirror" events (usually), but not events implemented through bytecode instrumentation. I think it will be hard to get an accurate list of Java-level JDK events in an automated way.

I've been updating a list of supported events here: https://github.com/oracle/graal/issues/5410 .

Some solutions may be:

  1. I can contribute and maintain a text file in this repo that holds the supported JFR events in Native Image based on version. This would be accurate, but requires manual maintenance (similar to what I already do for the tracking issue).
  2. You can parse the test files for event name strings like "jdk.*" inside JUnit @Test methods. Example. Currently, there are some events missing tests, but I can add new tests so that all Java-level events are covered.

What do you think?

parttimenerd commented 9 months ago

A combination of both approaches would probably be best, as this could make it more future proof.

I'll add the required code in the following weeks, but I'm of course happy for any code contributions.

Another issue is the missing examples for Graal VM events. Any ideas for a benchmark project that I could use to gather a sample JFR file?

roberttoyonaga commented 9 months ago

A combination of both approaches would probably be best, as this could make it more future proof.

Cross-referencing the two sources sounds good to me. I will fill out the tests so there is complete coverage for supported java-level events. I'll work on a file you can parse that has all the supported events. In this file we can have an "up-to-date-until" tag. You could possibly reference this tag and not display graalvm info for jdk versions later than it.

Another issue is the missing examples for Graal VM events. Any ideas for a benchmark project that I could use to gather a sample JFR file?

I sometimes use a Quarkus app for testing, but it's hard to make an app that will emit every single type of JFR event, and it's hard to maintain it as new GraalVM versions are released which might support new events. Maybe the best approach is to use the GraalVm unit tests themselves. You can build the JFR unit tests into a native image binary using mx. Then you can run the binary with JFR flags. The since unit tests cover all the JFR events that are supported, you should get an example of every event type.

You need the latest graalVM source code and latest mx buid tool in order to build GraalVM and to build the unit tests.

cd graal/substratevm
mx build
mx native-unittest -p com.oracle.svm.test.jfr
svmbuild/linux-amd64/junit/svmjunit  -XX:StartFlightRecording=filename=recording.jfr

The file is a little big though because the test generates lots of events. Around 17 MB.

parttimenerd commented 9 months ago

In this file we can have an "up-to-date-until" tag. You could possibly reference this tag and not display GraalVM info for jdk versions later than it.

Where do you put it? In the GraalVM source folder or in a folder in my tool?

The file is a little big though because the test generates lots of events. Around 17 MB.

That's not large. It should work :)

But I'll add a separate issue.

roberttoyonaga commented 9 months ago

Where do you put it? In the GraalVM source folder or in a folder in my tool?

I was originally thinking that it should go in your project repository because it would be hard to justify adding a file to GraalVM that is for the purpose of an unrelated project. However, now I think that I may be able to justify it if its a resource used for the GraalVM website sources. I will first try to add it to the graalVM project, then if unsuccessful, I can add it to your project.

roberttoyonaga commented 9 months ago

I updated my PR open to change the GraalVM website docs. I've added a markdown table with all the supported events and versions. https://github.com/oracle/graal/pull/7595

roberttoyonaga commented 7 months ago

The GraalVM doc changes are merged now. https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/JFR.md

You should be able to use them as a source to find all the supported JFR events. The docs are versioned, so the list of events will have to eventually be backported to the GraalVM for JDK21 and JDK17 updates repositories.

parttimenerd commented 7 months ago

This is great. I hope to add support for it soon.