Upstream JMH profilers I originally contributed to ktoso/sbt-jmh

[retronym]

ktoso/sbt-jmh contains JMH profilers based on Java Flight Recorder and async-profiler. These would be useful for all JMH users, not just the subset using it through SBT. There is interest in at least getting async-profiler support to upstream JMH.

Existing Docs

Using Oracle Flight Recorder

Flight Recorder / Java Mission Control is an excellent tool shipped by default in the Oracle JDK distribution. It is a profiler that uses internal APIs (commercial) and thus is way more precise and detailed than your every-day profiler.

To record a Flight Recorder file from a JMH run run it using the jmh.extras.JFR profiler:

jmh:run -prof jmh.extras.JFR -t1 -f 1 -wi 10 -i 20 .*TestBenchmark.*

All options can be discovered by running the help task:

sbt> jmh:run Bench -prof jmh.extras.JFR:help
Option                              Description
------                              -----------
--debugNonSafepoints <Boolean>      (default: [true, false])
--dir <Output directory>
--events <JfrEventType>             (default: [CPU, ALLOCATION_TLAB,
                                      ALLOCATION_OUTSIDE_TLAB, EXCEPTIONS,
                                      LOCKS])
--flameGraphDir <directory>       Location of clone of https://github.
                                      com/brendangregg/FlameGraph. Also
                                      can be provided as $FLAME_GRAPH_DIR
--flameGraphDirection <Directions>  Directions to generate flamegraphs
--flameGraphOpts                    Options passed to FlameGraph.pl
--flightRecorderOpts
--help                              Display help.
--jfrFlameGraphDir <directory>    Location of clone of https://github.
                                      com/chrishantha/jfr-flame-graph.
                                      Also can be provided as
                                      $JFR_FLAME_GRAPH_DIR
--jfrFlameGraphOpts                 Options passed to jfr-flame-graph
--stackDepth <Integer>              (default: 1024)
--verbose <Boolean>                 Output the sequence of commands
                                      (default: false)

This will result in flight recording file which you can then open and analyse offline using JMC.

Example output:

[info] Secondary result "JFR":
[info] JFR Messages:
[info] --------------------------------------------
[info] Flight Recording output saved to:
[info]     /Users/ktoso/code/sbt-jmh/sbt-jmh-tester/./test.TestBenchmark.range-Throughput-1.jfr

Export JFR to specific directory:

jmh:run -prof jmh.extras.JFR:dir={absolute}/{path}/{of}/{folder} -t1 -f 1 -wi 10 -i 20 .*TestBenchmark.*

Using async-profiler

Using async profiler is done by using the jmh.extras.Async profiler like this:

sbt> jmh:run Bench -prof jmh.extras.Async ...

All additional options are documented in it's help task:

sbt> jmh:run Bench -prof jmh.extras.Async:help

Option                              Description
------                              -----------
--asyncProfilerDir <directory>    Location of clone of https://github.
                                      com/jvm-profiling-tools/async-
                                      profiler. Also can be provided as
                                      $ASYNC_PROFILER_DIR
--dir <<directory>>                 Output directory
--event <AsyncProfilerEventType>    Event to sample (default: [CPU, HEAP])
--flameGraphDir <directory>       Location of clone of https://github.
                                      com/brendangregg/FlameGraph. Also
                                      can be provided as $FLAME_GRAPH_DIR
--flameGraphDirection <Directions>  Directions to generate flamegraphs
                                      (default: [BOTH, NONE, FORWARD,
                                      REVERSE])
--flameGraphOpts                    Options passed to FlameGraph.pl
--framebuf <Long>                   Size of profiler framebuffer (default:
                                      8388608)
--help                              Display help.
--threads <Boolean>                 profile threads separately (default:
                                      [false, true])
--verbose <Boolean>                 Output the sequence of commands
                                      (default: false)

Automatically generating flame-grapghs

Read more about flame graphs here:

To automatically generate flame graphs for a given benchmark you can invoke:

sbt> jmh:run Bench -f1 -wi 5 -i5 -prof jmh.extras.JFR:dir=/tmp/profile-jfr;flameGraphDir=/code/FlameGraph;jfrFlameGraphDir=/code/jfr-flame-graph;flameGraphOpts=--minwidth,2;verbose=true 

Where /code/FlameGraph and /code/jfr-flame-graph need to reflect actual paths of those tools on your system.

Upstreaming Plan

Async Profiler

• [x] Update the option parser in sbt-jmh to reflect what's available in the latest version of async-profiler
• [x] Review the best way to start/stop the profiler. Currently it calls $ASYNC_PROFILER_HOME/jattach load true <command>. There are other ways to do this now, not sure if they improve simplicity, portability, etc.
• [x] Review the integrated flame graph generation. A minimal version would remove the wrapper around flamegraph.pl. Users could either use the built-in flamegraph support that was since added to async-profiler, or post-process collapsed-stacks.csv themselves with flamegraph.pl.

Java Flight Recorder

• [x] Remove the automatic setting of UnlockCommercialFeatures. Require Oracle JDK users to add this with -jvmArgs.
• [x] Remove support for .jfr => flamegraph support? The third party tool (https://github.com/chrishantha/jfr-flame-graph) that is required to do this is mostly unsupported.

[retronym]

-prof async: https://mail.openjdk.java.net/pipermail/jmh-dev/2020-August/002966.html

[retronym]

-prof jfr: https://mail.openjdk.java.net/pipermail/jmh-dev/2020-August/002983.html