Feature request: jmh:benchmark-name task

[perlun]

Thanks for a nice plugin!

At the moment, ./gradlew jmh will run all JMH benchmarks in a project. This is often useful, but sometimes you might want to selectively run just one or a few benchmarks.

The pre-existing include setting is useful for this, but it would be even nicer if you could also provide a class name on the command line. Something like ./gradlew jmh-run pattern or similar. Feel free to come up with a better implementation detail. The key point is to be able to cherry-pick a particular test that should be executed, to avoid having to run the full test suite every time (which might be incredibly time consuming).

[bric3]

I tried the tasks.create<JMHTask>("benchmarkName") {} approach but this didn't work well for reasons I didn't get.

At this time I'm using this approach

jmh {
  warmupIterations.set(5) // -wi 5
  iterations.set(5) // -i 5
  warmup.set("1") // -w 1
  timeOnIteration.set("1") // -r 1
  benchmarkMode.set(listOf("avgt")) // -bm avgt
  timeUnit.set("ns") // -tu ns
}

tasks.create("bmName") {
  jmh.includes.add("TheBenchmark")
  jmh.resultsFile.set(project.file("${project.buildDir}/reports/jmh/the-benchmark-results.txt"))
  jmh.fork.set(1) // -f 1
  finalizedBy("jmh")
}

tasks.create("bmNameFork3Times") {
  jmh.includes.add("TheBenchmark")
  jmh.resultsFile.set(project.file("${project.buildDir}/reports/jmh/the-benchmark-forks-3-results.txt"))
  jmh.fork.set(3) // -f 3
  finalizedBy("jmh")
}

However this approach doesn't allow to chain these tasks.

[zabetak]

Running individual benchmarks via the command line (CLI) without requiring modifications to the build script would be a very nice feature indeed.

I suppose the most canonical way of doing this would be something like:

./gradlew jmh --includes=TheNameOfTheSpecificClassToRun

which is what is also proposed in https://github.com/melix/jmh-gradle-plugin/issues/239 via the @Option annotation.

As a workaround in Apache Calcite we are using custom Gradle project properties that override the respective includes (see https://github.com/apache/calcite/pull/3857).