search

melix / jmh-gradle-plugin

Integrates the JMH benchmarking framework with Gradle
Apache License 2.0
666 stars 88 forks source link

No benchmarks to run; check the include/exclude regexps with 0.4.6 and Gradle 4.8 #130

Closed jdpgrailsdev closed 6 years ago

jdpgrailsdev commented 6 years ago

I have a project using version 0.4.6 of the Gradle plugin with Gradle 4.8 and no matter what I do, I receive the following error when executing the jmh task:

> Task :jmh FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jmh'.
> A failure occurred while executing me.champeau.gradle.IsolatedRunner
   > Error during execution of benchmarks

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':jmh'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
        at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:262)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:246)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:136)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.execute(DefaultTaskPlanExecutor.java:201)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.executeWithTask(DefaultTaskPlanExecutor.java:192)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.workers.internal.DefaultWorkerExecutor$WorkExecutionException: A failure occurred while executing me.champeau.gradle.IsolatedRunner
        at org.gradle.workers.internal.DefaultWorkerExecutor$WorkerExecution.waitForCompletion(DefaultWorkerExecutor.java:272)
        at org.gradle.internal.work.DefaultAsyncWorkTracker.waitForItemsAndGatherFailures(DefaultAsyncWorkTracker.java:99)
        at org.gradle.internal.work.DefaultAsyncWorkTracker.access$000(DefaultAsyncWorkTracker.java:34)
        at org.gradle.internal.work.DefaultAsyncWorkTracker$2.run(DefaultAsyncWorkTracker.java:83)
        at org.gradle.internal.work.DefaultWorkerLeaseService.withoutLocks(DefaultWorkerLeaseService.java:196)
        at org.gradle.internal.work.DefaultWorkerLeaseService.withoutProjectLock(DefaultWorkerLeaseService.java:145)
        at org.gradle.internal.work.StopShieldingWorkerLeaseService.withoutProjectLock(StopShieldingWorkerLeaseService.java:83)
        at org.gradle.internal.work.DefaultAsyncWorkTracker.waitForCompletion(DefaultAsyncWorkTracker.java:79)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:139)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
        ... 30 more
Caused by: org.gradle.api.GradleException: Error during execution of benchmarks
        at me.champeau.gradle.IsolatedRunner.run(IsolatedRunner.java:40)
        at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:39)
        at org.gradle.workers.internal.WorkerDaemonServer.execute(WorkerDaemonServer.java:46)
        at org.gradle.workers.internal.WorkerDaemonServer.execute(WorkerDaemonServer.java:30)
        at org.gradle.process.internal.worker.request.WorkerAction.run(WorkerAction.java:101)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155)
        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137)
        at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
        ... 3 more
Caused by: No benchmarks to run; check the include/exclude regexps.
        at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:268)
        at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
        at me.champeau.gradle.IsolatedRunner.run(IsolatedRunner.java:38)
        ... 12 more

After doing some searching around, I noticed a few people mentioning that the build/jmh-generated-resources/META-INF/BenchmarkList file is getting wiped out. I can confirm that this is indeed happening. If I run the jmhCompileGeneratedClasses and jmhRunBytecodeGenerator tasks manually, I can confirm the version of the BenchmarkList file that gets generated in the build directory has my benchmarks in it. However, if I run the jmh task, it fails and if I inspect the file, it is empty. I have also tried the 0.4.5 (which is apparently the minimum for Gradle 4.8) without any luck either.

jdpgrailsdev commented 6 years ago

Update: If I run the jmhJar task explicitly, after it completes the BenchmarkList file included in the JAR file is blank. However, the file on disk in build/jmh-generated-resources/META-INF has the includes in it. I am not using the shadow JAR support and from the looks of the code, the contents of the generated resources should be in the JAR: https://github.com/melix/jmh-gradle-plugin/blob/master/src/main/groovy/me/champeau/gradle/JMHPlugin.groovy#L229.

jdpgrailsdev commented 6 years ago

I see what the issue is. The BenchmarkList and CompilerHints file show up in both the jmh-generated-classes and jmh-generated-resources directories. Because of this (and the order in which they are added to the JAR), the one in the jmh-generated-classes directory always wins. I will put a PR in to reverse the order to fix this issue.

melix commented 6 years ago

Closed by the PR above, thanks!

bitemyapp commented 4 years ago

Hey @melix I seem to still be having this issue with version 0.5.0 of the plugin.

META-INF/BenchmarkList in my project-name-jmh.jar is empty.

This is my gradle build:

plugins {
    java
    application
    id("me.champeau.gradle.jmh") version("0.5.0")
}

repositories {
    jcenter()
    mavenCentral()
}

dependencies {
    jmh("commons-io:commons-io:2.4")
    jmh("org.openjdk.jmh:jmh-core:1.21")
    jmh("org.openjdk.jmh:jmh-generator-annprocess:0.9")
    implementation("org.openjdk.jmh:jmh-core:1.21")
    implementation("org.openjdk.jmh:jmh-generator-annprocess:0.9")
}

application {
    mainClassName = "com.bitemyapp.Main" 
}

val fatJar = task("fatJar", type = Jar::class) {
    archiveBaseName.set("${project.name}-fat")
    manifest {
        attributes["Implementation-Title"] = "Benchmark test"
        attributes["Implementation-Version"] = "0.0.1"
        attributes["Main-Class"] = "com.bitemyapp.Main"
    }
    from(configurations.runtimeClasspath.get().map({ if (it.isDirectory) it else zipTree(it) }))
    with(tasks.jar.get() as CopySpec)
}

tasks {
    "build" {
        dependsOn(fatJar)
    }
}

Am I missing something? I tried to follow the README carefully.

To explicit, the error I'm getting is Caused by: No benchmarks to run; check the include/exclude regexps.

$ java -jar ./build/libs/java-workbench-jmh.jar -l
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/home/callen/work/java-workbench/build/libs/java-workbench-jmh.jar) to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Benchmarks:

Also shows no benchmarks were picked up.