melix / jmh-gradle-plugin

Integrates the JMH benchmarking framework with Gradle
Apache License 2.0
677 stars 87 forks source link

ByteCodeGenerator or annotation processor? #61

Open baohuahuang1 opened 8 years ago

baohuahuang1 commented 8 years ago

Hi,

I am using jmh-gradle-plugin 0.3.0. Base on the src code, this plugin is using org.openjdk.jmh.generators.bytecode.JmhBytecodeGenerator.

But when I tried to remove "jmh spec.external.jmhGeneratorAnnprocess" from my dependencies no META-INF/BenchmarkList is generated.

Is jmh-gradle-plugin still relying on the annoation process to generate META-INF/BenchmarkList? If so what's the purpose of running JmhBytecodeGenerator in task jmhRunBytecodeGenerator?

vyazelenko commented 8 years ago

@baohuahuang1 Since #29 was implemented jmh-gradle-plugin is using bytecode generator. This means that annotation processor is not used anymore.

I just did a quick test with simple build.gradle file:

plugins {
    id 'java'
    id 'me.champeau.gradle.jmh' version '0.3.0'
}

repositories {
    jcenter()
}

Now if I run gradle dependencies I get the following output for jmh configuration:

jmh
+--- org.openjdk.jmh:jmh-core:1.11.3
|    +--- net.sf.jopt-simple:jopt-simple:4.6
|    \--- org.apache.commons:commons-math3:3.2
\--- org.openjdk.jmh:jmh-generator-bytecode:1.11.3
     +--- org.openjdk.jmh:jmh-core:1.11.3 (*)
     +--- org.openjdk.jmh:jmh-generator-reflection:1.11.3
     |    \--- org.openjdk.jmh:jmh-core:1.11.3 (*)
     \--- org.openjdk.jmh:jmh-generator-asm:1.11.3
          +--- org.openjdk.jmh:jmh-core:1.11.3 (*)
          +--- org.openjdk.jmh:jmh-generator-reflection:1.11.3 (*)
          \--- org.ow2.asm:asm:5.0.3

As you can see annotation processor is not used.

Would you be able to share a sample project where problem can be reproduced?

baohuahuang1 commented 8 years ago

@vyazelenko Thanks for your response. Unfortunately I am unable to share a sample project. I will do more investigation on my end to see what the issue is and will update this thread if I find anything.

Baohua

srangwal commented 8 years ago

The issue listed here would occur when we would try to run the jmh benchmark. Compilation of the benchmarks didn't have issues.

We identified that this error, specifically,

Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList
    at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:96)
    at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:104)
    at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:256)
    at org.openjdk.jmh.runner.Runner.run(Runner.java:206)
    at org.openjdk.jmh.Main.main(Main.java:71)

Has sometime to do with the .gradle/ directory. Once we deleted this directory we were able to run the benchmarks without the issue.

We found that while running benchmarks we would see an error like No matching benchmarks. Miss-spelled regexp? and were able to eliminate it by, as above, deleting the .gradle/ directory.

aalmiray commented 8 years ago

Could you share more information about your setup? Is there by any chance any file dependency set on the root directory? Otherwise the .gradle directory should not cause a problem

srangwal commented 8 years ago

Sorry for the late reply. I am unable to share the setup (proprietary). If you can explain what you mean by

Could you share more information about your setup? Is there by any chance any file dependency set on the root directory?

I can dig deeper.

vyazelenko commented 8 years ago

@srangwal how are you running benchmarks? Using jmh task or how?

srangwal commented 8 years ago

Yes using task jmh

deaddowney commented 8 years ago

I see this too. It seems to happen randomly. I need to delete the .gradle directory to get back to normal.

TimvdLippe commented 7 years ago

Can confirm that I need to rm -rf .gradle/ to succesfully run ./gradlew jmh.

My configuration is

jmh {
    duplicateClassesStrategy = 'warn'
    fork = 2
    warmupIterations = 5
    iterations = 5
    timeUnit = 'ms'
    verbosity = 'EXTRA'
    include = ['.*']
}

My BenchmarkTest is

package mockitointegration;

import org.mockito.Mockito;
import org.openjdk.jmh.annotations.*;

@State(Scope.Thread)
public class MockitoBenchmarkTest {

    private Class<?> clazz;

    private ClassLoader loader;

    public MockitoBenchmarkTest() {
        loader = new IMethodsClassLoader();
        Thread.currentThread().setContextClassLoader(loader);
    }

    @Setup(Level.Invocation)
    public void setup() throws ClassNotFoundException {
        clazz = loader.loadClass("org.mockitousage.IMethods");
    }

    @TearDown
    public void teardown() {

    }

    @Benchmark
    public void simple_mock() {
        Mockito.mock(clazz);
    }
}
jest commented 7 years ago

I also had this problem, but removing .gradle was not necessary. Simply stopping the Gradle daemon (with either ./gradlew --stop or gradle --stop, depending on the way you run Gradle) was enough.