melix / jmh-gradle-plugin

Integrates the JMH benchmarking framework with Gradle
Apache License 2.0
673 stars 87 forks source link

jmh > 0 not possible with OpenJDK not possible #107

Open loxal opened 7 years ago

loxal commented 7 years ago

I am not using Oracle's JDK like most users on Windows but OpenJDK and apparent facing the following edge case because of this when I specify

jmh { fork = 1 # instead of fork = 0 }

C:\Users\alex\my\project\intrafind\if-sitesearch .\load-test.ps1 -w
args: -w
Processing 0 classes from C:\Users\alex\my\project\intrafind\if-sitesearch\build\classes\java\jmh with "reflection" generator
Writing out Java source to C:\Users\alex\my\project\intrafind\if-sitesearch\build\jmh-generated-sources and resources to C:\Users\alex\my\project\intrafind\if-sitesearch\build\jmh-generated-classes
# JMH 1.17.4 (released 219 days ago, please consider updating!)
# VM version: Cannot run program "C:\Users\alex\scoop\apps\openjdk\current\jre\bin\java.exe": CreateProcess error=206, The filename or extension is too long# VM invoker: C:\Users\alex\scoop\apps\openjdk\current\jre\bin\java.exe
# VM options: <none>
# Warmup: 20 iterations, 1 s each
# Measurement: 20 iterations, 1 ms each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: com.intrafind.sitesearch.jmh.LoadTest.search

# Run progress: 0.00% complete, ETA 00:00:40
# Fork: 1 of 1
<failed to invoke the VM, caught IOException: Cannot run program "C:\Users\alex\scoop\apps\openjdk\current\jre\bin\java.exe": CreateProcess error=206, The filename or extension is too long>
alexei-osipov commented 7 years ago

I'm on Oracle JDK and still get same error:

# Fork: 1 of 2
<failed to invoke the VM, caught IOException: Cannot run program "C:\Program Files\Java\jdk1.8.0_152\jre\bin\java.exe": CreateProcess error=206, The filename or extension is too long>

# Run complete. Total time: 00:00:00

Gradle 4.3 jmh-gralde-pligin 0.4.5-beta-2

Any ideas on what can be wrong?

loxal commented 7 years ago

@alexei-osipov I simply got rid of the plugin which I didn't need anyway. This is how you run JMH without the plugin which just works:

(1) https://github.com/loxal/muctool/blob/master/service/src/test/java/net/loxal/muctool/jmh/LoadBenchmark.java (2) https://github.com/loxal/muctool/blob/master/service/build.gradle


jmhReport {
    jmhResultPath = project.file("$buildDir/jmh-result.json")
    jmhReportOutput = project.file(buildDir)
}
task runJMH(type: JavaExec) {
    main = "net.loxal.muctool.jmh.LoadBenchmark"
    classpath sourceSets.test.runtimeClasspath
}
tasks.runJMH.finalizedBy tasks.jmhReport

...jmhReport is only when you need to generate a visual report. (3) ./gradlew runJMH

tdrevers commented 7 years ago

We are running into the same problem. The cause for the error in our case is that the windows command line is limited to 32768 characters. You can use the following to show the command line:

jmh {
  verbosity = 'EXTRA'
}

This shows in our case that the base classpath of 50 dependencies is extended with 200 other dependencies. These are basically: All Gradle jars, All configured Plugins and their dependencies. Looking at the plugin the JMHTask uses the correct classpath to configure the WorkerExecutor. The IsolatedRunner which is ran bij the WorkerExecuter is called with the extended classpath. So it seems the workerExecutor adds somewhere around 200 dependencies to the classpath. This classpath is used to call the Runner class from jmh itself.

The Runner class in jmh itself reuses its own classpath when forking. This is the actual point of failure.

So it seems the WorkerExecutor is not the correct way to start a separate process.

imanushin commented 6 years ago

The same is with Oracle JDK (9.0.1 inclusive). I agree with @tdrevers about the root cause

I found dirty hack:

  1. Create environment variable GRADLE_USER_HOME with value like "c:\g\"
  2. Copy all folders from your user to c:\g (there are ~5 folders with caches)
  3. Logoff/Logon (or close all applications and start them from Start Menu) to start use new variables
  4. Rerun the script

Important: please don't do this if these are several users on the same computer Why:

  1. Binaries are visible by everywere
  2. You extend your user profile to global resources
  3. For all project contributors - they will need to have adminitrative rights to work with your solution
axellimbach commented 6 years ago

In case anyone else is having the same problem, there is now an easy fix. The new jmh version 1.20 includes a flag for this problem.

jmh { jvmArgs = ['-Djmh.separateClasspathJAR=true']} jmh issue

denisrosca commented 6 years ago

@axellimbach did that work for you? I tried setting that argument and got the following error

* What went wrong:
Execution failed for task ':performance-benchmarks:jmh'.
> A failure occurred while executing me.champeau.gradle.IsolatedRunner
   > 'other' has different root
fduminy commented 6 years ago

Same error as @denisrosca with junit 5 performance benchmarks (see reference above).

Anything new on that subject ?

paulk-asert commented 5 years ago

I also get the 'other' has different root problem on windows if I have the JVM and project directory on different drives, e.g. C: and D:. Or perhaps it is the temp dir on another drive. Perhaps something like what was done for Ratpack might be needed: https://github.com/ratpack/ratpack/issues/387 In any case moving the project to the same drive as my JDK install folder fixed it for me - along with the separateClasspathJar switch mentioned above.

radhe-soni commented 5 years ago

I also get the 'other' has different root problem on windows if I have the JVM and project directory on different drives, e.g. C: and D:. Or perhaps it is the temp dir on another drive. Perhaps something like what was done for Ratpack might be needed: ratpack/ratpack#387 In any case moving the project to the same drive as my JDK install folder fixed it for me - along with the separateClasspathJar switch mentioned above.

Didn't work for me. I get below error - D:\step\Brightpeak\QA Automation code\step_bpf_aut\step_bpf_aut\step_bpf>gradlew step_benchmark:jmh Parallel execution with configuration on demand is an incubating feature.

Task :step_benchmark:jmh FAILED JMH version: 1.21 VM version: JDK 1.8.0_181, Java HotSpot(TM) 64-Bit Server VM, 25.181-b13 VM invoker: D:\Program Files\jdk1.8.0_181\jre\bin\java.exe VM options: -Djmh.separateClasspathJAR=true Warmup: 1 iterations, 10 s each Measurement: 1 iterations, 10 s each Timeout: 10 min per iteration Threads: 1 thread, will synchronize iterations Benchmark mode: Throughput, ops/time Benchmark: com.step_bpf.BenchMarkRunner.wellHelloThere

FAILURE: Build failed with an exception.

BUILD FAILED in 2s 5 actionable tasks: 2 executed, 3 from cache

paulk-asert commented 5 years ago

@radhe-soni Where is your windows temp directory?

tlinkowski commented 5 years ago

What worked for me was placing the following four resources on the same Windows drive:

  1. project containing the benchmark
  2. JDK
  3. Gradle user home dir (can be set using GRADLE_USER_HOME environment variable)
  4. TEMP dir (can be set using System properties through "java.io.tmpdir")
grv87 commented 5 years ago

Guys, I've prepared a patch for JMH fixing a problem with long classpath and different drives. Source is here.

It works for me, but could somebody give it a try so that we could be sure? Steps:

  1. Clone and build:
    hg clone https://bitbucket.org/grv87/jmh/ -b hotfix/long-classpath-absolute.v2 jmh
    cd jmh
    mvn -DskipTests=true install
  2. Use this snapshot version from build.gradle:
    repositories {
    mavenLocal()
    }
    jmh {
    jvmArgsAppend = ['-Djmh.separateClasspathJAR=true']
    jmhVersion = '1.22-SNAPSHOT'
    }

I'm planning to submit this patch at the end of the week.

tlinkowski commented 5 years ago

@grv87 Basil, thanks for the patch! Unfortunately, I won't be able to test it (at least not easily) because, after all & for some other reasons, I migrated everything to the same drive.

bingli224 commented 3 years ago

In my case, a solution is to upgrade gradle from version 5.6.2 to 6.8.3:

gradle wrapper --gradle-version 6.8.3 --distribution-type=bin

and change the plugin version::

buildscript {
  // ...
  dependencies {
    // ...
    classpath "me.champeau.gradle:jmh-gradle-plugin:0.5.0"
  }
}