nikgoodley-ibboost / gbench

Automatically exported from code.google.com/p/gbench
Apache License 2.0
0 stars 0 forks source link

BenchmarkWarumUp#run may loop very long or even forever #8

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Execute this Groovy program with indy enabled on for example JDK 1.7.11 and 
Groovy 2.1.0:

@Grab('com.googlecode.gbench:gbench:0.4.0-groovy-2.0')
import groovyx.gbench.Benchmark

import groovyx.gbench.BenchmarkBuilder

def r = benchmark {
    'concat' {
        '1'+'1'
    }
}
r.prettyPrint()

Due to the JVM not always doing the right thing for invokedynamic (yet), it is 
entirely possible, that the execution times never end up in the 5% variation 
that #stable desires. It does so not after even several minutes of trying to 
stabilize (I did see the warmup loop being executed many more than 20 times).

To avoid the user waiting forever there should be an upper limit on the number 
of tries. Not sure what to do after that though. Or maybe giving an error like 
"the benchmark did not stabilize after X tries, please use a better JVM"?

Using a JDK1.8 JVM resolve the problem as well for this case, but there might 
be other cases like this. In any way it would be good to give the user some 
feedback about what is going wrong.

Original issue reported on code.google.com by blackdra...@googlemail.com on 6 Mar 2013 at 4:45

GoogleCodeExporter commented 9 years ago
Thank you for reporting. I'll fix the prob shortly.

Original comment by nagaimas...@gmail.com on 7 Mar 2013 at 8:56

GoogleCodeExporter commented 9 years ago
add maxWarmUpTime option and set its default to 60 sec.

Original comment by nagaimas...@gmail.com on 8 Mar 2013 at 10:56