Aexyn / caliper

Automatically exported from code.google.com/p/caliper
Apache License 2.0
0 stars 0 forks source link

allocation intrument breaks my benchmark because it has non-deterministic allocations. #281

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
I was attempting to benchmark some code that uses guice (specifically creates a 
guice injector that tracks some multibinders) however I was unable to run the 
allocation instrument because I was getting non-deterministic allocation 
exceptions due to allocations that occurred from within the allocation 
instrumenting code (we must go deeper).  

here is a representative stack trace:
Experiment selection: 
  Instruments:   [allocation]
  User parameters:   {graphSize=[MEDIUM], style=[DECLARATIVE]}
  Virtual machines:  [default]
  Selection type:    Full cartesian product

This selection yields 1 experiments.
Starting trial 1 of 1: {instrument=allocation, benchmarkMethod=createInjector, 
vm=default, parameters={graphSize=MEDIUM, style=DECLARATIVE}}… 
com.google.caliper.runner.ProxyWorkerException: An exception occurred in a 
worker process.  The stack trace is as follows:
    java.lang.IllegalStateException: Your benchmark appears to have non-deterministic allocation behavior. Observed 1 instance(s) of char[10] (40 bytes) 
    at java.util.Arrays.copyOfRange(Arrays.java:2694)
    at java.lang.String.<init>(String.java:234)
    at com.google.monitoring.runtime.instrumentation.asm.ClassReader.readUTF(ClassReader.java:2137)
    at com.google.monitoring.runtime.instrumentation.asm.ClassReader.copyPool(ClassReader.java:327)
    at com.google.monitoring.runtime.instrumentation.asm.ClassWriter.<init>(ClassWriter.java:645)
    at com.google.monitoring.runtime.instrumentation.StaticClassWriter.<init>(StaticClassWriter.java:82)
    at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:168)
    at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.instrument(AllocationInstrumenter.java:195)
    at com.google.monitoring.runtime.instrumentation.AllocationInstrumenter.transform(AllocationInstrumenter.java:146)
    at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
    at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:424)
    at java.util.ArrayList$Itr.<init>(ArrayList.java:780)
    at java.util.ArrayList.iterator(ArrayList.java:774)
    at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:108)
    at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
    at sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:390)
    at sun.reflect.annotation.AnnotationParser.parseAnnotation(AnnotationParser.java:200)
    at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:88)
    at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:70)
    at java.lang.reflect.Method.declaredAnnotations(Method.java:714)
    at java.lang.reflect.Method.getAnnotation(Method.java:700)
    at com.google.inject.spi.InjectionPoint.getAtInject(InjectionPoint.java:478)
    at com.google.inject.spi.InjectionPoint.getInjectionPoints(InjectionPoint.java:676)
    at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:366)
    at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:385)
    at com.google.inject.internal.BindingBuilder.toInstance(BindingBuilder.java:84)
        ......

So the non-deterministic allocation is actually from within the 
allocationinstrumenter!  Either the instrumenter should be modified to not 
record its own allocations or caliper should be modified to eliminate them, 
since these allocations are actually being introduced by caliper, not my 
benchmark!

Original issue reported on code.google.com by lu...@google.com on 9 Dec 2013 at 6:01