search

hcoles / pitest

State of the art mutation testing system for the JVM
Apache License 2.0
1.7k stars 357 forks source link

ArrayIndexOutOfBoundsException when running mutationCoverage #481

Closed ReedOei closed 1 year ago

ReedOei commented 6 years ago

Reproduce steps:

git clone https://github.com/ReedOei/eunomia
cd eunomia
mvn compile test-compile pitest:mutationCoverage -e

I am using Java 8 on Ubuntu 16.04 LTS:

openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.16.04.1-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)

Here is the full output from Maven:

[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 1 modules...
[INFO] Installing Nexus Staging features:
[INFO]   ... total of 1 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building eunomia 1.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- pitest-maven:1.4.0:mutationCoverage (default-cli) @ eunomia ---
[INFO] Found plugin : Default csv report plugin
[INFO] Found plugin : Default xml report plugin
[INFO] Found plugin : Default html report plugin
[INFO] Found plugin : Static initializer code detector plugin
[INFO] Found plugin : Static initializer filter plugin
[INFO] Found plugin : Excluded annotations plugin
[INFO] Found plugin : Inlined finally block filter plugin
[INFO] Found plugin : Try with resources filter
[INFO] Found plugin : Implicit null check filter
[INFO] Found plugin : For each loop filter
[INFO] Found plugin : Logging calls filter
[INFO] Found plugin : Infinite for loop filter
[INFO] Found plugin : Long running iterator loop filter
[INFO] Found plugin : For loop counter filter
[INFO] Found plugin : Kotlin junk mutations filter
[INFO] Found plugin : Max mutations per class limit
[INFO] Found plugin : Equals shortcut equivalent mutant filter
[INFO] Found plugin : Trivial return vals equivalence filter
[INFO] Found plugin : Mutant export plugin
[INFO] Found shared classpath plugin : Default mutation engine
[INFO] Found shared classpath plugin : JUnit plugin
[INFO] Found shared classpath plugin : TestNG plugin
[INFO] Adding org.pitest:pitest to SUT classpath
[INFO] Mutating from /home/roei/Java/eunomia/target/classes
[INFO] Defaulting target classes to match packages in build directory
10:49:49 PM PIT >> INFO : ---------------------------------------------------------------------------
10:49:49 PM PIT >> INFO : Enabled (+) and disabled (-) features.
10:49:49 PM PIT >> INFO : -----------------------------------------
10:49:49 PM PIT >> INFO : +FFBLOCK        Filters mutations in code duplicated by finally block inlining
10:49:49 PM PIT >> INFO : +FSTATI         Filters mutations in static initializers and code called only from them
10:49:49 PM PIT >> INFO : +FSEQUIVEQUALS  Filters equivalent mutations that affect only performance in short cutting equals methods
10:49:49 PM PIT >> INFO : +FFEACH         Filters mutations in compiler generated code that implements for each loops
10:49:49 PM PIT >> INFO : +FINFINC        Filters mutations to increments that may cause infinite loops
10:49:49 PM PIT >> INFO : +FFLOOP         Filters any mutations to increments in for loops as they may cause timeouts
10:49:49 PM PIT >> INFO : +FRETEQUIV      Filters return vals mutants with bytecode equivalent to the unmutated class
10:49:49 PM PIT >> INFO : +FINULL         Filters mutations in compiler generated code that checks for null by calling getClass
10:49:49 PM PIT >> INFO : +FTRYWR         Filters mutations in code generated for try with resources statements
10:49:49 PM PIT >> INFO : +FKOTLIN        Filters out junk mutations in bytecode created by compiler for kotlin language features
10:49:49 PM PIT >> INFO : +FSTATINIT      Filters mutations in static initializers and code called only from them
10:49:49 PM PIT >> INFO : +FLOGCALL       Filters mutations in code that makes calls to logging frameworks
10:49:49 PM PIT >> INFO : +FINFIT         Filters mutations that may cause infinite loops by removing calls to iterator.next
10:49:49 PM PIT >> INFO : +FANN           Filters mutations in classes and methods with matching annotations of class or runtime retention
10:49:49 PM PIT >> INFO :   [annotation]    Annotation to avoid (full package name not required)
10:49:49 PM PIT >> INFO : -CLASSLIMIT     Limits the maximum number of mutations per class
10:49:49 PM PIT >> INFO :   [limit]         Integer value for maximum mutations to create per class
10:49:49 PM PIT >> INFO : -EXPORT         Exports mutants bytecode and other details to disk
10:49:49 PM PIT >> INFO : ---------------------------------------------------------------------------
10:49:49 PM PIT >> FINE : Running report with ReportOptions [targetClasses=[com.reedoei.eunomia.math.*, com.reedoei.eunomia.io.*, com.reedoei.eunomia.ast.*, com.reedoei.eunomia.data.*, com.reedoei.eunomia.util.*, com.reedoei.eunomia.functional.*, com.reedoei.eunomia.string.*, com.reedoei.eunomia.string.searching.*, com.reedoei.eunomia.string.matching.*, com.reedoei.eunomia.ast.resolved.*, com.reedoei.eunomia.latex.*], excludedMethods=[], excludedClasses=[], excludedTestClasses=[], codePaths=[/home/roei/Java/eunomia/target/classes], reportDir=/home/roei/Java/eunomia/target/pit-reports, historyInputLocation=null, historyOutputLocation=null, sourceDirs=[/home/roei/Java/eunomia/src/main/java, /home/roei/Java/eunomia/src/test/java], classPathElements=[/home/roei/Java/eunomia/target/test-classes, /home/roei/Java/eunomia/target/classes, /home/roei/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar, /home/roei/.m2/repository/org/dom4j/dom4j/2.1.0/dom4j-2.1.0.jar, /home/roei/.m2/repository/jaxen/jaxen/1.1.6/jaxen-1.1.6.jar, /home/roei/.m2/repository/com/github/javaparser/javaparser-core/3.6.10/javaparser-core-3.6.10.jar, /home/roei/.m2/repository/com/github/javaparser/javaparser-symbol-solver-core/3.6.10/javaparser-symbol-solver-core-3.6.10.jar, /home/roei/.m2/repository/com/github/javaparser/javaparser-symbol-solver-logic/3.6.10/javaparser-symbol-solver-logic-3.6.10.jar, /home/roei/.m2/repository/org/javassist/javassist/3.22.0-GA/javassist-3.22.0-GA.jar, /home/roei/.m2/repository/com/github/javaparser/javaparser-symbol-solver-model/3.6.10/javaparser-symbol-solver-model-3.6.10.jar, /home/roei/.m2/repository/org/eclipse/jgit/org.eclipse.jgit/4.10.0.201712302008-r/org.eclipse.jgit-4.10.0.201712302008-r.jar, /home/roei/.m2/repository/com/jcraft/jsch/0.1.54/jsch-0.1.54.jar, /home/roei/.m2/repository/com/googlecode/javaewah/JavaEWAH/1.1.6/JavaEWAH-1.1.6.jar, /home/roei/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar, /home/roei/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar, /home/roei/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar, /home/roei/.m2/repository/commons-codec/commons-codec/1.9/commons-codec-1.9.jar, /home/roei/.m2/repository/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.jar, /home/roei/.m2/repository/junit/junit/4.12/junit-4.12.jar, /home/roei/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-embedder/3.5.3/maven-embedder-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-settings/3.5.3/maven-settings-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-settings-builder/3.5.3/maven-settings-builder-3.5.3.jar, /home/roei/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.24/plexus-interpolation-1.24.jar, /home/roei/.m2/repository/org/apache/maven/maven-core/3.5.3/maven-core-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-repository-metadata/3.5.3/maven-repository-metadata-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-artifact/3.5.3/maven-artifact-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-resolver-provider/3.5.3/maven-resolver-provider-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/resolver/maven-resolver-impl/1.1.1/maven-resolver-impl-1.1.1.jar, /home/roei/.m2/repository/org/apache/maven/resolver/maven-resolver-spi/1.1.1/maven-resolver-spi-1.1.1.jar, /home/roei/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.3/org.eclipse.sisu.inject-0.3.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-plugin-api/3.5.3/maven-plugin-api-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-model/3.5.3/maven-model-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-model-builder/3.5.3/maven-model-builder-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/maven-builder-support/3.5.3/maven-builder-support-3.5.3.jar, /home/roei/.m2/repository/org/apache/maven/resolver/maven-resolver-api/1.1.1/maven-resolver-api-1.1.1.jar, /home/roei/.m2/repository/org/apache/maven/resolver/maven-resolver-util/1.1.1/maven-resolver-util-1.1.1.jar, /home/roei/.m2/repository/org/apache/maven/shared/maven-shared-utils/3.2.1/maven-shared-utils-3.2.1.jar, /home/roei/.m2/repository/com/google/inject/guice/4.0/guice-4.0-no_aop.jar, /home/roei/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar, /home/roei/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar, /home/roei/.m2/repository/javax/annotation/jsr250-api/1.0/jsr250-api-1.0.jar, /home/roei/.m2/repository/org/codehaus/plexus/plexus-utils/3.1.0/plexus-utils-3.1.0.jar, /home/roei/.m2/repository/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.jar, /home/roei/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.plexus/0.3.3/org.eclipse.sisu.plexus-0.3.3.jar, /home/roei/.m2/repository/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar, /home/roei/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.7.1/plexus-component-annotations-1.7.1.jar, /home/roei/.m2/repository/org/sonatype/plexus/plexus-sec-dispatcher/1.4/plexus-sec-dispatcher-1.4.jar, /home/roei/.m2/repository/org/sonatype/plexus/plexus-cipher/1.7/plexus-cipher-1.7.jar, /home/roei/.m2/repository/commons-cli/commons-cli/1.4/commons-cli-1.4.jar, /home/roei/.m2/repository/org/apache/commons/commons-lang3/3.7/commons-lang3-3.7.jar, /home/roei/.m2/repository/com/google/guava/guava/25.1-jre/guava-25.1-jre.jar, /home/roei/.m2/repository/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar, /home/roei/.m2/repository/com/google/errorprone/error_prone_annotations/2.1.3/error_prone_annotations-2.1.3.jar, /home/roei/.m2/repository/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar, /home/roei/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.14/animal-sniffer-annotations-1.14.jar, /home/roei/.m2/repository/org/checkerframework/checker-qual/2.5.2/checker-qual-2.5.2.jar, /home/roei/.m2/repository/org/checkerframework/jdk8/2.5.2/jdk8-2.5.2.jar, /home/roei/.m2/repository/org/pitest/pitest/1.4.0/pitest-1.4.0.jar], mutators=[], features=[], dependencyAnalysisMaxDistance=-1, jvmArgs=[], numberOfThreads=1, timeoutFactor=1.25, timeoutConstant=3000, targetTests=[], loggingClasses=[], maxMutationsPerClass=0, verbose=true, failWhenNoMutations=true, outputs=[HTML], groupConfig=TestGroupConfig [excludedGroups=[], includedGroups=[]], mutationUnitSize=0, shouldCreateTimestampedReports=true, detectInlinedCode=true, exportLineCoverage=false, mutationThreshold=0, coverageThreshold=0, mutationEngine=gregor, javaExecutable=null, includeLaunchClasspath=true, properties={}, maxSurvivors=0, excludedRunners=[], testPlugin=junit, includedTestMethods=[]]
10:49:49 PM PIT >> FINE : System class path is /usr/share/maven/boot/plexus-classworlds-2.x.jar
10:49:49 PM PIT >> FINE : Maximum available memory is 3509 mb
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.674 s
[INFO] Finished at: 2018-06-20T22:49:49-05:00
[INFO] Final Memory: 18M/375M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.pitest:pitest-maven:1.4.0:mutationCoverage (default-cli) on project eunomia: Execution default-cli of goal org.pitest:pitest-maven:1.4.0:mutationCoverage failed: 137 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.pitest:pitest-maven:1.4.0:mutationCoverage (default-cli) on project eunomia: Execution default-cli of goal org.pitest:pitest-maven:1.4.0:mutationCoverage failed: 137
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal org.pitest:pitest-maven:1.4.0:mutationCoverage failed: 137
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    ... 20 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 137
    at org.pitest.reloc.asm.ClassReader.readLabel(ClassReader.java:2406)
    at org.pitest.reloc.asm.ClassReader.createLabel(ClassReader.java:2422)
    at org.pitest.reloc.asm.ClassReader.readTypeAnnotations(ClassReader.java:2488)
    at org.pitest.reloc.asm.ClassReader.readCode(ClassReader.java:1652)
    at org.pitest.reloc.asm.ClassReader.readMethod(ClassReader.java:1238)
    at org.pitest.reloc.asm.ClassReader.accept(ClassReader.java:631)
    at org.pitest.reloc.asm.ClassReader.accept(ClassReader.java:355)
    at org.pitest.classinfo.ClassInfoVisitor.getClassInfo(ClassInfoVisitor.java:46)
    at org.pitest.classinfo.Repository.nameToClassInfo(Repository.java:70)
    at org.pitest.classinfo.Repository.fetchClass(Repository.java:60)
    at org.pitest.classinfo.NameToClassInfo.apply(NameToClassInfo.java:17)
    at org.pitest.classinfo.NameToClassInfo.apply(NameToClassInfo.java:7)
    at java.util.function.Function.lambda$andThen$1(Function.java:88)
    at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.pitest.classpath.CodeSource.getCode(CodeSource.java:43)
    at org.pitest.mutationtest.verify.DefaultBuildVerifier.verify(DefaultBuildVerifier.java:32)
    at org.pitest.mutationtest.tooling.MutationCoverage.verifyBuildSuitableForMutationTesting(MutationCoverage.java:223)
    at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:111)
    at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:116)
    at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:49)
    at org.pitest.maven.RunPitStrategy.execute(RunPitStrategy.java:35)
    at org.pitest.maven.AbstractPitMojo.analyse(AbstractPitMojo.java:436)
    at org.pitest.maven.AbstractPitMojo.execute(AbstractPitMojo.java:377)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    ... 21 more
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
hcoles commented 6 years ago

@ReedOei Thanks for the report. I've just reproduced this locally. At first glance it looks like an issue in the ASM library on which pitest depends, but a bit more digging will be required.

hcoles commented 6 years ago

I contacted the ASM team - this is in fact due to a bug in the java 8 compiler.

It is fixed in Java 8u172 (I've not confirmed this) and Java 9 (this I have confirmed - the bug does not manifest when compiled with java 9).

The ASM team position is that they will not support invalid bytecode generated by compiler bugs (which is reasonable).

It is possible to add a workaround for this into pitest so it ignores the invalid bytecode, or to have it fail with an improved error message. I'm not certain if it is worth the effort however as the 172 release begin to spread and replace 171 over the coming months.

ReedOei commented 6 years ago

@hcoles Thanks for looking into this!

I'll try using update 172 to see if that fixes it. Unfortunately I can't use Java 9 for this project, but it's good to know that it shouldn't be a problem once I am able to upgrade.

ReedOei commented 6 years ago

Can confirm that Java 8 update 172 fixes the problem for me. Thanks again!