Failed test issue

[slauriere]

When executing the tests via "mvn test", one test fails with the stacktrace below. Environment: Java 8, Maven 3.3.9, Ubuntu.

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.593 sec - in fr.inria.diversify.mutant.pit.PitTest

Results :

Failed tests: 
  AssertGeneratorTest.testGenerateAssert:48 expected:<...sWithBoolean();
    [// AssertGenerator add assertion
    org.junit.Assert.assertTrue(((fr.inria.sample.ClassWithBoolean)cl).getBoolean());
    // AssertGenerator add assertion
    org.junit.Assert.assertTrue(((fr.inria.sample.ClassWithBoolean)cl).getTrue());
    // AssertGenerator add assertion
    org.junit.Assert.assertFalse(((fr.inria.sample.ClassWithBoolean)cl).getFalse());
    ]cl.getFalse();
    c...> but was:<...sWithBoolean();
    []cl.getFalse();
    c...>

Tests in error: 
  MethodsAssertGeneratorTest.testBuildNewAssert:68 » UnsupportedOperation

Tests run: 82, Failures: 1, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:31 min
[INFO] Finished at: 2017-08-29T17:45:01+02:00
[INFO] Final Memory: 21M/254M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project dspot: There are test failures.

[danglotb]

Hello @slauriere

It is quite stange, because DSpot does not throw any UnsupportedOperationException. It must be from another component.

Could please provide the whole stack trace of the exception thrown by MethodsAssertGeneratorTest.testBuildNewAssert?

Thank

[slauriere]

hi @danglotb. I'm not sure how to get the whole stack trace, can you help? (I've tried to add the -X option to mvn, but I didn't get more). Thanks a lot.

[danglotb]

You may have to scroll up, and find it at the execution of the test.

Or maybe you could send me the whole trace of the mvn test CMD?

Thank you! :)

[slauriere]

OK! Here is the trace of mvn -X test: output.txt

[danglotb]

Okay, you have some compilation problems and I don't know why.

I'll build branch with a better verbose mode to investigate the problem.

Could please indicate which Java8 you use?

Thank!

[slauriere]

Great, thanks. Here is the Java version I'm using:

openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

[slauriere]

I also gave a try with the Java version below and I got the same error:

java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

testBuildNewAssert(fr.inria.diversify.dspot.assertGenerator.MethodsAssertGeneratorTest)  Time elapsed: 0.172 sec  <<< ERROR!
java.lang.UnsupportedOperationException: null
    at java.util.Collections$1.remove(Collections.java:4684)
    at java.util.AbstractCollection.removeAll(AbstractCollection.java:376)
    at fr.inria.diversify.dspot.support.TestCompiler.compileAndRun(TestCompiler.java:46)
    at fr.inria.diversify.dspot.assertGenerator.MethodsAssertGenerator.generateAsserts(MethodsAssertGenerator.java:54)
    at fr.inria.diversify.dspot.assertGenerator.MethodsAssertGeneratorTest.testBuildNewAssert(MethodsAssertGeneratorTest.java:68)

Running fr.inria.diversify.dspot.assertGenerator.AssertGeneratorTest
02:18 INFO: Run tests. (2)
02:18 WARN: 4 errors during compilation, discarding involved test methods
02:19 DEBUG: 0 new tests with assertions generated
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.358 sec <<< FAILURE! - in fr.inria.diversify.dspot.assertGenerator.AssertGeneratorTest
testGenerateAssert(fr.inria.diversify.dspot.assertGenerator.AssertGeneratorTest)  Time elapsed: 0.358 sec  <<< FAILURE!
org.junit.ComparisonFailure: expected:<...sWithBoolean();
    [// AssertGenerator add assertion
    org.junit.Assert.assertTrue(((fr.inria.sample.ClassWithBoolean)cl).getBoolean());
    // AssertGenerator add assertion
    org.junit.Assert.assertTrue(((fr.inria.sample.ClassWithBoolean)cl).getTrue());
    // AssertGenerator add assertion
    org.junit.Assert.assertFalse(((fr.inria.sample.ClassWithBoolean)cl).getFalse());
    ]cl.getFalse();
    c...> but was:<...sWithBoolean();
    []cl.getFalse();
    c...>
    at fr.inria.diversify.dspot.assertGenerator.AssertGeneratorTest.testGenerateAssert(AssertGeneratorTest.java:48)

[danglotb]

Hi,

It seems it is something else than the java version because we are running the same...

I create a new branch on my own fork, you would mine to run test using it? at the root of dspot:

git remote add danglotb http://github.com/danglotb/dspot.git
git fetch danglotb
git checkout verbose-mode
mvn clean test

and send me back the trace?

I am sorry of this way of debugging, but since I am not able to reproduce the error and no clue where it can come from, I do not know how to do in any other way.

Thank you @slauriere !

[slauriere]

Sure @danglotb, here it is: mvn-log.txt HTH

[monperrus]

duplicate of #98