STAMP-project / EvoCrash-demo

See https://stamp-project.github.io/botsing/ for the latest version of STAMP crash reproduction tool.
https://stamp-project.github.io/botsing/
Apache License 2.0
1 stars 2 forks source link

Generated test doesn't have any content #12

Closed MaelAudren closed 6 years ago

MaelAudren commented 6 years ago

The test is generated with a frame.level parameter at 6.

Generated test:

/*
 * This file was automatically generated by EvoSuite
 * Wed Jun 13 16:05:32 CEST 2018
 */

package org.ow2.proactive.scripting;

import org.junit.Test;
import static org.junit.Assert.*;
import org.evosuite.runtime.EvoRunner;
import org.evosuite.runtime.EvoRunnerParameters;
import org.junit.runner.RunWith;

@RunWith(EvoRunner.class) @EvoRunnerParameters(useVFS = true, useJEE = true) 
public class ScriptHandler_ESTest extends ScriptHandler_ESTest_scaffolding {
  @Test
  public void notGeneratedAnyTest() {
    // EvoSuite did not generate any tests
  }
}

Log

org.ow2.proactive.scripting.ScriptException:
    at org.python.jsr223.PyScriptEngine.scriptException(PyScriptEngine.java:202)
    at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:42)
    at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:47)
    at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:249)
    at org.ow2.proactive.scripting.Script.execute(Script.java:312)
    at org.ow2.proactive.scripting.ScriptHandler.handle(ScriptHandler.java:65)
    at org.ow2.proactive.scheduler.task.executors.InProcessTaskExecutor.execute(InProcessTaskExecutor.java:232)

Stack trace

Best individual has fitness: 3.0
Worst individual has fitness: 3.0

* EvoCrash: the generated stack trace:

* EvoCrash: the generated test case:
EvoSuiteFile evoSuiteFile0 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile1 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile2 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile3 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile4 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile5 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile6 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile7 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile8 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile9 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile10 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile11 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile12 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile13 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile14 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile15 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile16 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile17 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile18 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile19 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile20 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile21 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile22 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile23 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile24 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile25 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile26 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile27 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile28 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile29 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile30 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile31 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile32 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile33 = new EvoSuiteFile("/home/mael/stamp/wp3/EvoCrash-demo/Demo");
EvoSuiteFile evoSuiteFile34 = null;
PrintStream printStream0 = null;
PrintStream printStream1 = null;
Script<HijrahDate> script0 = null;
ScriptHandler scriptHandler0 = new ScriptHandler();
PrintStream printStream2 = null;
PrintStream printStream3 = null;
Script<Integer> script1 = null;
scriptHandler0.handle(script1, printStream3, printStream0);

>>>>>>>>>>>>>>>>>>>>>>>>>>>GGA was done in 601!<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

! Budget exceeded
! Unable to cover goal 0 org.crash.client.crashcoverage.CrashCoverageTestFitness@329e43b
* Search finished after 604s, 604 statements, best individual has fitness 0.0
* Covered 0/1 goals
* Target exception was set to: org.ow2.proactive.scripting.ScriptException
* EvoCrash: The target call is either public or protected!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The retrived call(s) to inject in the tests are:
>>>>>> handle
[MASTER] 15:47:12.794 [logback-1] WARN  TimeController - Phase INITIALIZATION lasted too long, 493 seconds more than allowed.
* Target exception was set to: org.ow2.proactive.scripting.ScriptException
* EvoCrash: The target call is either public or protected!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The retrived call(s) to inject in the tests are:
>>>>>> handle
* Minimizing test suite
* Going to analyze the coverage criteria
* Coverage analysis for criterion CRASH
* Target exception was set to: org.ow2.proactive.scripting.ScriptException
* EvoCrash: The target call is either public or protected!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The retrived call(s) to inject in the tests are:
>>>>>> handle
* Coverage of criterion CRASH: 0 %
* Total number of goals: 1
* Number of covered goals: 0
* Generated 0 tests with total length 0
* Resulting test suite's coverage: 0 %
* Generating assertions
* Resulting test suite's mutation score: 0 %
* Compiling and checking tests
* Permissions denied during test execution:
    - java.io.FilePermission:
        execute <<ALL FILES>>: 7
    - java.lang.RuntimePermission:
        loadLibrary.libsigar-amd64-linux.so: 1
    writeFileDescriptor: 36
* Writing JUnit test case 'ScriptHandler_ESTest' to src/GGA-tests/java
* Done!

* Computation finished
Exception in thread "main" java.lang.AssertionError: expected:<0.0> but was:<3.0>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:834)
    at org.junit.Assert.assertEquals(Assert.java:553)
    at org.junit.Assert.assertEquals(Assert.java:683)
    at eu.stamp_project.evocrash.demo.TestCreation.create(TestCreation.java:97)
    at eu.stamp_project.evocrash.demo.Application.main(Application.java:6)
MaelAudren commented 6 years ago

I had the issue with several other test/log/frame.level and I can provide them if they can highlight the issue.

pderakhshanfar commented 6 years ago

The fitness function should be 0. Before that, EvoCrash cannot generate a test for crash replication. As we can see in the output that you sent, the Best individual has fitness: 3.0

MaelAudren commented 6 years ago

@pderakhshanfar Is there a way to play with the parameters in order to generate a correct test ?

pderakhshanfar commented 6 years ago

Since EvoCrash is generating test cases randomly, even without changing configurations we may have some crash replications. The point is that we should apply EvoCrash on more than one stack trace. Because it can be much easier to replicate one stack trace (in some specific target frame) than doing it for another stack trace.