search

EvoSuite / evosuite

EvoSuite - automated generation of JUnit test suites for Java classes
http://www.evosuite.org
GNU Lesser General Public License v3.0
832 stars 341 forks source link

Problem with `-prefix` option #372

Closed jiseongg closed 3 years ago

jiseongg commented 3 years ago

Context

When following the Tutorial_Experiments, I found the errors in some instruction.

Steps to Reproduce

I've written scripts reproducing this error. All code block here can be executed with copy-and-paste. I suppose that -prefix option has a problem. I found it when investigating the Tutorial_Experiments

1. Install EVOSUITE

# install evosuite
git clone https://github.com/evosuite/evosuite
cd evosuite
mvn clean install -DskipTests -DskipITs

2. Download tutorials

# download tutorial
wget https://evosuite.org/files/tutorial/Tutorial_Experiments.zip
unzip Tutorial_Experiments.zip

# Change evosuite-plugins version from 1.0.3 to 1.0.6 in pom.xml

Here, I've changed the version of evosuite-standalone-runtime and evosuite-maven-plugin in pom.xml file from 1.0.3 to 1.0.6, since there is a build failure with 1.0.3.

3. Run EVOSUITE

In this step, there is a branch for two cases, one occurring errors and the other without errors. For the command, I've followed the instruction in the section Running an experiment of Tutorial_Experiments First one:

export EVOSUITE="java -jar $HOME/.m2/repository/org/evosuite/evosuite-master/1.1.1-SNAPSHOT/evosuite-master-1.1.1-SNAPSHOT.jar"

cd Tutorial_Experiments
$EVOSUITE -setup target/classes target/dependency/commons-collections-3.2.2.jar

# this does not work
$EVOSUITE -criterion branch -prefix tutorial -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore

The output of the first case is like below

* EvoSuite 1.1.1-SNAPSHOT * Creating new evosuite.properties in /home/vagrant/Tutorial_Experiments/evosuite-files * Classpath: target/classes:target/dependency/commons-collections-3.2.2.jar * EvoSuite 1.1.1-SNAPSHOT * Analyzing classpath (generating inheritance tree) - target/classes - target/dependency/commons-collections-3.2.2.jar * Found 9 matching classes for prefix tutorial * Current class: tutorial.Bank ... * Computation finished [MASTER] 07:07:40.016 [logback-2] ERROR ClientNodeImpl - Error when generating tests for: tutorial.Bank with seed 1627974458963. Configuration id : null java.lang.IllegalArgumentException: Unsupported type of fitness function: class org.evosuite.coverage.branch.BranchCoverageSuiteFitness at org.evosuite.ga.metaheuristics.TestSuiteAdapter.mapFitnessFunctionToTestCaseLevel(TestSuiteAdapter.java:646) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) at org.evosuite.ga.metaheuristics.TestSuiteAdapter.addFitnessFunctions(TestSuiteAdapter.java:653) at org.evosuite.strategy.WholeTestSuiteStrategy.generateTests(WholeTestSuiteStrategy.java:67) at org.evosuite.TestSuiteGenerator.generateTests(TestSuiteGenerator.java:642) at org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:220) at org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:150) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [MASTER] 07:07:40.130 [main] ERROR SearchStatistics - No statistics has been saved because EvoSuite failed to generate any test case [MASTER] 07:07:40.263 [main] ERROR TestGeneration - failed to write statistics data ...

However, if I run without -prefix option, manually specifying each class name, it works correctly. Here is this second case:

export EVOSUITE="java -jar $HOME/.m2/repository/org/evosuite/evosuite-master/1.1.1-SNAPSHOT/evosuite-master-1.1.1-SNAPSHOT.jar"

cd Tutorial_Experiments
$EVOSUITE -setup target/classes target/dependency/commons-collections-3.2.2.jar

# this works
$EVOSUITE -criterion branch -class tutorial.ATM -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.ATMCard -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.Bank -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.BankAccount -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.Owner -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.CurrentAccount -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.SavingsAccount -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.Company -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore
$EVOSUITE -criterion branch -class tutorial.Person -Doutput_variables=TARGET_CLASS,criterion,Size,Length,MutationScore

The output is normal like below

* EvoSuite 1.1.1-SNAPSHOT * Creating new evosuite.properties in /home/vagrant/Tutorial_Experiments/evosuite-files * Classpath: target/classes:target/dependency/commons-collections-3.2.2.jar * EvoSuite 1.1.1-SNAPSHOT * Going to generate test cases for class: tutorial.ATM * Starting Client-0 * Properties loaded from /home/vagrant/Tutorial_Experiments/evosuite-files/evosuite.properties * Connecting to master process on port 4635 * Analyzing classpath: - target/classes - target/dependency/commons-collections-3.2.2.jar * Finished analyzing classpath * Generating tests for class tutorial.ATM * Test criterion: - Branch Coverage * Total number of test goals for DYNAMOSA: 20 * Using seed 1627973535788 [Progress:> 0%] [Cov:> 0%] Starting evolution * Initial Number of Goals in DynaMOSA = 10 / 20 [Progress:> 1%] [Cov:===================================100%] * Search finished after 2s and 8 generations, 10671 statements, best individual has fitness: 0.0 * Minimizing test suite * Going to analyze the coverage criteria ... *Writing tests to file * Writing JUnit test case 'ATM_ESTest' to evosuite-tests \ Done!

* Computation finished ... (success for all classes)

Thus, I think there is an error in processing -prefix option.

Expected result

Running with -prefix <package> name should have same results with running with manually specifying all classes in <package>.

Additional info

Platform Information