Exception while generating test through MONOTONIC/RANDOM_SEARCH/LIPS using Evosuite-1.1.0

[neelofarhassan]

Context

1. When generating test with the version 1.1.0 using default algorithm (do not explicitly set -Dalgorithm), the statistics.csv reports the algorithm as "MONOTONIC_GA", however, test generation logs show that the algorithm is DYNAMOSA.
2. Furthermore, it throws an exception if I use LIPS, RANDOM_SEARCH or MONOTONIC_GA as a search algorithm.

Steps to Reproduce

Please break down here below all the needed steps to reproduce the issue. [If possible, please upload an example of the project you are generating tests for.]

I am trying to generate tests for the class "net.kencochrane.a4j.DAO.Product" from a4j in SF110 benchmark

EvoSuite Arguments

1. java -jar /Downloads/evosuite-1.1.0/master/target/evosuite-master-1.1.0.jar -class net.kencochrane.a4j.DAO.Product -projectCP build/classes/:lib -Dshow_progress=false -Djunit_check=false -Dfilter_assertions=false -Dtest_comments=false -Duse_separate_classloader=true -Doutput_variables=TARGET_CLASS,algorithm,criterion,Total_Branches,Covered_Branches,BranchCoverage,Lines,Covered_Lines,LineCoverage,MethodCoverage,Size,Length,MutationScore,Total_Goals,Covered_Goals -Dsearch_budget=120

2. java -jar /home/unimelb.edu.au/nneelofar/Downloads/evosuite-1.1.0/master/target/evosuite-master-1.1.0.jar -class net.kencochrane.a4j.DAO.Product -projectCP build/classes/:lib -Dshow_progress=false -Djunit_check=false -Dfilter_assertions=false -Dtest_comments=false -Duse_separate_classloader=true -Doutput_variables=TARGET_CLASS,algorithm,criterion,Total_Branches,Covered_Branches,BranchCoverage,Lines,Covered_Lines,LineCoverage,MethodCoverage,Size,Length,MutationScore,Total_Goals,Covered_Goals -Dsearch_budget=120 -Dalgorithm=MONOTONIC_GA

Current Result

Exception for 2:

• EvoSuite 1.1.0
• Going to generate test cases for class: net.kencochrane.a4j.DAO.Product
• Starting Client-0
• Properties loaded from /home/unimelb.edu.au/nneelofar/Documents/Matilda/software-testing/fault-localization/SF110-20130704-src/2_a4j/evosuite-files/evosuite.properties
• Connecting to master process on port 13818
• Analyzing classpath:
• Inheritance tree loaded from evosuite-files/inheritance.xml.gz [MASTER] 13:17:02.016 [logback-1] WARN InheritanceTree - Class not in inheritance graph: java.util.function.Consumer [MASTER] 13:17:02.020 [logback-1] WARN InheritanceTree - Class not in inheritance graph: java.util.function.Predicate [MASTER] 13:17:02.021 [logback-1] WARN InheritanceTree - Class not in inheritance graph: java.util.function.UnaryOperator [MASTER] 13:17:02.038 [logback-1] WARN InheritanceTree - Class not in inheritance graph: java.util.function.Function
• Finished analyzing classpath
• Generating tests for class net.kencochrane.a4j.DAO.Product
• Test criteria:
  • Line Coverage
  • Branch Coverage
  • Exception
  • Mutation testing (weak)
  • Method-Output Coverage
  • Top-Level Method Coverage
  • No-Exception Top-Level Method Coverage
  • Context Branch Coverage
• Computation finished [MASTER] 13:17:02.178 [logback-2] ERROR ClientNodeImpl - Error when generating tests for: net.kencochrane.a4j.DAO.Product with seed 1602209820730. Configuration id : null java.lang.UnsupportedOperationException: isMaximizationFunction() called on mock at org.evosuite.ga.FitnessFunctionMock.isMaximizationFunction(FitnessFunctionMock.java:62) at org.evosuite.ga.localsearch.DefaultLocalSearchObjective.addFitnessFunction(DefaultLocalSearchObjective.java:71) at org.evosuite.ga.metaheuristics.GeneticAlgorithm.addFitnessFunction(GeneticAlgorithm.java:436) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.evosuite.ga.metaheuristics.GeneticAlgorithm.addFitnessFunctions(GeneticAlgorithm.java:440) at org.evosuite.strategy.MOSuiteStrategy.generateTests(MOSuiteStrategy.java:89) at org.evosuite.TestSuiteGenerator.generateTests(TestSuiteGenerator.java:630) at org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:205) at org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:150) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [MASTER] 13:17:02.283 [main] ERROR SearchStatistics - No statistics has been saved because EvoSuite failed to generate any test case [MASTER] 13:17:02.385 [main] ERROR TestGeneration - failed to write statistics data

Thanks

[apanichella]

Hi @neelofarhassan, to use MONOTONIC_GA with Whole-suite generation, you would need to run the command java -jar evosuite.jat -generateSuite -Dalgorithm=MONOTONIC_GA <other args>

[neelofarhassan]

Thanks @apanichella,

What if I want to use MONOTONIC_GA with test cases generation instead of whole-suite? In the previous version, ONOTONIC_GA was the default algorithm and I always assumed that all other algorithms except WSA follow single-test generation strategy, until explicitly specified. Please correct me if I am wrong.

Could you please direct me to any documentation that explains the implementation of all the algorithms in evosuite (especially if it is different from the original implementation )?

[apanichella]

The default configuration (in the previous version of EvoSuite) was MONOTONIC_GA but for the whole-suite approach. A quite complete list of algorithms implemented in EvoSuite is described in the following articles:

A. Panichella, F. M. Kifetew, and P. Tonella. "A large scale empirical comparison of state-of-the-art search-based test case generators." Information and Software Technology 104 (2018): 236-256. J. Campos, Y. Ge, N. Albunian, G. Fraser, M. Eler, A. Arcuri, "An empirical evaluation of evolutionary algorithms for unit test suite generation", Information and Software Technology 104 (2018): 207-235,