Hello, I'm extending EvoSuite in order to add an additional coverage criterion.
Without going into deep details, I can say that a single coverage goal is identified by a triple (targetClass, targetMethod, targetLine). My approach does not generate tests for targetClass, but for any classes that is able to reach targetClass, directly or indirectly.
So a common scenario is: CUT -> ... -> TargetClass
Most of the cases, instrumentation works fine: I can fetch all bytecode lines I need, I can extract CFGs and CDGs in order to build by coverage goals... but I think I found a particular case where EvoSuite has some problem (or maybe it's me that I don't properly set a secret Property).
Briefly:
I have a CUT A that should cover a target of class B. B is not called directly by A, in fact A passes though B2, one of its subclass.
B2 does nothing in particular: it just inherit targetMethod(), without modifyng it;
A calls the targetMethod() of B2 (that should directly reference targetMethod() of B);
For unknown reason, the instrumentation does not correctly record the fact that targetMethod() of B is called.
I try to summarize some of my console prints
When I run the above scenario, this is what comes out:
WARN org.evosuite.assertion.CheapPurityAnalyzer - org.example.hierarchy.BadChild was not found in the inheritance tree. Using DEFAULT value for cheap-purity analysis
It's just a simple warning and it always happen when I have a hierarchy in my classpath. The above example if problematic, but there are other instances where I use hierarchies without problems with the said message still present (later I will provide two examples of hierarchies).
I am able to successfully fetch other things (the X says that everything is okay, a missing X means that it is not okay):
[x] Bytecode of targetClass
[x] Bytecode of targetMethod
[x] Bytecode of targetLine
[x] Public Methods of targetClass (I don't need these and it's just for debugging)
[x] Is targetClass called? (debugging)
[ ] Is targetMethod called? (debugging... this is the first oddity)
[ ] CDG of targetClass + targetMethod (I really need this... this is the second oddity)
Whenever I try to get the CDG I get this error:
ERROR org.evosuite.rmi.service.ClientNodeImpl - Error when generating tests for: com.examples.with.different.packagename.myapp.MyCUTCallsJARDirectlyHierarchyV2 with seed 2. Configuration id : null
java.lang.IllegalStateException: expect GraphPool to know CDG for every method for which an instruction is known
at org.evosuite.graphs.cfg.BasicBlock.getCDG(BasicBlock.java:146) ~[classes/:na]
at org.evosuite.graphs.cfg.BasicBlock.getControlDependencies(BasicBlock.java:191) ~[classes/:na]
at org.evosuite.graphs.cfg.BytecodeInstruction.getControlDependencies(BytecodeInstruction.java:452) ~[classes/:na]
at org.evosuite.coverage.vulnerability.VulnerabilityFitnessFactory.getAllControlDependencies(VulnerabilityFitnessFactory.java:103) ~[classes/:na]
at org.evosuite.coverage.vulnerability.VulnerabilityFitnessFactory.getCoverageGoals(VulnerabilityFitnessFactory.java:73) ~[classes/:na]
at org.evosuite.coverage.vulnerability.VulnerabilitySuiteFitness.<init>(VulnerabilitySuiteFitness.java:22) ~[classes/:na]
at org.evosuite.coverage.FitnessFunctions.getFitnessFunction(FitnessFunctions.java:138) ~[classes/:na]
at org.evosuite.strategy.TestGenerationStrategy.getFitnessFunctions(TestGenerationStrategy.java:87) ~[classes/:na]
at org.evosuite.strategy.WholeTestSuiteStrategy.generateTests(WholeTestSuiteStrategy.java:68) ~[classes/:na]
at org.evosuite.TestSuiteGenerator.generateTests(TestSuiteGenerator.java:681) ~[classes/:na]
at org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:245) ~[classes/:na]
at org.evosuite.rmi.service.ClientNodeImpl$1.run(ClientNodeImpl.java:157) ~[classes/:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_242]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_242]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_242]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_242]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242]
ERROR org.evosuite.statistics.SearchStatistics - No statistics has been saved because EvoSuite failed to generate any test case
ERROR org.evosuite.executionmode.TestGeneration - failed to write statistics data
Steps to Reproduce
I have provided some example projects for this issue. Check here: https://github.com/emaiannone/exploit-generation/tree/fitness_experiments/examples.
Here you can find all steps (in README) to launch some tests on five artificial projects both with the JAR and JUnit tests
Current Result
As said in the README, you should see an error for the last example (Example 4-Version 2). You should a similar stacktrace of the one above.
Expected result
I expect to see something similar to Example 4-Version 1, but it's not happening :(
Additional info
If you need more clarification just write in the comments :)
Context
Hello, I'm extending EvoSuite in order to add an additional coverage criterion. Without going into deep details, I can say that a single coverage goal is identified by a triple (targetClass, targetMethod, targetLine). My approach does not generate tests for targetClass, but for any classes that is able to reach targetClass, directly or indirectly.
So a common scenario is:
CUT -> ... -> TargetClass
Most of the cases, instrumentation works fine: I can fetch all bytecode lines I need, I can extract CFGs and CDGs in order to build by coverage goals... but I think I found a particular case where EvoSuite has some problem (or maybe it's me that I don't properly set a secret Property).
Briefly:
targetMethod()
, without modifyng it;targetMethod()
of B2 (that should directly referencetargetMethod()
of B);targetMethod()
of B is called.I try to summarize some of my console prints
When I run the above scenario, this is what comes out:
It's just a simple warning and it always happen when I have a hierarchy in my classpath. The above example if problematic, but there are other instances where I use hierarchies without problems with the said message still present (later I will provide two examples of hierarchies).
I am able to successfully fetch other things (the X says that everything is okay, a missing X means that it is not okay):
Whenever I try to get the CDG I get this error:
Steps to Reproduce
I have provided some example projects for this issue. Check here:
https://github.com/emaiannone/exploit-generation/tree/fitness_experiments/examples
.Here you can find all steps (in README) to launch some tests on five artificial projects both with the JAR and JUnit tests
Current Result
As said in the README, you should see an error for the last example (Example 4-Version 2). You should a similar stacktrace of the one above.
Expected result
I expect to see something similar to Example 4-Version 1, but it's not happening :(
Additional info
If you need more clarification just write in the comments :)
Thank you!