Open alisevych opened 1 year ago
@korifey @denis-fokin @Markoutte May be the team can consider a common solution for handling changing data? That can be Random, Date/DateTime, data from files/other resources, etc. How to determine them? and how to mock?
We should definitely reproduce every test, traversed by concrete executor. My proposal is to enhance concrete executor. It must automatically fixate behavior of certain methods and return back information to engine information about it.
E.g. if concolic executor find some new Random()
it should return value to engine that will generate static mocking for this case.
@SBOne-Kenobi could you try to elaborate this idea.
concolic executor find some new Random()
Also need to find Random usages. And other Random implementations can be used.
@korifey Would it be too long to rerun every test generated by Fuzzing? - to check if received data is changing from run to run
Also, there is the same problem with Random when generating tests for vertexSum method from recursion package. It is worth to mention that it reproduces not every time. Steps to reproduce:
One (the test below) of the two generated tests will fail when running because of Random usage inside vertexSum method.
///region FUZZER: SUCCESSFUL EXECUTIONS for method vertexSum(int)
/**
* @utbot.classUnderTest {@link Recursion}
* @utbot.methodUnderTest {@link Recursion#vertexSum(int)}
*/
@Test
@DisplayName("vertexSum: n = 134217732 (mutated from 4) -> return 293591018")
public void testVertexSum() {
Recursion recursion = new Recursion();
int actual = recursion.vertexSum(134217732);
assertEquals(293591018, actual);
}
///endregion
Description
Fuzzing is generating a test for Random without mocks - when Mock everything outside class is checked. The test asserts the value returned by Random and fails.
To Reproduce
Expected behavior
There should be only valid tests generated.
Possible solutions
Fuzzer can use mock Random(?) Generated tests should be double-checks that they return same results(?) other solution(?)
Actual behavior
Fuzzing is generating a test for Random without mocks. It asserts the value returned by Random and fails.
Visual proofs
The following test is generated by Fuzzer:
Environment
Windows 10 Pro IntelliJ IDEA 2022.2.2
Additional context
Originally posted by @alisevych in https://github.com/UnitTestBot/UTBotJava/issues/1102#issuecomment-1269804477