When fed by the same random seed Evosuite produces, in some situations, different results. I have experienced this loss in determinism since version 1.1.1 with a modified version of Evosuite that we are using for our test generator TARDIS, but the issue can be tracked to the unmodified Evosuite source code. I detected a source of nondeterminism in the use of the Collectors.toSet() method to gather the assignable classes from a test cluster in the code of the Set<Class<?>> ClassCastManager.getAssignableClassesFromTestCluster(Predicate<Class<?>>) method:
The Collectors.toSet() collector is unordered, as reported in the documentation: Because of this, the instantiation of the generic parameters during the creation of a test may picks different concrete classes under different runs of Evosuite with same random seed. I suggest to fix this issue by replacing line 325 with
When fed by the same random seed Evosuite produces, in some situations, different results. I have experienced this loss in determinism since version 1.1.1 with a modified version of Evosuite that we are using for our test generator TARDIS, but the issue can be tracked to the unmodified Evosuite source code. I detected a source of nondeterminism in the use of the
Collectors.toSet()method to gather the assignable classes from a test cluster in the code of theSet<Class<?>> ClassCastManager.getAssignableClassesFromTestCluster(Predicate<Class<?>>)method:https://github.com/EvoSuite/evosuite/blob/8730aedd6cbcbeac1d7f246b4f223bda395212c9/client/src/main/java/org/evosuite/seeding/CastClassManager.java#L325
The
Collectors.toSet()collector is unordered, as reported in the documentation: Because of this, the instantiation of the generic parameters during the creation of a test may picks different concrete classes under different runs of Evosuite with same random seed. I suggest to fix this issue by replacing line 325 withassuming that the source of the stream is deterministically ordered.