In amplification the PhLDirectoryBasedImageRepositoryTest (Pharo-Launcher project) I saw some of the generated assertions are not valid in mutation coverage analysis.
The failure happens in the assertion generated to assert FileReference>>#modificationTime value. the value is flaky (it's different when the assertion is generated and when it's called in mutation testing).
In A-Amp stage, we have a step to detect flaky tests. In this step, we re-run the observation a couple of times and if any value is changed, we mark that as flaky value.
The problem here is the value doesn't change in the short term (for example it changes each second or each minute). So, it is failed to detect in the A-Amp stage. But, it returns other values in the selection stage.
In amplification the
PhLDirectoryBasedImageRepositoryTest(Pharo-Launcher project) I saw some of the generated assertions are not valid in mutation coverage analysis.The failure happens in the assertion generated to assert
FileReference>>#modificationTimevalue. the value is flaky (it's different when the assertion is generated and when it's called in mutation testing).In A-Amp stage, we have a step to detect flaky tests. In this step, we re-run the observation a couple of times and if any value is changed, we mark that as flaky value.
The problem here is the value doesn't change in the short term (for example it changes each second or each minute). So, it is failed to detect in the A-Amp stage. But, it returns other values in the selection stage.