A way to baseline / "grandfather" existing code in

[arudolph-cxt]

Is your feature request related to a problem? Please describe. When trying to include infection into an existing project, the amount of issues might be too overwhelming to deal with right from the get go. Makes it hard/impossible to still allow sane min values be set for the ci pipeline to pass.

Describe the solution you'd like it would be beneficial to add a baseline feature similar to what psalm offers, that ignores issues from old code and only takes new code into account. Instead of adding all files to an ignorelist or adding a feature that only considers files starting at a certain timestamp, a generated ignorelist that references concrete code parts/mutations could be of great help.

Describe alternatives you've considered Alternatives that come to mind that would work with the current featureset:

• Adding all current files to the ignorelist. The programmer then would have to remember to remove edited files from the ignorelist each time new code is added, thus also adding old code into the mix that happens to be within the file
• Running the test and determining the current percentage and setting this as a min baseline, requiring via ci-pipeline new code to be "at least better". Has the drawbacks of programmers having to adjust the percentage every once in a while, might cause issues with the percentage going below the baseline for legit mutation cases. Also I feel this would defeat the purpose of the tool

Additional context See psalm documentation for the related feature: https://psalm.dev/docs/dealing_with_code_issues/#using-a-baseline-file

[BackEndTea]

I like this feature in tools like psalm/phpstan etc. However, i don't think it is a good fit for infection.

For static analysis tools, its all or nothing, the checks either pass or don't. And if 1 check doesn't pass, then the process 'fails'. Infection doesn't require everything to pass. In fact, under default circumstances, literally every mutation could fail, and infeciton would still 'pass'.

We already have the ability to set a 'baseline' so to speak, by using --min-msi or the 'covered' variant. I know this isn't exaclty like the feature you are suggesting, since it does allow adding new 'faults'. But it does require that the percentage at least stays the same.

[arudolph-cxt]

My worry with this is that there are legit cases where the min-msi could be reduced without adding bad code or bad tests, just by virtue of the code being susceptible to mutations that dont change its logic

as outlined here: https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1#ecda

[sanmai]

Related: https://github.com/infection/infection/issues/669#issuecomment-475530312

[Danack]

Apologies in advance if advertising other packages is verboten.

There is a base-lining tool available from here: https://github.com/DaveLiddament/sarb

It doesn't currently support base-lining for Infection, but according to the maintainer, it should be possible to add that.

The reason I'm mentioning it is that it seems to me that the rules surrounding a 'base-lining'/'grandfathering' existing code is something that's going to need to work across all of the tools that analyze code for quality.

Adding this piecemeal to each code quality project sounds like it could produce a huge duplication of effort. Adding it to a specific baselining tool could avoid a large amount of work.

[Slamdunk]

@BackEndTea my 2 cents on this topic: I found way easier and quicker to set minimum score to 100% and fill ignoreSourceCodeByRegex to reach 100% where necessary, then setting minimum score below 100%.

Because mathematically, below 100% you are automatically allowing (i.e. go unnoticed) new escaped mutant on new line-of-codes proportionally to the missing percentage.

[Slamdunk]

Hi, with the landing of --git-diff-lines and given the nature of this tool, we dropped the need for a baseline altogether and went for --git-diff-lines --min-msi=100 for our PRs.

[maks-rafalko]

Agree, I have the same thoughts: do we really need baselining when we have --git-diff-lines --min-msi=100 now?