stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.78k stars 187 forks source link

Ability to skip project mutation when source / test files did not change #2532

Open CzBuCHi opened 1 year ago

CzBuCHi commented 1 year ago

Is your feature request related to a problem? Please describe. When running stryker on large solution, stryker will mutate every single project it can find even if given project / project tests didnt change wastling time.

Describe the solution you'd like

Describe alternatives you've considered wrote my own implementation here

Additional context

rouke-broersma commented 1 year ago

Have you tried the baseline feature?

CzBuCHi commented 1 year ago

yes, but with no success :(

currently i have stryker.cmd file commited to git with this content:

@echo off
dotnet stryker -r progress -r html -o -O StrykerOutput
move StrykerOutput\reports\mutation-report.html SOLUTION_NAME.report.html
rmdir StrykerOutput /S /Q

this works as intended, but mutates everything everytime ...


to my understanding baseline require --version and --with-baseline arguments

i managed to run stryker with this:

-- grab sha of last commit in this dir and save it to baseline
FOR /F "tokens=* USEBACKQ" %%F IN (`git log -1 --format^=%%H -- .`) DO (
SET baseline=%%F
)

dotnet stryker -r progress -r html -o -O StrykerOutput --with-baseline:%baseline% --version:HEAD

now i have StrykerOutput\baseline\master\stryker-report.json file, but executing it again will result in mutating code again ... true, some mutants got status Result based on previous run but not all ...

also not sure hot to tell stryker to 'mutate all projects where any file is changed from last commit' (plan is to run stryker in pre-commit hook and cancel commit if stryker fails)

rouke-broersma commented 1 year ago

@CzBuCHi yes we do mutate everything every time, we don't currently have a mechanism to skip mutating. However we should skip mutation testing for the majority of mutants, which is the largest time save. We provide you with some information in the logs about which files were seen as changed, this can help you determine why perhaps the result is not what you're expecting.