Closed kwlin closed 4 years ago
This would mean a breaking change. Our "dotnet test" runner parses the console output for the number of tests executed. I've seen an example of a run on a franch system where the parser found 0 tests because the language was different in the console output.
If we stop stryker on 0 tests found, users on a non english system could not use stryker anymore.
I'm more fond of a less rigorous solution like logging a warning on no test cases found. For your case we should fix the vstest runner instead I think.
Logging a warning sounds like a good idea
@richardwerkman We could at the very least implement this for vstest. We could also look into using -t|--list-tests
to list the found testcases. Depending on the output of this command we might be able to more accurately detect the total number of unit tests independent of language.
If we stop stryker on 0 tests found, users on a non english system could not use stryker anymore.
That is a bug IMHO. We should be able to parse the lines on all languages. Maybe look for a number in the output, or something a bit stricter, but not exactly text matching. Or at least make the regex configurable, so that people can fix this themselves.
If you know for sure that 0 tests have run, then I would suggest the following:
Mutate like you normally would, but skip the running of the tests. Just directly report that all mutants survived because we couldn't find any tests. That is the most logical way of handling it, no need for a warning or abort.
Implemented by #506 and released since 0.12.0
When no initial tests found (because of missing nuget package (MSTest.TestAdapter)), stryker should abort the run instead.
For example: