testcase in report for dotnet framework

[rouke-broersma]

Is your feature request related to a problem? Please describe. With #1850 we will be able to display test files in the json and html report. We will also be able to display testcases and which mutants they cover for dotnet (core). For dotnet framework we can't yet extract the testcases. This is because the CodeFilePath is null, so we can't match the testcase back to the testfile it belongs to.

See ProjectMutator.EnrichTestProjectsWithTestInfo

We should figure out how to get this information so dotnet framework unit tests are also visible in the report.

[dupdob]

Interestingly, VS does report those information properly. I tried altering the discovery options but failed so far. I am experiencing other problems with test discovery in solution mode, and I think this doc page is a good source to understand the issue: https://github.com/microsoft/vstest/blob/main/docs/RFCs/0013-Test-Adapter-Lookup.md

In short, VS IDE identifies the test adapter using a different strategy than VsTest, and VsTest does not like having different versions for the same test runner.

[dupdob]

I have soem bad news, I am afraid. While digging into VsTest code, I ran into this page: https://github.com/microsoft/vstest-docs/blob/main/RFCs/0010-Source-Information-For-Discovered-Tests.md

It explains that when VsTest can not resolve source information thanks to Roslyn, the VS Test Explorer then digs into PDB files. Which means, Stryker has to do the same

[rouke-broersma]

@dupdob it doesn't explicitly list however that roslyn cannot be used for dotnet framework. It only differentiates between managed and unmanaged test projects.

I'm not convinced dotnet framework test projects would be called unmanaged in this context, but if that's the case I would rather just not support test discovery on dotnet framework 🤷

[dupdob]

Granted, I may have jumped to conclusions here. But, this matches so much what we experience, that it looks like a fair assumption