Closed aarndt closed 8 months ago
@aarndt you can try:
dotnet test --collect "Code Coverage;CoverageFileName=merged.coverage" --results-directory D:\jakubch1\codecoverage\samples\Calculator\TestResults
but this will still generate final coverage report under path with random guid. Check: https://github.com/microsoft/vstest/issues/2378
IMO the best way to resolve it is use dotnet-coverage
for collection. This will give you also the best performance. Please check two examples:
https://github.com/microsoft/vstest/issues/2378#issuecomment-1713464163
https://github.com/microsoft/codecoverage/blob/main/samples/Calculator/scenarios/scenario21/README.md
Added more documentation about using dotnet-coverage
in this scenario:
https://github.com/microsoft/codecoverage/blob/main/samples/Calculator/scenarios/scenario25/README.md
@jakubch1 thank you for the quick response and helpful example. Much appreciated!
This worked well on my windows machine. However, an empty coverage file is generated when I run the command on a M2 ARM64 Macbook (some of our devs work with devcontainers on m1/m2 macs). I attempted this with both x64 and arm64 versions of the dotnet coverage
tool. As far as I can tell, using dotnet test --collect "CodeCoverage;Format=Cobertura"
then following it up with reportgenerator
to combine/convert the output is the only method that works in both scenarios. I suspect this is due to dynamic instrumentation not being supported on macOS arm64, but I must admit my understanding of this is limited.
Edit: For extra context, our use case is developers being able to easily measure and track code coverage in devcontainer-based projects in VS Code. I would like to create a task that generates a coverage report along with a coverage file that can be consumed by the coverage-gutters extension.
Yes, this is not working because macOS arm64 is not supported for dynamic instrumentation. So I recommend to do step back and use:
dotnet test --collect "Code Coverage;CoverageFileName=merged.coverage" --results-directory D:\jakubch1\codecoverage\samples\Calculator\TestResults
and then use some scripting to find final coverage report. dotnet test --collect
has some additional callback from Test Platform regarding dlls that will be tested and can by default statically instrument dlls (in envs where dynamic is not supported).
There is also a way to make dotnet-coverage
work in this scenario but I don't recommend it as it is quite complicated. Basically you have to build solution before running tests and then specify using --include-files all dlls that should be statically instrumented. So final command would be something like:
dotnet-coverage collect --include-files "**/bin/Debug/*.dll" -f cobertura -o report.cobertura.xml "dotnet test --no-build"
This will make sure that all dlls are statically instrumented before command dotnet test --no-build
is executed. This procedure can take time if your solution is big so this is another reason this approach is not recommended.
Closing as no work needed here.
This is in regard to the automatic merging of coverage files when tests are run at the solution-level, as described in https://devblogs.microsoft.com/dotnet/whats-new-in-our-code-coverage-tooling/
I pulled the referenced repo with sample code, and while the project-level coverage files were merged into a single file, the project/folder where the file was saved differed between test runs.
2nd execution
As the results show, the coverage of the first run was saved within the
Calculator.Server.IntegrationTests\TestResults
folder, while the file for the 2nd execution was saved inCalculator.Core.Tests\TestResults
I would expect a solution wide file to be saved in the solution root, or under a
TestResults
folder in the sln root by default. I can still manually merge with thedotnet-coverage
tool, but I was excited at the prospect of not needing the extra steps.I am running dotnet 7.0.203 on Windows 11. I get the same results with and without the
dotnet-coverage
tool installed. I am developing in VS Code, but don't believe that is relevant because all of the code samples are from the command line.