Build Quality Check step does not match the code coverage with VS test code coverage for dot net core

[pratikshadange]

Hi,

I have added BuildQualityCheck step in the build definition. Dot net core Test project is added for the core web api. I have used .Net Core build step before BuildQuality Check step. When code coverage is checked using Test Explorer in Visual studio, it shows 80% or more. But the same code on build shows ~30% in Build Quality check step. Shouldn't they show similar values for code coverage? I also added Code Coverage Protector/Comparer step to see the difference between code covered in two successive builds , that also shows shows the values like ~30%.

[ReneSchumacher]

Hi,

did you run the tests with the Visual Studio Test task or the Dotnet task and are you sure that all code coverage data has been properly published before running the Build Quality Checks task?

One thing that stands out in your screenshots is that the coverage file only lists 2556 total blocks while our task reports 18762 total blocks. This is a huge difference and I assume that the task is looking at the aggregated result of multiple code coverage data sets. Could you post a screenshot of the code coverage summary shown in the build result page?

René

[pratikshadange]

I ran the dot net core step for Tests. It executed all the unit test cases successfully and published using one more dot net core step to publish command. Then Build Quality check step is placed.

The code coverage blocks shown different and thats why the difference is. Current Code coverage shown after build using Build quality check screenshot --

Its shows more blocks because of Automapper and Moq.dll. It should not check for references, right ?

For same code when I run test cases locally in VS Test explorer and then run code coverage for test cases, it shows-

[ReneSchumacher]

I'm not an expert on dotnet core but I believe that the standard test run started by dotnet test does not exclude references from coverage instrumentation. You'll probably have to use runsettings as described at https://docs.microsoft.com/en-US/dotnet/core/tools/dotnet-test?tabs=netcore21 and https://github.com/Microsoft/vstest-docs/blob/master/docs/RunSettingsArguments.md to exclude those assemblies.

[pratikshadange]

Thanks Rene. That worked. I had tried RunSettings before but need to remember to add RunSettings arguments correctly in the dot net test build step :). Now with settings arguments it works correctly. We need to provide module paths to be excluded in the settings file. This excluded the mentioned references from code coverage.

[ReneSchumacher]

Great to hear that you solved the issue. I'm closing the issue now, but if you need anything else, please feel free to open another issue or send an email to PSGerExtSupport (at) microsoft.com.

Happy building! René