ReneSchumacher
commented
4 years ago Hi @JaimeHu,
this behavior is a technical limitation of Azure DevOps. All code coverage data sets that are published to Azure DevOps are merged into a single aggregated coverage data set. Thus, the BQC task cannot distinguish between the data published by the first job and the data published by the second job.
If you're using VSTest for test execution and coverage calculation, there is a workaround for this. Each coverage data set can be associated with a platform and configuration property. You can set those properties in the Reporting Options section of the VSTest task. Only data sets associated with the same platform and configuration are merged by Azure DevOps. Thus, you can simply configure different platform/configuration settings for the two jobs and then configure the BQC tasks to just look at coverage values for those settings.
Unfortunately, this workaround is not available for other test tasks.
René
JaimeHu
We have a pipeline that runs two jobs to build two separate projects in our repo. Each has their own unit test step and build quality step to check code coverage. Whichever job runs first is fine but the second job checks the code coverage for both jobs instead of just the current job. As far as I can tell the code coverage results are stored in the same location so whichever job finishes does a code coverage check for both projects together.
For example: Job A has code coverage check 50% and is right at the threshold Job B has code coverage check 70% and is right at the threshold
Job A finishes first and passes the code coverage check Job B finishes second but the code coverage check looks at the sum of code coverage for Job A and Job B which is less than 70% and fails.
I think we need a way to specify which code coverage we want checked.