[BQC] Status passed when build times out

[mjvankampen]

When getting the coverage data times out (##[warning]Unable to get code coverage data within the maximum wait time.) the policy status is marked as OK. In my case the build is marked as failed so the PR is not able to complete but it still seems a bit weird?

The status reported is Code coverage policy passed with 0% (0/0 branches).

[ReneSchumacher]

Can you send me the full configuration of the task so I can check what's going on? The timeout is just a warning and the policy assumes zero coverage as a result. In most cases this should fail the policy but this actually depends on the compare value.

Policy result and PR status should always go together but I might have missed something.

[mjvankampen]

Maybe it is due to the uncovered branches part, having zero uncovered branches is of course very nice

          - task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@6
            displayName: "Check build quality against previous commit"
            condition: not(eq(variables['Build.Reason'], 'PullRequest'))
            continueOnError: true
            inputs:
              runTitle: "MasterUnitTestCoveragePrevious"
              checkCoverage: true
              coverageFailOption: build
              coverageType: branches
              coverageDeltaType: absolute
              forceCoverageImprovement: false
              explicitFilter: true
              useUncoveredElements: true
          - task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@6
            displayName: "Check build quality against target branch"
            condition: eq(variables['Build.Reason'], 'PullRequest')
            continueOnError: true
            inputs:
              runTitle: "MasterUnitTestCoveragePullRequest"
              baseBranchRef: $(System.PullRequest.TargetBranch)
              checkCoverage: true
              coverageFailOption: build
              coverageType: branches
              coverageDeltaType: absolute
              forceCoverageImprovement: false
              explicitFilter: true
              useUncoveredElements: true

[ReneSchumacher]

Ah, that's it! I haven't thought about the timeout in case you're using uncovered elements. When using covered elements, zero coverage usually breaks the policy. That is why we don't really care about the timeout in that case. However, zero uncovered elements by default passes the policy. I'll change that so the policy fails if there's no coverage at all.

I'm still wondering why your build failed, though. The PR status and policy result should always be the same.

[ReneSchumacher]

We're in the process of releasing a fix for this. After lots of feedback about the additional security scopes and realizing that they don't affect our task at all, we came to the decision to remove all security scopes and re-release the task. I just put your fix into that release.

[mjvankampen]

My build failing had nothing to do with bcq! Sorry about that, but that is why I noticed.

[ReneSchumacher]

Thanks for clarifying that. I believe that this issue should be fixed then so I'm closing it now. If you still see unexpected behavior, please feel free to get back to us again!