Code coverage check is not failing the build

[IvanAlekseev]

Hello, this extension worked perfectly for me for some time, but now it doesn't fail the build when coverage goes down. Can you point me at what am I doing wrong here?

Here is a build task:

      - task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@6
        displayName: 'Check code coverage'
        inputs:
          checkWarnings: false
          checkCoverage: true
          coverageFailOption: build
          coverageType: branches
          baseBranchRef: refs/heads/master

It produces this log:

Starting: Check code coverage
==============================================================================
Task         : Build Quality Checks
Description  : Breaks a build based on quality metrics like number of warnings or code coverage.
Version      : 6.1.2
Author       : Microsoft Premier Services
Help         : [[Docs]](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/overview.md)
==============================================================================
SystemVssConnection exists true
Validating code coverage policy...
Successfully read code coverage data from build.
Total branches: 3181
Covered branches: 1595
Code Coverage (%): 50.1415
Found baseline build with ID 120641.
Successfully read code coverage data from build.
Total branches: 3132
Covered branches: 1614
Code Coverage (%): 51.5326
Finishing: Check code coverage

According to the log it should fail, but it doesn't.

[ReneSchumacher]

Hi Ivan,

sorry for the delay (I'm currently on vacation). Could you run your build again with the variables System.Debug and BQC.LogRawData set to true and send the log file to PSGerExtSupport@microsoft.com so I can take a look at the data?

Thanks, René

[IvanAlekseev]

Hi René, Thanks for your reply. I sent an email with logs as requested. Also managed to make it work by setting one more parameter on the task:

ignoreDecreaseAboveUpperThreshold: false

[ReneSchumacher]

Hi Ivan,

sorry again for the late response. We have analyzed the issue and you seem to have hit a bug that is indeed related to the ignoreDecreaseAboveUpperThreshold setting. If this is set to true, the task assumes percentage values for coverage and passes the policy if the value is above the default threshold of 80%. In your case it assumes that the absolute number of lines is the percentage value, thus, everything above 80 lines will pass the policy.

The current workaround is to set ignoreDecreaseAboveUpperThreshold to false. I'll let you know when the issue is fixed.

René

[ReneSchumacher]

Hi @IvanAlekseev.

the last month have been pretty busy and we couldn't work on the task until this week. Just a short heads-up: the issue is fixed now and the code changes are going into our review process today. An updated version should be available early next week. I'm going to post another comment here, when we're done.

René

[ReneSchumacher]

Hi @IvanAlekseev,

we just published version 6.2.0 of the BQC task that fixes the problem you reported. I'm closing this issue now. Let me know if you still see problems with the task.

Happy building, René