Check code coverage for branches fails where it shouldn't

[IvanAlekseev]

After updating packages from coverlet.collector 1.2.0 to 1.2.1 I receive the following error:

2020-05-04T11:26:18.3763280Z ##[section]Starting: Check code coverage
2020-05-04T11:26:18.3913330Z ==============================================================================
2020-05-04T11:26:18.3913672Z Task         : Build Quality Checks
2020-05-04T11:26:18.3913989Z Description  : Breaks a build based on quality metrics like number of warnings or code coverage.
2020-05-04T11:26:18.3914270Z Version      : 6.3.0
2020-05-04T11:26:18.3914494Z Author       : Microsoft Premier Services
2020-05-04T11:26:18.3914874Z Help         : [[Docs]](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/overview.md)
2020-05-04T11:26:18.3915276Z ==============================================================================
2020-05-04T11:26:18.9682065Z SystemVssConnection exists true
2020-05-04T11:26:19.3532435Z Using IdentifierJobResolver
2020-05-04T11:26:19.4317118Z Validating code coverage policy...
2020-05-04T11:26:22.2385887Z Successfully read code coverage data from build.
2020-05-04T11:26:22.2404134Z Evaluating coverage data from 1 filtered code coverage data sets...
2020-05-04T11:26:22.2417763Z Total branches: 146
2020-05-04T11:26:22.2418263Z Covered branches: 137
2020-05-04T11:26:22.2418692Z Code Coverage (%): 93.8356
2020-05-04T11:26:22.3917805Z Found baseline build with ID 154359.
2020-05-04T11:26:22.7080420Z Successfully read code coverage data from build.
2020-05-04T11:26:22.7089847Z Evaluating coverage data from 1 filtered code coverage data sets...
2020-05-04T11:26:22.7141374Z Total branches: 154
2020-05-04T11:26:22.7147322Z Covered branches: 143
2020-05-04T11:26:22.7153594Z Code Coverage (%): 92.8571
2020-05-04T11:26:22.7182405Z [ERROR] The code coverage value (93.8356%, 137 branches) is lower than the value of the previous build! The previous build had a coverage value of 92.8571% (143 branches).
2020-05-04T11:26:22.7225948Z ##[error]The code coverage value (93.8356%, 137 branches) is lower than the value of the previous build! The previous build had a coverage value of 92.8571% (143 branches).
2020-05-04T11:26:22.7251158Z ##[error]At least one build quality policy was violated. See Build Quality Checks section for more details.
2020-05-04T11:26:22.7332024Z ##[section]Finishing: Check code coverage

My task setup:

      - task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@6
        displayName: 'Check code coverage'
        inputs:
          checkWarnings: false
          checkCoverage: true
          coverageFailOption: build
          coverageType: branches
          baseBranchRef: refs/heads/master

In my understanding 93.8356% is definitely higher than 92.8571% 😉 I guess the task tries to compare the number of branches covered, not the percentage. Can it be fixed?

[ReneSchumacher]

Hey,

you didn't configure the coverage delta type to it uses absolute values by default (see https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/CodeCoveragePolicy.md#user-content-coveragedeltatype). Just add coverageDeltaType: 'percentage' to the inputs and it'll work.

I guess I'll put a new item on the backlog to put the delta type into the message as well. I'd love to change the default to percentage, but that would be a breaking change and I'm not sure it's worth the effort.

[IvanAlekseev]

@ReneSchumacher it works with updated settings. Thank you for so quick reply. Having percentages by default will certainly be a better option

      - task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@6
        displayName: 'Check code coverage'
        inputs:
          checkWarnings: false
          checkCoverage: true
          coverageFailOption: build
          coverageType: branches
          coverageDeltaType: percentage
          baseBranchRef: refs/heads/master

[ReneSchumacher]

Got it. We're in the process of releasing a new version and this will be v7.0.0 and have the new default. Since this is a breaking change, you'll have to update the task version in your YAML file to benefit from the new default value.