ReneSchumacher
commented
4 years ago Hi,
this looks like a simple configuration mistake. Each policy in BQC can run in one of two modes: fixed threshold or previous value.
If you want to ensure a certain policy value (i.e., a maximum number of warnings or a minimum value for code coverage), you use the fixed threshold mode and set the threshold to the specific value. I can see from the logs that you have configured two instances of BQC with fixed threshold for both line and branch coverage and they seem to work as expected.
When you switch to previous value mode, BQC compares the current values with the ones from the previous build and automatically breaks the build if the current value is lower than the previous one. In that mode, though, there is an additional variance option that you can set to allow some decrease. This has been an ask from an internal team as their code coverage sometimes varies slightly (e.g., it drops 0.001% due to some code rearrangements). Thus, if you specify a variance value, the policy only fails if the current value is lower than the previous value minus the variance. Imagine your previous build had a coverage of 80% and you set a variance of 5. The policy only fails now if the current value is lower than 75%, otherwise you just see the warning that coverage has decreased within variance.
To fix you issue you either need to remove the variance so the policy fails as soon as its value has dropped, or you need to lower the variance to an acceptable value.
Happy building, René
Hello,
I am trying to fail the Build if coverageVariance for Lines and Branches is above variance value. This works fine when coverageType is Lines and not Branches.
Version : 7.1.0 Lines Coverage
Successfully read code coverage data from build. Evaluating coverage data from 1 filtered code coverage data sets... Total lines: 10953 Covered lines: 8985 Code Coverage (%): 82.0323 Found baseline build with ID 187597. Successfully read code coverage data from build. Evaluating coverage data from 1 filtered code coverage data sets... Total lines: 10968 Covered lines: 9006 Code Coverage (%): 82.1116 [ERROR] The code coverage (percentage) value (82.0323%, 8985 lines) is lower than the value of the previous build including variance! The previous build had a coverage value of 82.1116% (9006 lines).
[error]The code coverage (percentage) value (82.0323%, 8985 lines) is lower than the value of the previous build including variance! The previous build had a coverage value of 82.1116% (9006 lines).
[error]At least one build quality policy was violated. See Build Quality Checks section for more details.
Finishing: Check-Previous-Lines-Coverage with refs/heads/master
Branch coverage
Successfully read code coverage data from build. Evaluating coverage data from 1 filtered code coverage data sets... Total branches: 3087 Covered branches: 2335 Code Coverage (%): 75.6398 Found baseline build with ID 187597. Successfully read code coverage data from build. Evaluating coverage data from 1 filtered code coverage data sets... Total branches: 3111 Covered branches: 2356 Code Coverage (%): 75.7313 **[WARNING] Code coverage descreased within configured variance.
[warning]Code coverage descreased within configured variance.**
[SUCCESS] Code coverage policy passed with 75.6398% (2335/3087 branches). Finishing: Check-Previous-Branches-Coverage with refs/heads/master
Why Branch variance throwing warning instead of Error?