MicrosoftPremier / VstsExtensions

Documentation and issue tracking for Microsoft Premier Services Visual Studio Team Services Extensions
MIT License
56 stars 14 forks source link

Coverage has gone down but quality check still passes #87

Closed yowl closed 4 years ago

yowl commented 4 years ago

History: Was running ok in DevOps hosted agent, have moved the agent to an Azure VM and now build succeeds even when coverage goes down. (Could be coincidence or I've screwed something up)

From the log file:

2020-04-15T15:08:34.1303875Z Description  : Breaks a build based on quality metrics like number of warnings or code coverage.
2020-04-15T15:08:34.1304313Z Version      : 6.3.0
2020-04-15T15:08:34.1304648Z Author       : Microsoft Premier Services
2020-04-15T15:08:34.1305230Z Help         : [[Docs]](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/overview.md)
2020-04-15T15:08:34.1305855Z ==============================================================================
2020-04-15T15:08:36.1344021Z SystemVssConnection exists true
2020-04-15T15:08:37.1509369Z Using IdentifierJobResolver
2020-04-15T15:08:37.3089765Z Validating build warnings policy...
2020-04-15T15:08:37.4737353Z Counting warnings from tasks:
2020-04-15T15:08:37.4738181Z - Build LLCoolJ dependencies
2020-04-15T15:08:37.6523681Z -   Total warnings: 0, Filtered warnings: 0
2020-04-15T15:08:37.6525186Z - Build solution src/www/Silverlight/Demurrage/Demurrage.sln
2020-04-15T15:08:38.4998289Z -   Total warnings: 10, Filtered warnings: 10
2020-04-15T15:08:38.5000333Z Total warnings: 10
2020-04-15T15:08:38.5001764Z Filtered warnings: 10
2020-04-15T15:08:38.8232429Z Found baseline build with ID 1186.
2020-04-15T15:08:39.3097070Z Counting warnings from tasks:
2020-04-15T15:08:39.3097580Z - Build LLCoolJ dependencies
2020-04-15T15:08:39.6275014Z -   Total warnings: 0, Filtered warnings: 0
2020-04-15T15:08:39.6276074Z - Build solution src/www/Silverlight/Demurrage/Demurrage.sln
2020-04-15T15:08:40.1529127Z -   Total warnings: 10, Filtered warnings: 10
2020-04-15T15:08:40.1609330Z Total warnings: 10
2020-04-15T15:08:40.1641185Z Filtered warnings: 10
2020-04-15T15:08:40.1655577Z [SUCCESS] Warnings policy passed with 10 warning(s).
2020-04-15T15:08:40.1656157Z Validating code coverage policy...
2020-04-15T15:08:43.3391429Z Successfully read code coverage data from build.
2020-04-15T15:08:43.3408401Z Evaluating coverage data from 1 filtered code coverage data sets...
2020-04-15T15:08:43.3418055Z Total branches: 12722
2020-04-15T15:08:43.3418646Z Covered branches: 5634
2020-04-15T15:08:43.3419352Z Code Coverage (%): 44.2855
2020-04-15T15:08:43.5042165Z Found baseline build with ID 1186.
2020-04-15T15:08:43.9721203Z Successfully read code coverage data from build.
2020-04-15T15:08:43.9729772Z Evaluating coverage data from 1 filtered code coverage data sets...
2020-04-15T15:08:43.9735763Z Total branches: 12688
2020-04-15T15:08:43.9736497Z Covered branches: 5633
2020-04-15T15:08:43.9737153Z Code Coverage (%): 44.3963
2020-04-15T15:08:43.9758207Z [SUCCESS] Code coverage policy passed with 44.2855% (5634/12722 branches).
2020-04-15T15:08:44.0062784Z ##[section]Finishing: Check build quality
2020-04-15T15:08:44.0233405Z ##[section]Starting: Checkout CMS@refs/pull/258/merge to s

So you can see the coverage has gone down from 44.3963 to 44.2855. The build is configured in the GUI in devops, not in yaml, where it looks like:

image

yowl commented 4 years ago

Hmm, maybe I changed it to absolute from percentage by accident