The task filters out warnings generated by custom scripts

[andrey-noskov]

Our build has a custom step with an inlined powershell script that generates the following build issues:

2018-09-03T10:14:28.7777479Z ##[warning]package = Newtonsoft.Json, version = 10.0.2
2018-09-03T10:14:28.7788388Z ##[warning]package = Newtonsoft.Json, version = 10.0.3

The 'Build Quality check' task correctly finds them and although 'Warnings Filters' configuration section is empty, the task for some reasons filters the warnings out:

2018-09-03T10:14:28.8498187Z ##[section]Starting: Check build quality - warnings
2018-09-03T10:14:28.8505999Z ==============================================================================
2018-09-03T10:14:28.8506391Z Task         : Build Quality Checks
2018-09-03T10:14:28.8506756Z Description  : Breaks a build based on quality metrics like number of warnings or code coverage.
2018-09-03T10:14:28.8507118Z Version      : 5.0.4
2018-09-03T10:14:28.8507552Z Author       : Microsoft Premier Services
2018-09-03T10:14:28.8507865Z Help         : [[Docs]](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/overview.md)
2018-09-03T10:14:28.8508191Z ==============================================================================
2018-09-03T10:14:30.1089306Z SystemVssConnection exists true
2018-09-03T10:14:30.4248780Z Validating build warnings policy...
2018-09-03T10:14:30.4952368Z Counting warnings from tasks:
2018-09-03T10:14:30.4952921Z - Build solution **\*.sln
2018-09-03T10:14:30.4961811Z -   Running MSBuild log analysis...
2018-09-03T10:14:30.6748254Z - Build package consolidation report
2018-09-03T10:14:30.7419809Z Total warnings: 2
2018-09-03T10:14:30.7420374Z Filtered warnings: 0
2018-09-03T10:14:30.8242986Z Found baseline build with ID 5597786.
2018-09-03T10:14:30.9515299Z Counting warnings from tasks:
2018-09-03T10:14:30.9515726Z - Build solution **\*.sln
2018-09-03T10:14:30.9516596Z -   Running MSBuild log analysis...
2018-09-03T10:14:31.1302823Z Total warnings: 0
2018-09-03T10:14:31.1304193Z Filtered warnings: 0
2018-09-03T10:14:31.1468255Z ##[section]Finishing: Check build quality - warnings

What could be the reason?

[ReneSchumacher]

Hi Andrey,

could you run your build again with the variable System.Debug set to true and then either post the task log here or send it to our support email address PSGerExtSupport (at) Microsoft.com? The debug log has all the information I need about the task configuration in your build so I should be able to see the root cause for the behavior.

Thanks, René

[ReneSchumacher]

Hi again,

I believe I already found the issue and it's probably best for us to change that behavior. When you activate warning statistics (as I believe you did), the task does not directly use the warnings counts from the build timeline. Instead, it runs the log file of each inspected task through a special log analyzer. We only have such an analyzer for MSBuild-based tasks (MSBuild and VSBuild). All other tasks use a generic log analyzer that only works, if you configure proper warning filters.

Since you didn't configure any warning filters, the analyzer simply ignores the whole log file which is not very helpful :). I'll update the task to only run the generic analyzer if there are warning filters. That should fix your issue.

[andrey-noskov]

thanks. indeed, once I deactivated warning statistics, the task handled the case correctly

2018-09-03T12:06:42.4053286Z ##[section]Starting: Check build quality - warnings
2018-09-03T12:06:42.4061897Z ==============================================================================
2018-09-03T12:06:42.4062228Z Task         : Build Quality Checks
2018-09-03T12:06:42.4062549Z Description  : Breaks a build based on quality metrics like number of warnings or code coverage.
2018-09-03T12:06:42.4062841Z Version      : 5.0.4
2018-09-03T12:06:42.4063091Z Author       : Microsoft Premier Services
2018-09-03T12:06:42.4064105Z Help         : [[Docs]](https://github.com/MicrosoftPremier/VstsExtensions/blob/master/BuildQualityChecks/en-US/overview.md)
2018-09-03T12:06:42.4064666Z ==============================================================================
2018-09-03T12:06:43.6176021Z SystemVssConnection exists true
2018-09-03T12:06:43.9311352Z Validating build warnings policy...
2018-09-03T12:06:43.9928390Z Counting warnings from tasks:
2018-09-03T12:06:43.9929072Z - Build solution **\*.sln
2018-09-03T12:06:43.9929364Z - Build package consolidation report
2018-09-03T12:06:43.9929883Z Total warnings: 2
2018-09-03T12:06:43.9930279Z Filtered warnings: 2
2018-09-03T12:06:43.9983574Z ##[error]At least one build quality policy was violated. See Build Quality Checks section for more details.
2018-09-03T12:06:44.0053693Z ##[section]Finishing: Check build quality - warnings

[ReneSchumacher]

Thanks for confirming my first assumption. The fix is already in our CI/CD pipeline and should go out today. I'll update the issue once it is public.

[ReneSchumacher]

Hi Andrey,

the new version is online. You can now enable warning statistics and still get the correct warning count. If you still run into issues with the new version, please let me know.

Happy building! René