BQC count warnings in version 7/8?

[Bouke]

This is somewhat related to #125, but now I'm trying to use the following documented use-case:

Warning Filters (Tasks) can also be used to count warnings if a build task does not log warnings as build issues, as long as the task logs the number of warnings to its log file. To do so, specify a warning filter that contains exactly one unnamed capture group that matches the number of warnings. The captured number will be added to the total warning count as well was the filtered warning count.

I'm building using dotnet build and I want to compare the build error count. Whether the build fails or not is not relevant, however I want to reduce the overall build errors.

The output from dotnet build is the following:

(...)
    108 Warning(s)
    60 Error(s)

So my task looks like this:

    - task: DotNetCoreCLI@2
      displayName: dotnet build
      inputs:
        command: 'build'
        arguments: '--framework net5.0 --configuration $(buildConfiguration)'
        workingDirectory: Source/
      continueOnError: true
    - task: BuildQualityChecks@8
      displayName: 'Check build errors'
      inputs:
        checkWarnings: true
        warningFailOption: 'build'
        showStatistics: true
        warningTaskFilters: '/^dotnet build$/'
        warningFilters: '/(\d+) Error\(s\)/'
        baseDefinitionId: '25'
        baseRepoId: 'ef191fe7-612d-4dbc-9b0a-defb197d4ec6'
        baseBranchRef: '$(System.PullRequest.TargetBranch)'
        runTitle: 'Check Build Errors'

On v8 this gives the following incorrect result:

     Total warnings: 110, Filtered warnings: 0
[SUCCESS] Warnings policy passed with 0 filtered warning(s) out of 110 total warning(s).

On v6 this produces the expected outcome:

     Total warnings: 170, Filtered warnings: 60
[ERROR] The number of filtered warnings (60 out of 170) has increased since the previous build! The previous build had 0 filtered warning(s) out of 0 total warning(s).

How to achieve the behaviour of v6 in v8?

[ReneSchumacher]

Hi @Bouke,

sorry for the delay, I have been on vacation over the holidays. Let me take a look at our code. There shouldn't have been any changes to that particular behavior and our tests for this feature are still green. Yet, we might have missed something as BQC got more and more complex.

[ReneSchumacher]

Hi @Bouke,

I have found the root cause for this and it is a change that we introduced in version 7.2.0, in which we decided to analyze all tasks that properly log warnings as build issues in the same way. This made it possible for more tasks to be included in our warning statistics but had the side effect of using warning filters only to filter task logs. If you need to have the special behavior of the warning filters, you need to set the parameter inclusiveFiltering to true. This basically tells BQC to run the warning filter analysis in addition to the regular analysis.

I'm just wondering why you would treat errors as warnings? Shouldn't it be better to just break the build if errors occur during compilation?

René

[Bouke]

Hi @ReneSchumacher, thank you for pointing me to inclusiveFiltering: true, that seems to resolve my problem. It's now not completely doing what I expect, as it counts the number of ##[warning] (514) + the result of warningFilters (33 Errors(s)) = 547 warnings. I expected to only see the warningFilters, but alas: this will do.

I'm just wondering why you would treat errors as warnings? Shouldn't it be better to just break the build if errors occur during compilation?

I understand that it seems backwards. We have a solution that's targeting .NET Framework 4.8, but we're in the process of adding .NET 5 support. Not all projects are .NET 5 ready and thus cause a build failure. Over time, I want to increase the number of projects supporting .NET 5. I employ BQC to enforce this.

[ReneSchumacher]

Got it. And I really do believe that we actually broke the feature you are looking for :( By forcing all "consumable" logs through a specific log analyzer, we removed the possibility to just grab the warnings from that special filter. I'll put that on the backlog and see if I can make this work again either this or next week. Sorry for that.

[ReneSchumacher]

Hi @Bouke,

I didn't get a chance to work on this earlier, but I'm happy to announce that the next release of the task will again allow you to use the special capturing group filter syntax to evaluate only a single aggregated number of warnings from a log file. It disables warning statistics as we cannot do both (and it doesn't really make sense to enable both). The new version will be public next week.

[ReneSchumacher]

Hi again,

the new version is out now (v8.0.3). If you switch off inclusive filtering again, you should see the old behavior again. Let me know if that doesn't meet your expectations.

I'm going to close this issue now as the new release fully resolves it.

Happy building! René