MicrosoftPremier / VstsExtensions

Documentation and issue tracking for Microsoft Premier Services Visual Studio Team Services Extensions
MIT License
56 stars 14 forks source link

Version 7 no longer counts warnings (warningFilters) #125

Closed Bouke closed 3 years ago

Bouke commented 3 years ago

I'm running a linter (eslint) task as part of my build. This linter produces output like this:

X\app\components\catalog\category-node.ts(30,5): warning @typescript-eslint/explicit-module-boundary-types : Missing return type on function.
X\app\components\catalog\category-node.ts(60,13): warning @typescript-eslint/no-non-null-assertion : Forbidden non-null assertion.
X\app\components\catalog\category-node.ts(69,33): warning @typescript-eslint/no-non-null-assertion : Forbidden non-null assertion.
app/templates/product-catalog/category/products/create.hbs
  18:12  error  Do not use `action` as <button {{action ...}} />. Instead, use the `on` modifier and `fn` helper.  no-action
  20:12  error  Do not use `action` as <button {{action ...}} />. Instead, use the `on` modifier and `fn` helper.  no-action

I want to verify the lint issue count against the baseline, to ensure no new issues are introduced. The count is determined by counting the occurrences of the words "error" or "warning". This used to work fine with version 6, but version 7 is reporting 0 issues.

#version 6:
    - task: BuildQualityChecks@6
      inputs:
        checkWarnings: true
        warningFailOption: 'build'
        showStatistics: true
        warningTaskFilters: '/^Lint/'
        warningFilters: '/(warning|error)/'
        baseDefinitionId: '25'
        baseRepoId: 'ef191fe7-612d-4dbc-9b0a-defb197d4ec6'
        baseBranchRef: '$(System.PullRequest.TargetBranch)'

#version 7:
    - task: BuildQualityChecks@7
      inputs:
        checkWarnings: true
        warningFailOption: 'build'
        showStatistics: true
        warningTaskFilters: '/^Lint/'
        warningFilters: '/(warning|error)/'
        baseDefinitionId: '25'
        baseRepoId: 'ef191fe7-612d-4dbc-9b0a-defb197d4ec6'
        baseBranchRef: '$(System.PullRequest.TargetBranch)'

This is the output from both tasks:

Version      : 6.4.4
(...)
Running task warnings analysis...
Counting warnings from tasks:
  - Lint X
Waiting for log information of task 'Lint X'...
     Running generic task log analysis...
     Total warnings: 538, Filtered warnings: 538
Overall total warnings: 538
Overall filtered warnings: 538
Found baseline build with ID 15602.
Running task warnings analysis...
Counting warnings from tasks:
  - Lint X
     Running generic task log analysis...
     Total warnings: 526, Filtered warnings: 526
[SUCCESS] Warnings policy passed with 538 warning(s).
Finishing: Check lint issue count
Version      : 7.6.2
(...)
Running task warnings analysis...
Counting warnings from tasks:
  - Lint X
     Running generic task log analysis...
     Total warnings: 0, Filtered warnings: 0
Overall total warnings: 0
Overall filtered warnings: 0
Found baseline build with ID 15601.
Running task warnings analysis...
Counting warnings from tasks:
  - Lint X
     Running generic task log analysis...
     Total warnings: 0, Filtered warnings: 0
Overall total warnings: 0
Overall filtered warnings: 0
[SUCCESS] Warnings policy passed with 0 warning(s).
Statistics generation took 64.144ms
Finishing: Check lint issue count

Any help would be appreciated.

Bouke commented 3 years ago

Maybe the new filter regex with named capture groups is required and the old regex no longer works? I've changed my regex to the following, which now match the first three log entries:

        warningFilters: '/D:\\a\\1\\s\\(?<filename>.+?)\((?<line>\d+),(?<column>\d+)\): (warning|error) (?<identifier>[^ ]+) : (?<message>.+)$/'

image

Note that the fourth and fifth errors aren't matched by this, but that's not a concern anymore as I'm reformatting the output using python so that it matches the regex.

ReneSchumacher commented 3 years ago

Hi @Bouke,

sorry for the late response. I'm currently busy delivering customer workshops but will look into your issues after I'm done with the workshop today. The new capturing group feature is most likely the root cause for this, but I'll have to check to be 100% sure. You could either use a non-capturing group in your original filter (i.e., /(?:warning|error)/) or switch to named capturing groups as you did. The latter is preferred as it also supports warning statistics for custom warning filters.

René

ReneSchumacher commented 3 years ago

Hi again,

I have looked at the differences between v6.4.4 and v7.6.2 and I am still not sure why it stopped working. Version 6.4.4 already had the feature that a capturing group was used to find the total number of warnings in a log. Thus, if warnings weren't properly reported but you had a summary log message like "Total warnings: xyz", you could extract that xyz with a capturing group and we would simply add that number to the overall number of warnings. If the group captured any non-numeric value, the task should basically ignore it.

Having said that, while this feature is still available in v7.6.2, I'd still recommend using properly named capturing groups to support warning statistics or - if you don't need statistics - switch to a non-capturing group. Both resolves the issue. Let me know if you need anything else.

ReneSchumacher commented 3 years ago

I'm closing this issue since I believe it is resolved by changing the regular expression.