BuildQualityChecks@9 : the pull request target branch baseline is not updated upon completion

[Vasilievski]

Context

• Extension: BuildQualityChecks
• Environment: Azure DevOps Server
  • Server version: Azure DevOps Server 2022 RC1 (AzureDevOpsServer_20220728.1)
• Agent type: self-hosted
  • Agent version: 2.204.0
• Pipeline type: yaml pipeline

Describe the problem and expected behavior I installed BuildQualityChecks in order to enforce a "warnings number do not grow" policy. Everything works fine except that the baseline of the target branch is not lowered when a pull request removes warnings.

Let's say the baseline is 2 before pull request A, and I remove 1 warning in this pull request. I expect the target branch baseline to be 1 upon pull request completion, but it is not the case. It's an issue because pull request B can introduce a new warning the baseline goes back to 2 and the pull request can be completed.

Here is the pipeline configuration yml of the step :

- task: BuildQualityChecks@8
  inputs:
    checkWarnings: true
    warningFailOption: 'build'
    showStatistics: true
    createBuildIssues: true
    runTitle: 'Warnings Checks'
    baseBranchRef: '$(System.PullRequest.TargetBranch)' 

Do I missunderstand anything ?

Thank you,

[ReneSchumacher]

Hi @Vasilievski,

this is a very common scenario but it requires a specific setup. The best way to achieve what you want is to have one pipeline definition with a CI trigger on your main/master branch. Then, enforce pull requests by setting up a branch policy for your main/master branch and use the CI pipeline as a build verification check. Finally, create a baseline by running the pipeline on main/master once.

With this setup, every pull request automatically triggers your pipeline. Even without setting baseBranchRef, this pipeline will fall back to the target branch. Once the PR is completed and code is merged back to main/master, the CI trigger will start the pipeline on main/master and this run updates the baseline value.

[ReneSchumacher]

@Vasilievski Did my response resolve the issue, or do you need more help?

[ReneSchumacher]

Closing this issue for now due to lack of feedback. If you need additional help, please comment and I will reopen the issue.