search

MicrosoftPremier / VstsExtensions

Documentation and issue tracking for Microsoft Premier Services Visual Studio Team Services Extensions
MIT License
59 stars 14 forks source link

PR reaching back 2 baseline builds #178

Closed pjohnst5 closed 2 years ago

pjohnst5 commented 2 years ago

Describe the context

Describe the problem and expected behavior BuildQualityChecks has been working great so far. Only lately that new PRs are being compared to a master branch build, that is 2 merges old

What I mean by that is that, I merged change y into master, let's say with 57% Then I merged change x into master, let's say with %50

New PRs are comparing the coverage to change x's merge to master 03/24, of 57% Expect the new PRs to compare coverage to change y's merge to master 03/28, of 50%

PR build: https://dev.azure.com/msazure/One/_build/results?buildId=53613620&view=results

y's merge build(older): https://dev.azure.com/msazure/One/_build/results?buildId=53432671&view=results Had 57%

x's merge build( more recent than y's, so want it to compare to this): https://dev.azure.com/msazure/One/_build/results?buildId=53575006&view=results

image

Task logs Run your pipeline with the following variables:

Send the task log to PSGerExtSupport@microsoft.com and reference your GitHub issue.

Attention: The log file may contain sensitive data (e.g., server or organization names, project names, variable information). Please do not attach the log to your GitHub issue and or remove the information from the log file before attaching or sending.

ReneSchumacher commented 2 years ago

Hi @pjohnst5,

this is expected behavior. BQC only compares with succeeded or partially succeeded (if enabled) builds. You 50% coverage build failed, so BQC compares to the last successful build, which is your 57% coverage build.

If you need to reset the coverage baseline to a lower value, you can set the variable BQC.ForceNewBaseline (see here) to true, which forces the coverage policy to pass and lets the build succeed. Future builds then use the new succeeded build with the lower code coverage value as baseline.

pjohnst5 commented 2 years ago

Thanks @ReneSchumacher ! I'll give that a shot

pjohnst5 commented 2 years ago

Hey sorry, I can't seem to reset the build that each run keeps looking back at @ReneSchumacher This is the build that all PRs look back at still: https://msazure.visualstudio.com/One/_build/results?buildId=53432671&view=results

I tried clicking "Run new" in top right, that just starts off another run off tip of master

I tried running the pipeline on the same merge commit to master as the build ^, that didn't change anything

How can I reset this build? Thanks!!

pjohnst5 commented 2 years ago

I also set the variable you mentioned in the pipeline, and do see it for new builds, then I turn it off and run, but still they look back to the old build linked above ^

pjohnst5 commented 2 years ago

This is the replica attempt, to reset on the stuck build: https://msazure.visualstudio.com/One/_build/results?buildId=53667640&view=results

ReneSchumacher commented 2 years ago

Hi @pjohnst5,

the build you tried to reset is still failing, this time because of the publish code coverage step. You need to ensure that your build succeeds, otherwise it is not considered as a baseline for BQC.

ReneSchumacher commented 2 years ago

Hi @pjohnst5,

could you resolve the issue, or do you need additional help?

ReneSchumacher commented 2 years ago

Hi @pjohnst5,

I believe this issue is already resolved, so I'm closing it. If you still need help, please feel free to either create a new one or contact me internally again.