[Helix-machines] PR builds should not fail when secrets need cycling

riarenas commented 8 months ago

Currently if secret manager detects that there is a secret that is nearing expiration and needs rotation, the helix-machines PR builds will fail, blocking the PR and the work.

Our process is for secret rotation to happen as part of FR duties using the scheduled pipelines that run at the beginning of the week, so blocking a dev's unrelated work with this seems inappropriate.

We should consider only running the secret validation during main builds, or running it during PRs in such a way that doesn't block the build.

Release Note Category

[ ] Feature changes/additions
[ ] Bug fixes
[ ] Internal Infrastructure Improvements
Release Note Description

riarenas commented 8 months ago

An example build where this happened: https://dnceng.visualstudio.com/internal/_build/results?buildId=2372673&view=results

dougbu commented 7 months ago

This should be a quick fix — just delete the ValidateSecrets stage from azure-pipeline-pr.yaml, leaving it just in azure-pipelines.yml.

I'm placing this in the Improve helix-machines Infrastructure epic. Feel free to move elsewhere if you disagree.

dotnet / dnceng

[Helix-machines] PR builds should not fail when secrets need cycling #1976

Release Note Category

Release Note Description