[Bootcamp] Show the resolution and any steps required by the contributors to bring their PRs to green after a SEV is resolved

[huydhn]

Pitch

With the recent change to show the list of active SEVs on Dr. CI message, we have opened up a new way to communicate this critical piece of information to PyTorch contributors beyond the meta-only Tree Huggers channel. On the other hand, once a SEV is resolved, its message would disappear from Dr. CI, which leaves several important aspects open:

• What are the steps (if any) required by the contributors to bring their PRs to green? It could be retrying for infra SEVs, or rebase pass master to get the fixes, or just to verify and force merge
• Surfacing the SEVs resolution for 12 hours after they're closed to focus what users can do to mitigate the issues and to prevent them from happening again (if possible)

Solution

We have already queried all SEVs in Dr.CI including resolved ones. A simple approach would be to keep them a while longer, i.e. 1 day, and to parse the SEVs body to gather the above information.

If the mitigation wasn't filled out or is left as the template default, assume no mitigation is needed.

References

For getting started with development read these two READMEs:

• https://github.com/pytorch/test-infra/blob/main/README.md
• https://github.com/pytorch/test-infra/blob/main/torchci/README.md

Dr CI When a PR has fails any CI checks, we have a github bot called Dr. CI which leaves a comment at the top of the PR calling out the failures. Dr CI bot entrypoint (triggered by Github webhooks): https://github.com/pytorch/test-infra/blob/main/torchci/lib/bot/drciBot.ts

Example Dr. CI output with CI failures:

• https://github.com/pytorch/pytorch/pull/87488#issuecomment-1287282815 Those links should be pointing to the relevant id under https://hud.pytorch.org/pr/87488 instead

Sev: Sevs are github issues filed against pytorch/pytorch with the ci: sev label. They're created using a template Example: https://github.com/pytorch/pytorch/issues/92626

cc @clee2000 @ZainRizvi @pytorch/pytorch-dev-infra

[ZainRizvi]

+1, this would be pretty handy to include

[ZainRizvi]

This will require formatting the SEV body somehow to include a Mitigation section that Dr. CI can parse and include on any PRs that it comments on