Release infra: fail on reverted commits

[dvoytenko]

Describe the new feature or change to an existing feature you'd like to see

If commit was found to be problematic, it's reverted. However, it still could end up in a release branch without its revert counterpart. This would create a bad release. However, this situation is fairly easily avoidable.

Description

Naming:

• COMMIT - the bad commit that was reverted.
• PR(COMMIT) - the pull request that created the bad commit.
• REVERT - the reverting commit.
• PR(REVERT) - the pull request that created the reverting commit.
• BRANCH - the branch created for the release.

Once the BRANCH is created, we scan the new commits in this branch. If a COMMIT is found in this branch, there must also be REVERT. If there's no REVERT, the release should fail (or warn).

Technicals

Let's look at the recent failure with https://github.com/ampproject/amphtml/pull/27319 and the reverting https://github.com/ampproject/amphtml/pull/27422.

It's not clear that GitHub API itself provides the data that we want. For instance, here's the reverted commit and pull request. I didn't find any good information to indicate that it was reverted. Thus we may have to create a label "Reverted" and set it on the reverted pull request.

The reverting commit and pull request on the other hand provide plenty of information. Such as:

• Commit message includes "This reverts commit 17636f5a..."
• Pull request body includes "Reverts ampproject/amphtml#27319"
• Default branch name is "revert-27319-..."

Thus, we could look for "Reverted" label and see if there's a matching reverting commit available.

/cc @ampproject/wg-infra

[rsimha]

In addition, based on this week's learnings, we should fail any release that contains a commit without an accompanying PR.

[rsimha]

@dvoytenko We don't use the wg-infra repo to track FRs, we only use it for status updates. I'm moving this issue to the amphtml repo.

Edit: Moved to https://github.com/ampproject/amphtml/issues/27771. Continue discussion there.