search

hyperledger / cacti

Hyperledger Cacti is a new approach to the blockchain interoperability problem
https://wiki.hyperledger.org/display/cactus
Apache License 2.0
335 stars 277 forks source link

ci(github): workflow to ensure PR description & commit message parity #2214

Closed petermetz closed 2 weeks ago

petermetz commented 1 year ago

Description

As a maintanier/reviewer I want to have automation that makes sure that people don't just fill out the PR description on GH and leave their commit message basically empty so that the real explanation for the changes are contained in the git history and not in a database that we don't control.

Acceptance Criteria

  1. Implement the solution via a GitHub check like the PR title linting that we have in place right now.
  2. Make sure to have a manual override built-in (available only to the reviewers on a case-by-case basis) so that in special cases the check can be marked as passing (but the default is to fail the check and not let the PR get merged without the commit message being legible)
  3. Set it up in a way so that it's part of the checks executed for every pull request and it gets executed when the PR gets updated as well (e.g. if I force push something and change the commit message to something non-legible it should start failing again)
petermetz commented 3 months ago

Further context: You can use this GitHub Action from the marketplace to make API calls directly from the workflow YAML: https://github.com/octokit/request-action

The above then enables you to retrieve the PR description and the PR commit's message through the GitHub API.

Finally with this information being available you can run a comparison and issue a warning state on a check you wrote so that people who are the authors of a pull request where the commit message and the pull request description are mismatched can get to know about this problem.

To determine the exact endpoints to use to retrieve the information above, check the GitHub REST API docs: https://docs.github.com/en/rest?apiVersion=2022-11-28

So to tie it all together: You'd create a check job in one of our yaml files which would then run on pull requests and perform the data retrieval through the GitHub API to get the PR description and commit message, compare the two and set the status of the check to failed or succeeded based on that.

petermetz commented 1 month ago

@zondervancalvez Some test to demonstrate what it should do on a high level:

Test case 1:

PR Description: "asdf" Commit 1 message in the PR: "asdf"

^^ The check should pass here because everything from the PR description is present.

Test case 2: PR Description: "asdf - some other information as well" Commit 1 message in the PR: "asdf"

^^ The check should FAIL here because the person updated the PR description with new information that is missing from the git commit message.

Test case 3:

PR Description: "asdf" Commit 1 message in the PR: "abcdefg" Commit 2 message in the PR: "asdf"

^^ The check should PASS here because there was a commit in the pull request that contained the information from the PR description. It's OK if commit 1 didn't have it because at least commit 2 did so the information won't be lost.