search

kubernetes-sigs / prow

Prow is a Kubernetes based CI/CD system developed to serve the Kubernetes community. This repository contains Prow source code and Hugo sources for Prow documentation site.
https://docs.prow.k8s.io
Apache License 2.0
129 stars 99 forks source link

fix: do not apply approved label if change requested #270

Closed sdowell closed 2 months ago

sdowell commented 2 months ago

An outstanding CHANGES_REQUESTED state prevents a PR from being merged even if another reviewer approves the PR. This change makes it so that the approved label cannot be added until all change requested states have been resolved.

This prevents a scenario where tide will repeatedly try to merge a PR and be rejected by GitHub due to the changes requested state.

Fixes: https://github.com/kubernetes-sigs/prow/issues/269

k8s-ci-robot commented 2 months ago

Welcome @sdowell!

It looks like this is your first PR to kubernetes-sigs/prow 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/prow has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

k8s-ci-robot commented 2 months ago

Hi @sdowell. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
k8s-ci-robot commented 2 months ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sdowell Once this PR has been reviewed and has the lgtm label, please assign cjwagner for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[pkg/plugins/OWNERS](https://github.com/kubernetes-sigs/prow/blob/main/pkg/plugins/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
netlify[bot] commented 2 months ago

Deploy Preview for k8s-prow ready!

Name Link
Latest commit 1bdbd873928ceb4782f69ad3093462f4da2cf3d6
Latest deploy log https://app.netlify.com/sites/k8s-prow/deploys/66da441c77ef460008c02e7d
Deploy Preview https://deploy-preview-270--k8s-prow.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

matthyx commented 2 months ago

/ok-to-test

petr-muller commented 2 months ago

/hold

An outstanding CHANGES_REQUESTED state prevents a PR from being merged even if another reviewer approves the PR.

I may be wrong (will have to look at GH docs) but I don't think the above is universally true, IIRC this was a branch protection configuration option.

https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#require-pull-request-reviews-before-merging

Says:

If a person with admin permissions chooses the Request changes option in a review, then that person must approve the pull request before the pull request can be merged. If a reviewer who requests changes on a pull request isn't available, anyone with write permissions for the repository can dismiss the blocking review.

I'm not convinced the PR is the right fix. When reviewActsAsApprove is not enabled, I don't think approve should take it into account at all. If the repo is configured via BP to require reviews for merging (leading Tide to loop merge attempts) then it is basically misconfigured and Tide should surface this problem better (I believe we had an issue for this class of problems - it was not just request changes, but other cases where BP prevents PRs from merging but Tide does not know about it).

sdowell commented 2 months ago

I may be wrong (will have to look at GH docs) but I don't think the above is universally true, IIRC this was a branch protection configuration option.

I think you're right, I see also the wording of the BP setting:

When enabled, pull requests targeting a matching branch require a number of approvals and no changes requested before they can be merged.

Is there a doc/guide anywhere on how to properly set up GitHub, and branch protection in particular, for Prow/Tide?

If the repo is configured via BP to require reviews for merging (leading Tide to loop merge attempts) then it is basically misconfigured and Tide should surface this problem better (I believe we had an issue for this class of problems - it was not just request changes, but other cases where BP prevents PRs from merging but Tide does not know about it).

Yeah this is really my main gripe, the behavior came as a bit of a surprise and it took some digging for us to understand what was going wrong. I see it as more of an implementation detail how it's addressed, but would like for the error to be more transparent. Do you have any suggestions?

petr-muller commented 2 months ago

Is there a doc/guide anywhere on how to properly set up GitHub, and branch protection in particular, for Prow/Tide?

Unfortunately I'm not aware of any. Many advanced branch protection features enforcing PR merge eligibility are younger than Tide and the big Prow orgs (kube, openshift) are mostly not using them. Generally Tide expects a PR to be mergeable once it passes Tide's criteria (query match + required jobs pass) and considers inability to merge to be "user errors".

I think that in today's world this does not stand. I'd prefer the solution to be

  1. Tide to properly surface any occurrences when it thinks it may merge but is disallowed to do so
  2. Tide to check as many mergeability properties as possible in its isAllowedToMerge method to avoid thinking it may merge a PR when GH would disallow it

It is not entirely straightforward. 1) is a problem because the status context (which right now is the only PR-facing Tide user interface, maybe with the exception of Tide PR dashboard which people don't really use that much) is controlled by a separate controller than the one that actually tries to merge. 2) is also not easy because checking mergeability for BP would require Tide asking about more properties of the PR (and possibly the repo, to check BP requirements).

sdowell commented 2 months ago

I think that in today's world this does not stand. I'd prefer the solution to be

  1. Tide to properly surface any occurrences when it thinks it may merge but is disallowed to do so
  2. Tide to check as many mergeability properties as possible in its isAllowedToMerge method to avoid thinking it may merge a PR when GH would disallow it

Ack - closing this PR as the solution should take a different approach