Bot to automatically generate cherry-picking PRs

brettcannon commented 7 years ago

Since the decision has been made to work with cherry-picks instead of forward merging of branches, it would be convenient to have a bot that would generate pull requests based on cherry-picking for any pull requests that affect multiple branches. The most likely design is a bot that monitors merged pull requests with key labels applied that delineate what branches the pull request should be cherry-picked into. The bot would then generate cherry-pick pull requests for each label and remove the labels as the pull requests are created (this allows for easy detection when automatic cherry-picking failed).

soltysh commented 7 years ago

I'm interested in helping with this one 😄

brettcannon commented 7 years ago

Yeah, I think this will be one of those high-impact improvements that will definitely make PR acceptance more pleasurable compared to the hg workflow. I think this should be number 2 on the TODO list once #6 is dealt with (as I see the NEWS file being the stickiest point in accepting PRs and it also makes this issue easier by lowering potential merge conflicts).

berkerpeksag commented 7 years ago

Note that I already have a working bot (tested it on https://github.com/fayton/cpython), but I didn't had time to clean it up and make it open source in the past months.

My plan is to work on bug fixes for 3.5.3, 3.6.1 and python.org over the holiday. I can help you after mid-January if you had time to create a prototype during the holiday (or you can help me to clean up my bot if none of us can work on a prototype before mid-January -- I codenamed it "Duke Nukem Forever" :))

soltysh commented 7 years ago

👍 @berkerpeksag we can sync and I'd love to help with that.

berkerpeksag commented 7 years ago

Just an FYI, I've finally cleaned-up my bot and set up on my test repository. There is a missing feature though: it doesn't create a new pull request; it automatically pushes the cherry-picked commit into maintenance branches.

See https://github.com/fayton/cpython/pull/2/#issuecomment-278313431 for an example. Note that it also automatically fills the committer field by inspecting the "core developers" team created on the organization.

brettcannon commented 7 years ago

What about if the cherry-pick fails either due to testing or the code not patching cleanly? And how do you keep track of what has (not) successfully been cherry-picked (I'm assuming some label on the original PR)?

Carreau commented 7 years ago

What about if the cherry-pick fails either due to testing or the code not patching cleanly?

The other case I have seen is auto backport of an already backport patch. So the 2 regular failure of one of my bot is either:

empty commit after cherry-picking.
conflict.
Other (remote commit disappeared in the meantime/ GitHub down/ API rate limited.).

Seeing that now with github you can edit in the UI, you could try to commit and push the conflict on another branch, and resolve it in the GitHub UI as well.

berkerpeksag commented 7 years ago

What about if the cherry-pick fails either due to testing or the code not patching cleanly?

It reverts to the current state of upstream branch and adds a comment to the original pull request.

And how do you keep track of what has (not) successfully been cherry-picked (I'm assuming some label on the original PR)?

I'm using a simple sqlite database to keep track of the state [1], but we can add a label to the original PR too.

[1] This way it's also easy to quit early if the commit has already been cherry-picked.

WhyNotHugo commented 7 years ago

Since the decision has been made to work with cherry-picks instead of forward merging of branches, it would be convenient to have a bot that would generate pull requests based on cherry-picking for any pull requests that affect multiple branches.

Out of curiosity, why was this decision taken? PEP-0512 doesn't seem to explain any of the benefits of doing this.

brettcannon commented 7 years ago

@hobarrera couple of reasons. One is people instinctively create PRs against master on GitHub and merge-forward wouldn't like that. It also allows you to accept commits on master while any discussion about backporting happens. Finally it makes the 2.7 no longer the odd-ball branch that never merges anywhere.

brettcannon commented 7 years ago

Just so no one feels too much pressure for getting a bot up, we could also write a Python script to automate this locally as a hold-over. I would expect a CLI like backport --commit <hash> --versions [comma-separated Python versions] [--upstream <remote for python/cpython>] [--fork <remote to personal fork>]. This would do:

git fetch {upstream}
for version in versions:
    git branch {version} backport-{version}-{commit}
    if git cherry-pick ...:
        git push {fork} backport-{version}-{commit}
    else:
        print(f"cherry-pick for {version} failed", file=sys.stderr)

Could even get really fancy and if someone provides GitHub credentials have the PR also be automatically created.

Carreau commented 7 years ago

https://github.com/minrk/ghpro may be of help:

backport-pr apply 4.x 1234

Made to work with merge commit, but I'm guessing that shouldn't be hard to change.

ncoghlan commented 7 years ago

Having just done the backports for https://github.com/python/cpython/pull/149 I don't think this will work very well until after Misc/NEWS is dealt with :)

Mariatta commented 7 years ago

I started working on my own cherry-picking script 😛 It's here in case anyone wants to try it out: https://github.com/mariatta/chic_a_cherry_picker It does not handle failures or merge conflicts :( And ... I tested it in production 😅 https://github.com/python/cpython/pull/474, https://github.com/python/cpython/pull/475, and https://github.com/python/cpython/pull/476 were cherry-picked using this script.

brettcannon commented 7 years ago

@Mariatta as long as it errors out when a cherry-pick fails due to merge conflicts I think that's acceptable (especially for simple cases like doc updates that don't touch Misc/NEWS).

Mariatta commented 7 years ago

Hi, I'm just starting to look into turning cherry_picker.py into a bot. Currently, each of us push the backport branch into our own fork of CPython. When implemented as a bot, where should the backport branch be pushed to? Will it need to have it's own GitHub repo and account? Thanks 😃

ncoghlan commented 7 years ago

The bot will definitely need its own account (as with the Knights and Sir Bedevere), but won't necessarily needs its own repo. Specifically, we have a free choice between two architectures:

The bot has its own dedicated fork, and submits PRs from there
The bot makes its working branches directly in the main CPython repo, and deletes them when the PR is merged

The main argument in favour of (1) is that it makes the bot's workflow consistent with the manual workflow we're currently using, which means we can share the automation scripts. It also has the advantage of continuing to keep PR working branches in the main repo to a minimum.

I don't actually see any strong arguments in favour of (2) - we can use that model ourselves as maintainers, but we don't, in order to keep our maintenance workflow as similar as we can to the external contributor workflow.

ncoghlan commented 7 years ago

Naming Tangent: can we call the cherry picking bot Dennis?

Rationale: http://www.intriguing.com/mp/_scripts/peasant.php :)

berkerpeksag commented 7 years ago

It would be great if we all could stop working on our own bot at the same time.

brettcannon commented 7 years ago

@berkerpeksag @soltysh @Mariatta who has code for a bot and where is it?

Mariatta commented 7 years ago

Sorry I haven't actually started working on any bot :slightly_frowning_face: Please continue on :smiley:

berkerpeksag commented 7 years ago

Sorry, I missed this earlier. I have code and a test repo on GitHub. I still need to publish the source code on GitHub, but I can only work on it weekends and I usually don't have a reliable internet connection on weekends. I hopefully will do it this weekend. Thanks for the ping!

Carreau commented 7 years ago

I do have this which I need to convert to gitgethub, and we are using to backport for IPython/Jupyter.

Mariatta commented 7 years ago

Hi,

I created the repository to host the cherry-picker bot at https://github.com/python/miss-islington

The bot's account has also been created at https://github.com/miss-islington

I made an initial PR: https://github.com/python/miss-islington/pull/1

According to Wikipedia, Miss Islington is the name of the witch in Monty Python and the Holy Grail sketch. Both Brett and Guido were consulted in choosing the name, and they gave me their +1s :)

The workflow:

PR that has "needs backport to X.Y" is merged
@miss-islington leaves a comment saying it will work on backporting it to X.Y
backport PR created
If there are errors, @miss-islington comments back saying it couldn't do the backport.

Some things to note:

In this workflow, @miss-islington will receive the credit for backporting the PR. In other words, @miss-islington will appear as a contributor to CPython. Misc/Acks entry is not needed.
What about CLA?
the "needs backport to X.Y" label must already been applied before you merge the PR on master.
if you don't want PRs to be automatically backported by @miss-islington, maybe don't apply the "needs backport to X.Y" label? Or only apply it after you merge? I think it's still useful to have this label to remind ourselves that this PR still needs to be backported somehow.

Any thoughts? Anyone up to reviewing the PR?

terryjreedy commented 7 years ago

Thank you for persisting with this. New 'autobackport 3.6' and 'autobackport 2.7' labels would a) let you merge the PR without being sure it is perfect, and b) let people volunteer to test it -- live -- or not. Having asked for it, I would. The only way I can 'review' the patch, beyond a cursory reading for apparent sanity, is to test it in action. As long as you are sure that the PR will not push anything to cpython, or otherwise do damage, and think it should work, I am willing to try it.

Mariatta commented 7 years ago

Reopening the issue since the bot is not deployed yet 😅

Mariatta commented 7 years ago

@miss-islington made her first backport PR! https://github.com/python/cpython/pull/3369 🌮 :tada: Closing this issue. Any further problem can be reported in https://github.com/python/miss-islington

python / core-workflow

Bot to automatically generate cherry-picking PRs #8