Support federated pull requests #184

Open stevenroose opened 7 years ago

From @stevenroose on May 25, 2016 11:24

Currently, users can only make pull requests if they have an account on the same Gogs instance. It should also be possible to make pull request from external repositories like GitHub or other Gogs/GitLab repo's.

This could be integrated with https://github.com/gogits/gogs/issues/1297 and https://github.com/gogits/gogs/issues/3130.

Copied from original issue: gogits/gogs#3131

From @roblabla on May 25, 2016 12:9

Somewhat related : https://github.com/gogits/gogs/issues/2210

This is the number one feature for a personal hosted Git service! Would be even greater with #185 !

This could be integrated with git-appraise integration too ?

@ekozan formal proposals are welcome, but I do see git-appraise integration could be a good companion for federated pull requests (to basically have reviews travel across federated nodes with the rest of the code, right?)

GitLab is tracking this issue here, so maybe you could word out a streamlined workflow. They are planning to mention the feature during their summit.

@bkcsoft maybe you can help with keeping the GitLab specs open enough to allow for federating PR between GitLab and Gitea too ?

@strk is @bkcsoft affiliated with GitLab?

@strk I could steer it in the direction of just sending patch-files between servers (maybe using webhooks?). Which is what I suggest Gitea should do as well. Makes it really easy not having to push/pull between servers :) @stevenroose Yes

Well, seems like they're already thinking or using git request-pull -p (which sends the patch along) so it should be cross-platform compatible :slightly_smiling_face:

They are planning to mention the feature during their summit.

https://gitlab.com/gitlab-org/gitlab-ce/issues/4013 is tagged as moonshot, unassigned, no milestone and no MR in sight. So maybe not get your hopes up just yet :slightly_smiling_face:

@bkcsoft We might take the lead here :) If we can get GitLab and GitHub on board, that would end the locking currently imposed by these platforms

@bkcsoft We might take the lead here :) If we can get GitLab and GitHub on board, that would end the locking currently imposed by these platforms

I can't believe that GitHub wants to solve the lockin issue :P

No but if other platforms lead the way, people might demand they follow. And people might migrate :)

Software like Gerrit kind of allows for that

@stevenroose do you have reference about the Gerrit support ?

If we implements Gerrit, could we invite Golang team to use Gitea? 😄

@strk With Gerrit, you package your commits using git-review and push them to a certain ref and it will show up in Gerrit as a code review. No need to create a Gerrit fork. It does not use git request-pull though.

You'd still need write permissions to the repo to push those reviews to the ref, right ?

In that case the missing bit would still be eventually tracking a different repo holding the review ref ?

Yeah it is different. It pushes individual bits with write permission.

With federated PRs, Gitea should periodically (or on request) check the branch reference for new changes.

AFAIK, git request-pull does not use git commits at all. It merely generates a list of commits between local and remote, and print it on stdout. We'd need to specify a way to send those changes to remotes/to pull them from remotes

Git request-pull is part of the standard git install though, unlike git appraise or git review.

@roblabla Yeah the flow would be to save the git request-pull to a file and upload that (like how submitting patches used to work in the past). Or either enter a URL to a branch of a remote repo so that Gitea can update the PR.

Unless I'm missing something git request-pull references a specific commit, while github/gitea pull requests reference a (possibly moving) branch.

Do we want federated request to track branches rather than specific commits ? I think we do want a PR (thread of comments/discussion) to track a branch.

it references specific commits yes. So we'd need to continuously re-fetch the data 😒

On Tue, Dec 20, 2016 at 12:20:34AM -0800, bkcsoft wrote:

it references specific commits yes. So we'd need to continuously re-fetch the data 😒

We can only fetch new commits if we have a reference to a branch name, which is not in git request-pull output. So if we want to keep the track-a-branch approach, we cannot rely on git request-pull.

A PR creator could ask for a remote URL and a branch name, check out the remote branch locally, perform some checks (ie: refuse to create PR with a huge number of changes), and create the PR.

Then a button and a webhook could be then made available to fetch more commits, if any, from the remote branch. Only the PR author should be given the ability to request updates. Placing a git hook on his fork to hit the destination webhook would make the update experience smoother.

--strk;

I would suggest that when creating a PR as a guest account, you'd have a tab with "external" or "federated' or whatever that has two options:

a text input field for a branch URL, with a "fetch" or "test" button maybe to check for reachability and availability of tracking
a file upload field to upload a .diff, .patch file or the output of a git-request-pull; this might also be a textarea

In the latter case, once the PR is created, users should be able to overwrite their previous file in order to change the commits in the PR.

Also, in the case of unavailability of automatic branch updates, you could require a user to manually trigger a refetch. In almost all of the cases, users commit to a PR'ed branch exclusively in the light of the commit, so they can just refetch whenever to update the branch. (Take into account that often, updates to a commit are because of feedback in the PR, so they have the page open anyway.)

As my thesis promotor put it: "The real opposition to a change does not come from people who are against it but from people who keep saying it's not enough."

I don't think this feature usefull, you can add two remote for this case. for example branch github for gh, master for gogs, I use both in my work with one repository. and git review is another things. don't make big feature to solve little problem. anyway, gogs focus on small business or self hosting. not for public cloud service.

@renothing I don't think you fully understand the proposal. It has nothing to do with being able to compare or checkout code from different remote sources. Instead, it is about allowing people to submit pull requests or patches without having to be registered to your system. If I publish a piece of software on my Gitea instance, I want others to be able to create a PR without requiring them to go through the hassle of registering, forking, pushing their changes to my instance and then creating a PR.

@stevenroose
as I said, it's quite a little demand scene. think about it, how many people need merge a PR from gist? I don't think it's necessary to add this feature to gitea core. maybe it's ok for a plugin when gitea plugin architecture ready.

@renothing If I understand the issue being tackled here correctly, it goes far beyond merging just a simple gist. It tackles merging between multiple gitea different instances, and even between a GitHub repository and one on a gitea instance. It allows one to merge changes from any remote branch on any supported git server on the internet.

@stevenroose Correct me if I'm wrong :smiley:

@sbrl you're right, but if somebody want to do contribution for your repo, he will need fetch your code first, he must have to keep two remote(one for himself, one for yours). it didn't reduce works at all. this feature just reduce login/register step. it didn't help too much, maybe send email path is better choice like linux kernel.

i'm not ok with you @renothing

When you work with git you have always many remote
It's a must have
login step are fucking boring
gitlab will do it too : https://gitlab.com/gitlab-org/gitlab-ce/issues/4013
it's a main feature so it's not a plugin

I'm all for having support for federated pull requests, anyway I don't think they could replace logging in, as you still want to somehow grant/revoke permissions to file PRs and comment on the corresponding thread.

The login part should be taken care of by OpenID Login (there's another ticket about that part).

@renothing As far as I can tell, in actual fact it'll be the server that automatically fetches the code from the remote automatically, meaning that the dude doing the pull request just has to push to a branch on his own server, and doesn't have to have 2 remotes (Again, correct me if I'm wrong).

Besides, if you don't like it, I'm sure there will be an option to disable it - nobody is telling you that you have to use it :smiley:

Indeed. You might used GitHub, I use my own Gitea. If you want to contribute to my project, I don't want you to have an account on my Gitea instance, because you use GitHub. I want you to push a feature into a branch on your repo on GitHub, then come back to my instance, the project homepage and file a PR from your GitHub branch into the master one. Without you needing to create an account, all you do is login with your GitHub account, (maybe do a CAPTCHA) and paste the URL to your GitHub branch.

As I maybe observed in another PR (the OpenID one) even if someone logs in with a remote authentication mechanism (OpenID in my case, GitHub/OAuth2 in the example you make) he still needs a record onto the local instance, if not else to be able to partecipate in the discussion about his proposed changes, and maybe get notified by mail of answers.

So allowing to reference branches on remote git servers has mostly to do with removing the need for a contributor to be granted permission to and actually create a new git repository for code he already hosts somewhere else. To give basically the freedom to host forks on arbitrary hosters.

Maybe one day we'll get fine-grained permissions enough to be able to have users that can send PRs but not create local repositories/forks.

to keep gitea core simple,I don't like this feature at all. it will make core features more complex, not only pull request system, also with permission system and hard to integration with 3rd CI system. anyway, the main authors have rights to choose.

@renothing Well, the main authors aren't the ones that should choose, it's the users that should.

On CI, I don't think that's an issue. What CI does is just pull a branch and run some scripts. Since the CI system is probably separate from the Gogs instance anyways, it really makes no difference to pull a branch from the Gogs repo or from any other remote repo.

Also, a permission system should be in place anyways. But it shouldn't be more complicated than just having an extra checkbox for "Guest accounts". You have permissions like commit, make issues, make PRs and comment and then you would have three kind of users members of the repo, registered accounts at Gogs, and Guests. Ez πz

The idea of a single flag for a "guest account" seems interesting. But then no guest could be "watching" changes, right ?

You'd be filing a federated PR and not get the comments on it, unless your email (and prefs) are known by the system.

Do you agree that a local record must be created anyway for the user ?

@strk. You would. Guests login with OpenID, so their e-mail address would be known. The user db should just make a distinction between "real" users and guest ones.

With "not member of this instance", I didn't mean that there was no record of them whatsoever in the db.

You'd still need to know which email (and thus user) to notify upon receiving comments on a ticket, so there must be an association between a ticket (PR) and a "user" (guest or not) - this means still having a record in database, for that association.

BTW, OpenID provided email isn't necessarely trust/confirmed, so if you want to be sure about email you still need to take care of that

Interesting design document by NotABug people with ideas about a federated git hosting service: https://notabug.org/NotABug.org/notabug/src/master/README.md

OAuth for GH/BB/GL could be used to link once account, then if you allow it Gitea lists your remote repos and branches and you can create PRs to any repo that has a common ancestor (finding that is gonna be such a pain...). Only issue I see is that all the other system (GitHub, BitBucket, GitLab) has different APIs for fetching this data, we'd need to support all or none. I say plugins :trollface:

@strk That design doc look nice in theory, but in pratice no one supports it (at least GH/BB/GL doesn't, feel free to point out someone that does, OR at least has actuall intent of doing so) so in reality it's mainly trivia until at least one other has implemented it.
One might counter that with "but we could be first!", if we're first and then everyone else settles on another "standard" we're left stranded and have to take personal hobby time to replace our existing flow with said other standard, possibly breaking the current flow. If we instead use some existing technique that sort-of viable, and the others settle on something we could implement that instead. But until said time I'd refrain from introducing "federation" when it's not exactly federated. Do it once, and do it right 🙂

This is really a chicken and egg problem. Until someone implements federation, nobody will implement federation.

EDIT: I'd also argue that if we can federate across gogs instances, we've already made a first step towards federation.

@roblabla Right, I'm not against pull requests across instances, what I am saying though is use existing functionality, such as the various providers APIs :)

I'd prefer a generic solution to one that is bound to known API, especially from proprietary services.

I agree with @strk. Git itself is quite powerful, I would try to resort as much as possible to core Git functionality.

If federation is implemented f.e. based on Git URI's, adding UI sugar for hosting providers is not that hard. For Gitea, we use our own API's to extract information from the PR. For GH/GL/... we can either not support them, so just support copy-paste Git URI's, or implement a UI widget to retrieve these URI's from the API's. Or even simpler, map PR URL's to Git URI's.

One issue is that there is no standard way to represent a branch in a remote repo, as far as I'm aware. I think that I saw git://provider/repo.git#refs/mybranchref before, but I'm not sure how standard that is.

On Wed, Mar 22, 2017 at 09:47:06AM -0700, Steven Roose wrote:

One issue is that there is no standard way to represent a branch in a remote repo, as far as I'm aware. I think that I saw git://provider/repo.git#refs/mybranchref before, but I'm not sure how standard that is.

I don't see how that's an issue. Just say that in order to submit a "federated pull request" you'll have to provide all the informations that "git request-pull" asks you to provide ...

It doesn't necessarely be a single URL, can be a whole form too...

How about asking GH / GL for their feedback on a proposed standard? Could be an opening into a wider discussion on the subject

Sure, but we need that proposed standard first ...

Pagure (https://pagure.io/pagure) system supports remote pull request, which I found useful while working on Fedora packages repository. You do need to be registered to make a remote pull request.

See: https://docs.pagure.org/pagure/usage/pull_requests.html#remote-git-to-pagure-pull-request

Related: https://github.com/gogits/gogs/issues/4437

go-gitea / gitea

Support federated pull requests #184