Federation for organization, repositories and users

[jonasfranz]

See https://owncloud.org/features/federation/

I would like to see a feature like the federation feature of next cloud. It gives the ability to share repositories, organizations or users between multiple Gitea instances.

This has the advantages that business users could share their projects with other companies. This feature would reduce the management and administrativ overhead.

This makes it easier for beginners to start with Gitea.

It could uses the "Mirror"-feature of Gitea.

[kolaente]

It would probably be enough to support federated repositories

[jonasfranz]

@kolaente The federation feature make explicitly for users sense. If you want to share repositories you should use the mirror feature. But is would be very convenient for the project manager to add users from other git instances.

[strk]

See also #184 (is this a duplicate of that?)

[kolaente]

@strk kind of, but i think this one goes further

[jonasfranz]

@strk Kind of. But this issue includes a full integration of the federation feature for organization not only for pull requests.

[strk]

You mean for authorization of remote users ? As I think it's useful to split "federation" in many different small things to implement, or a single ticket would get just too big.

[jonasfranz]

@strk I agree with the idea to split this thing into many tickets. This ticket might be the user federation ticket doesn't it? But I don't want authentication for users of other platforms. I want the ability to add other users. The user will receive an invite from the user's Gitea instance. He will access the repository or organization from the his instance.

[strk]

Granting permissions to someone in a federation requires being able to identify that someone (a global address). As you mentioned owncloud I think owncloud uses "user>@<domain" as the identifier, but I dunno what protocol it uses for that. Friendica/GNUSocial and other OStatus implementing federations can also use "user>@<domain" mapping to something else via the Webfinger standard (which is open as to allow for specifying different protocols). Another community (the IndieWeb one, see indieweb.org) uses web addresses to identify people, as it's used with OpenID up to version 2.0. New OpenID (OpenID-Connect) uses again Webfinger.

[jonasfranz]

I think that the webfinger standard might be a good solution. But I think that it also could conflict with the git email of an user.

[strk]

Where's the conflict ?

Rather, what I dislike about Webfinger is that it needs control of the domain root (which you don't have with OpenID URL).

[schmittlauch]

Regarding "what standard do we want to choose" I just want to point you to ActivityPub which is currently being finished by the Social Federation working group of the W3C. Some project implementing it (or currently working on that) are pump.io, Mastodon and MediaGoblin.

They don't use WebFinger though as they dislike the idea of a fixed .well-known path, but there are ideas about interoperability.

[JasCodes]

This feature will be truly game changer; keybase.io recently came out with client side encrypted git, thats also interesting.

[MatejLach]

Just want to point out that ActivityPub is now done.

Adding support for AP would make Gitea compatible with a growing number of federated servers, such as Mastodon, PeerTube, NextCloud and HubZilla, broadening its reach considerably, not to mention a standout differentiator against GitLab. It would also have the potential to dethrone GitHub as the go-to hosting for open-source projects, since most are here for the community pull request workflow that AP would allow in a decentralised manner, increasing privacy and eliminating a single point of failure for a large percentage of the free software community.

Anyway, my hope is this gets implemented, it has the potential to be revolutionary!

[tboerger]

As already stated within the chat, ActivityPub in go is a PITA because it's hard to do in a static language like go.

[MatejLach]

@tboerger Interesting. The ActivityPub spec is quite inheritance-heavy OO, but that should be possible to model in Go with struct embedding, however as far as I know, there's nothing like serde in Rust (https://docs.serde.rs/serde_json/value/index.html), which simplifies things quite a lot, however there's some effort to implement ActivityPub in Go, which may be a good start since it not only already implements json-ld parsing, but also defines the core vocabulary for ActivityStreams.

What specific annoyances are you thinking of here?

[JasCodes]

@MatejLach Another project https://github.com/go-fed/activity

[yookoala]

Relevant proposal in gogs repository: https://github.com/gogs/gogs/issues/4437

In Gitlab's repository: https://gitlab.com/gitlab-org/gitlab-ee/issues/4517

[cwebber]

Hi! ActivityPub co-editor here. I'm fairly busy at the moment but I'd like to see this happen... if you need questions feel free to ping me, or ask in #social on irc.w3.org

[cjslep]

Please do not hesitate to reach out to me on Mastodon for any questions/concerns/comments surrounding the https://github.com/go-fed/activity library that @jas99 mentioned. I obviously have a vested interest in the outcome of the decision, but would happy to provide candid information about my experiences working in the go+ActivityPub intersection. I agree with @tboerger that implementing ActivityPub in go is a steep cliff.

[lunny]

Maybe we could create a new repository named index and set up the new domain index.gitea.io to do that?

[jonasfranz]

Why do we need an index server? @lunny

[jaywink]

Would be awesome if we could have different projects discussing a common implementation of the ActivityPub protocol (ie using the same extension for verbs, etc). This would enable users of gitea, gogs and gitlab to work together seamlessly just like users of various social media platforms that federate can discuss seamlessly.

Could this be the place? -> https://github.com/git-federation/gitpub

[jonasfranz]

@jaywink great idea!

[r0bbie]

This would be amazing! I think Nextcloud (/Owncloud) is the perfect example of how this could work with the ideal implementation, as @JonasFranzDEV suggested.

With Nextcloud, if I want to share a folder with a user on another instance, I can easily do so and setup a shared folder across both instances.

If I could do that for a repository hosted on my Gitea instance, granting a user on another federated instance access to the repo, with that repo then appearing in their Gitea interface with full ability to interact with issues etc (with some clear denotation in the UI that this repo was hosted on another instance), that would be pretty amazing.

The goal being discussed to adopt a shared standard between Gitea and other open source self-hosted projects (such as GitLab CE) definitely makes sense, and it would be great if this was adopted allowing federation between Gitea, Gogs, GitLab, etc.. Migrating from GitHub for private projects is easy with nothing really lost, but for public open source projects we need to acknowledge a big benefit of GitHub is the community. I've discovered a lot of projects actually on GitHub. If there was some way of federating popular projects, users activity feed (i.e. being able to follow a friend on another instance and follow their activity, liked projects, with privacy settings taken into account etc) would be excellent - if that would be possible.

Federated pull requests would be a great first step towards this (#184), and likely the most crucial missing feature at the moment.

[mrenvoize]

11 months since the last comment here.. I'm wondering if there are still plans afoot?

[lafriks]

When there is some kind of standard agreed upon than yes

[nogweii]

Current discussions are happening as part of the forgefed project, so follow along there if you want to know more: https://github.com/forgefed/forgefed

As @lafriks mentioned, once there is a agreed standard, then various projects can implement it.

[strk]

The correct URL for forgefed project is now https://notabug.org/peers/forgefed

[dessalines]

Seems like this should be of prime importance now, considering github is now removing accounts of ppl from entire countries.

[jaywink]

ForgeFed also now has a discussion forum. Would be great to get someone from Gitea involved.

[teotikalki]

+1 for this. Working from a local self-hosted instance but not being isolated due to that choice would truly be the best of both worlds.

[jaywink]

The ForgeFed working group would desperately need developers from the current forges to give comments: https://talk.feneas.org/t/working-group-instructions/196

[teotikalki]

I'd just like to add that before Microsoft inspired a mass migration away from Github this wouldn't have been a killer feature... NOW it is. Less and less of the repos for OS projects that I'm researching are on Github now (MAYBE mirrored here).

I've read that ones Github commit history can read like a cv and that one of the reasons that the software world is so career mobile is that a person can self-teach valuable skills, EASILY DEMONSTRATE them (public github history), and thus qualify for better earnings/etc. If your contribution history is spread out over dozens of private servers it is FAR less visible/useful.

Also, any of those servers could go down at any time and that part of your history is now gone. A proper implementation of federated repositories would mean that I could participate in dozens of projects on dozens of instances (from my instance) and if they ALL went down I'd still have my 'federated git profile'.

[aschrijver]

Linking to the ForgeFed roadmap (there's funding for those who'll work on it):

https://notabug.org/peers/forgefed/issues/87

[zyansheep]

Perhaps, for a simple implementation, gitea could run an ipfs node in the background hosting local repository files (using ipns to point to the latest versions of the repositories). We could then have a simple gossip protocol implementation to find other gitea instances and get ipns hashes & preliminary data (stars, name). The benefit to using ipfs would be when users on one instance want to view or fork repositories on other instances, it would contribute to hosting parts of that repository and make accessing the repo faster for network as a whole.

Using ipfs/ipns could also be used for distributing user data such as starred repositories, pull requests, bio, etc. each node would only store names & ipns hashes for users on other networks that the instance cares about and it would be trivial to request profile data for unknown users.

ipfs already has a go implementation and for instance discovery something like this could be used.

[remram44]

There is no requirement for peer-to-peer storage here, what you describe doesn't seem required or even related to the problem at hand. I don't think there's interest in moving away from the Git storage format and transfer protocol. Maybe you should open a separate issue if you see a benefit here (I certainly don't).

[zyansheep]

There is no requirement for peer-to-peer storage here

I think gitea would benefit greatly from peer-to-peer sharing of repositories so that popular repos would be redundant in the case of instances going down. While someone can just mirror a repository to their own instance, it would fragment the development of a project if there is no central place to commit to. If the git protocol was layered on IPFS, it would allow for deduplication when forking projects on a single instance (so an entire copy wouldn't have to be stored).

what you describe doesn't seem required or even related to the problem at hand.

The reason federation is so useful (and why people want it so much) is that it allows collaboration across instances. The InterPlanetary Name System (IPNS) has identical behavior to OpenID because it can be used to identify a user who has permission to update certain data (e.g. their repositories and personal profile). An IPNS address could uniquely identify a user from another instance without necessarily having to tie that user to a specific instance. Lets give an example: Alice is self-hosting an instance of gitea at aliceland.net When Alice creates a new account, gitea would create an empty profile, host it on IPFS and then generate a unique IPNS address that points to that profile. Whenever Alice creates a new repository or updates her profile in any way, gitea will (behind the scenes) create a new IPFS record, unpin the old one, and reassign Alice's IPNS address to it. Now lets say Bob wants to submit a merge request from his mirror of the repository on bobland.net to aliceland.net. When Bob originally forked Alice's repo to bobland.net he made a note of the IPNS of Alice's repo as well as the IPFS location of the commit he forked from. When Bob wants to open a merge request, he writes his message and then bobland.net will put the following things in an IPFS block:

• Bob's IPNS user address
• The IPFS address of the commits Bob wants to pull in
• The IPFS address of the commit in Alice's repo that should be modified
• Bob's Messsage and Title for the merge request
• Signature of data with Bob's private IPNS profile key

Bobland.net will then send the IPFS address to aliceland.net which can then choose to totally ignore it, OR parse the address, verify the commits, create a IPNS address for the comment thread (that points to all the comments) and then notify Alice that some guy named Bob on instance Bobland.net has sent a merge request for 3 commits over IPFS. The comments would also be hosted over IPFS and accessible via an IPNS address. This pattern of using IPNS for mutable data (like a comment thread) and IPFS for immutable data (like comments and commits) can be applied for most inter-instance federation.

I don't think there's interest in moving away from the Git storage format and transfer protocol.

This approach to federation wouldn't have to move away from the Git format. Git can simply be layered and stored on ifps (while also taking advantage of deduplication). Git's Merkle Tree system doesn't necessarily have to be integrated with IPFS's (although that would be cool) and the git transfer protocol would still be the same, IPFS would just moderate the communication between instances.

[remram44]

Can you just open a separate issue? This one is about something specific, and ForgeFed is already working on a protocol. Better yet, bring it up with them.

Piling on issues with comments like "what about ipfs/filecoin/blockchain" just seems rude.

[secondtruth]

+1

GitLab is discussing this feature, too: https://gitlab.com/gitlab-org/gitlab/-/issues/6468

[cjslep]

I pinged on the gitea dev discord a few days ago as a FYI, and have been proactively been trying to reach out to some of the folks behind ForgeFed, as with go-fed v1 being done & documented it would be nice to get an instance of gitea federated on ActivityPub though it is no small effort. I think the gitea folks are busy w/ other priorities atm. Unfortunately, I have had no success getting a hold of any of the ForgeFed folks. :(

Some of us in the ActivityPub community, coming off APConf2020, are experimenting with having a grassroots lightweight documentation process on a gitea instance and it feels weird to not be able to federate with it yet.

[delbonis]

Git already has all the features necessary for decentralized mirroring, the only functionality missing is what's necessary to coordinate it, which is exactly what ForgeFed does. IPFS has no need to be here, and given how disastrously their mainnet launch went I'm not sure it's safe to get to deeply tied with other projects by Protocol Labs.

[thebalaa]

I'm interested in collaborating on any of the existing initiatives. Let's try to put a working group together. Can suggest this Matrix channel for further discussion #noteworthy:tincan.community

[cjslep]

FYI, I've kicked off the specific design discussion of federating via ActivityPub + ForgeFed in #14186. I would like specifics to stay there, so if anyone has more of the big-picture or strategy-level concerns ("why AP + ForgeFed and why not X"), this place remains intact for that purpose.