search

gitcoinco / web

Grow Open Source
https://gitcoin.co
Other
1.78k stars 771 forks source link

As a repo maintainer, I want SourceCred integrated into Gitcoin, so I can payout all of my contributors at once. #2160

Closed owocki closed 4 years ago

owocki commented 6 years ago

User Story

As a repo maintainer, I want SourceCred integrated into Gitcoin, so I can payout all of my contributors at once.

Why Is this Needed

Summary: SourceCred has a great algorithm for figuring out who should get credit for development on an open source repo.

We want to provide an interface for where a funder can payout many contributors at once, and we think SourceCred could be a way to do this.

They are MIT Licensed, and Dandelion said we could do a prototype integration with them.

Description

Type: Feature

Current Behavior

No way to tip out all my contributors at once.

Expected Behavior

A user can input their github organization or gitub repo, and be spit out a list of users and the amount of cred they have.

I would like to see this information populated in the advanced payout view on Gitcoin (or ... at least a copy of that page )

Screenshot below of what that page looks like. Its already coded up.. We just need to copy it, and introduce a way of getting the SourceCred info into it! 1_ygjyhnowxjig3f6jzm_cha

Definition of Done

User can put in their org/repo name on SourceCred and get a prefilled advanced payout form on Gitcoin

Please propose an architeccture of how to bridge the two projects before you start coding!

owocki commented 6 years ago

cc @decentralion @ceresstation @coderberry for feedback!

coderberry commented 6 years ago

@owocki Is there a way to limit sourcecred to a specific PR, or will it be distributed based on historical ranking?

owocki commented 6 years ago

i dont think so.. @decentralion would know for sure tho!

wchargin commented 6 years ago

Is there a way to limit sourcecred to a specific PR, or will it be distributed based on historical ranking?

This is a good question. There are various answers depending on what you mean (but the short answer is probably “no”).

It’s important to note that the cred computation is fundamentally holistic: each connection can have far-reaching effects. If an issue is important, that makes its author more important, which makes other issues by that author more important. This “transitivity” is critical; it’s why the mechanism works at all.

For instance, one could imagine restricting the data given to SourceCred to only include interactions on a particular pull request (say, itself, its comments, and its reviews). Running SourceCred on this subgraph would generate pretty poor signal, because SourceCred is missing all the context and history of the rest of the repository.

But it is still clearly a useful thing to be able to ask questions like, “how much cred does @someone have from the backend section of the code? …from test code? …from this particular PR?” These seem like important tools for visibility/transparency, and also to recognize that contributions take many forms: documentation cred, UI design cred, and bit-hacking cycle-crunching cred are all valuable but certainly distinct.

We want to have tools to explore these questions in detail, but we don’t have them yet. Figuring out what exactly we want to ask and what it means mathematically are prerequisites to actually implementing anything like this.

There are some reasonable approximations that you can make today, though. For instance, you could run SourceCred on the graph of a whole repository, and then look at how cred flows from a PR directly to its neighbors, or to its neighbors’ neighbors (but with some precautions taken around high-degree nodes like users).

wchargin commented 6 years ago

Another approach: Run SourceCred to get a base attribution a0. Then, add a high prior on the pull request that you’re interested in—say, mark it as 8× more important than other pull requests. Re-run SourceCred to get an attribution a1. Compare the two attributions to see how they differ: nodes that have higher cred in a1 than a0 are “more important from the perspective” of the node granted a high prior.

I haven’t thought about this approach much, either. :-)

owocki commented 6 years ago

@wchargin these are really interesting thoughts... thank you for them! i agree that the holistic view makes more sense.

im going to bounty this and see what the community comes up with

gitcoinbot commented 6 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 250.0 DAI (250.0 USD @ $1.0/DAI) attached to it.

teamdandelion commented 6 years ago

Just to make this a little more concrete, here's a prototype instance of SourceCred running on Gitcoin.

teamdandelion commented 6 years ago

@wchargin and I aren't sure if things are ready on SourceCred's side for work on this issue to progress. we haven't created an API yet for retrieving data from SourceCred (https://github.com/sourcecred/sourcecred/issues/704), so currently the data is only computed in the browser, which isn't the most convenient place to get it from.

@mul1sh can you elaborate about when you've integrated SourceCred before?

mul1sh commented 6 years ago

@decentralion I came across the repo 2 months ago and just played around with it. Didn't do any serious integration really 😃 . Anyway i'll stop working on this issue and when SourceCred has an api then someone else can start work.

mbeacom commented 6 years ago

@decentralion Nice project! Agreed. It might be a decent idea for us to hold off on this integration until there is a concrete mechanism for retrieving the results outside of attempting to scrape the UI. Definitely looking forward to seeing how the project progresses!

teamdandelion commented 6 years ago

Cool, thanks for understanding all. I'll circle back once sourcecred/sourcecred#704 is resolved. A few different people have asked for this, and it is on the critical path for some major UI improvements I've got planned, so I expect I'll prioritize it before long.

owocki commented 6 years ago

@wchargin and I aren't sure if things are ready on SourceCred's side for work on this issue to progress

ah i jumped the gun on this.. canceling the funded issue now!

gitcoinbot commented 6 years ago

Issue Status: 1. Open 2. Cancelled

The funding of 250.0 DAI (250.0 USD @ $1.0/DAI) attached to this issue has been cancelled by the bounty submitter

mzargham commented 5 years ago

@owocki Is there a way to limit sourcecred to a specific PR, or will it be distributed based on historical ranking?

The answer to this is actually YES! (soon)

More details to come, but the SourceCred project at the Odyssey Hackathon introduced personalized PageRank into a fork of the codebase. In this implementation it is possible to specify a seed vector which can be one specific node and get cred scores relative to this point of reference.

Our prototype from the Hackathon is here: https://sourcecred.io/odyssey-hackathon/ note that in addition to working on the ranking algorithm itself, a lot of focus was put on capturing non-technical labor through a manual mode plugin and a visual explorer UI was created.

@decentralion is diligently working to integrate the hackathon work into the core SourceCred codebase, though they probably won't rush it, as SourceCred has a high bar for code quality, something hackathons are not known for.(Personally, I am anxiously awaiting code review from @wchargin). Interested parties can check out the pull request here: https://github.com/sourcecred/sourcecred/pull/1129

Looking forward to seeing more discussion on this integration.

kuhnchris commented 5 years ago

so @owocki did the talks go anywhere?

teamdandelion commented 5 years ago

From a technical perspective, SourceCred is ready for this integration (the CLI outputs scores, both total and time-scoped, so you could do all-time payouts or payouts for recent contributions). From an integration standpoint, right now depending on SourceCred is still a little cumbersome as we aren't yet published on npm--but that will change soon(tm). https://github.com/sourcecred/sourcecred/issues/1232