search

gnolang / gno

Gno: An interpreted, stack-based Go virtual machine to build succinct and composable apps + gno.land: a blockchain for timeless code and fair open-source.
https://gno.land/
Other
900 stars 378 forks source link

PR Review throttling #2515

Open moul opened 4 months ago

moul commented 4 months ago

I propose that we define and set up a throttling mechanism for PR reviews to encourage contributors to make better contributions and discourage laziness.

In the future, we'll have various DAOs, including the "Code Review" one, where we'll be able to assess contribution quality and create per-PR and per-user reputation profiles. But in the meantime, I'm looking for a system where we give a higher chance of getting reviewed first to good contributors and newcomers, and progressively slow down the reviews of non-newcomers who aren't making an effort to provide high-quality contributions.

We face two challenges here:

  1. Define a simple rule set. I propose that the more often a contributor switches their PR to "non-draft" status, the slower the review will be. Reviewers should make rounds of reviews and maybe manually set the reviewed PR as draft again, so the contributor can spend time fixing before requesting a new review. Other ideas?

  2. Determine how to slow down reviews. One solution could be to use labels, which would allow reviewers to filter PRs. Another solution could be to create a GitHub project with a "weight" column managed by a bot, which could be used to sort PRs by weight. Over time, we could improve the weight calculation to manage more cases.

thehowl commented 4 months ago

I think, more than anything, that we're facing an issue of "prioritization"

We don't want to throttle PRs, but perhaps spend less time of the core team "schooling" contributors, and making sure newcomers and high-quality contributions get reviewed & merged faster

But then again, it has to be a system where even the PRs with low priority, after a while, do get at the top of the list. Otherwise, we just end up with PRs that are forgotten forever (which is similar to what's happening and happened).

There's another thing which I started discussing with @Kouteki, more in general, relating to how we can try to scale reviews: start delegating code reviews to high-quality non-staff contributors which have a proven expertise in a given area. I'm thinking @jefft0, @tbruyelle, @notJoon. I know it's not part of their job descriptions, but if the Gno core team is to be a mix of those who are good "OSS maintainers" and of "researching" programmers (ie. without much of a tendency to do a lot of reviews / codebase management); then we should extend the OSS maintainers beyond the core team. (This is something bound to happen anyway eventually; like Go core team is not composed only of people working at the Google Go team).

moul commented 3 months ago

Everything @thehowl said is correct - but this is another problem that needs to be addressed, and the sooner we define the scoring system, the sooner we can start counting points and adjusting the system itself.

This proposal aims to incentivize contributors to provide high-quality work and discourage laziness, by slowing down the review process for those who frequently switch PRs to "non-draft" status without proper preparation.