Fix: votes for quality review and operational groups should be internal to the group

[mbernst]

Our prior practice for the code review group was that nominations were voted on by members of the code reviewers group. Example:

We changed this to a vote of the entire Collective in the governance documents at the time that we redrafted a section of the documents so that the groups were all blank slate — because otherwise we wouldn’t be able to populate any of the groups! But since several of the groups are now not blank slate, I would suggest we codify our prior practice, since members of the group (like in guilds) are the ones with the expertise to assess someone’s abilities in quality review or operational group membership. I believe that was the original intent.

My proposal

I propose that we revise the governance documents for operational groups and quality review groups to state that while nominations are open to the entire community, the vote for inclusion is open only to members of the group. Special case: if the group is empty, the vote is open to the whole community.

Specifically, the text "To gain these privileges, the person must be nominated by a member of the Collective including a paragraph describing the individual's qualifications, and then a one-week vote in a proposal is open to all members of the collective.” Will be revised to say "To gain these privileges, the person must be nominated by a member of the Collective including a paragraph describing the individual's qualifications. A one-week vote in a proposal is then open to all current members of the group; or, if the group is empty, open to the whole Collective."

---

Use comments to share your response or use emoji 👍 to show your support. To officially join in, add yourself as an assignee to the proposal. To break consensus, comment using this template. To find out more about this process, read the how-to.

[shirishgoyal]

BREAKING CONSENSUS

If for empty group, inclusion can be decided by collective, what makes them ineligible afterwards. Why are we questioning their judgement later?

But since several of the groups are now not blank slate, I would suggest we codify our prior practice, since members of the group (like in guilds) are the ones with the expertise to assess someone’s abilities in quality review or operational group membership. I believe that was the original intent.

Except code-reviewers, my understanding is that the rest of the groups are blank slate right now. Please let me know if that is not the case.

As per the poll below, the original intent was to keep these decisions with the entire community as compared to resting it with the few who can exclude others if they so decide irrespective of consent of community to include them.

Suggestion

The voting rights for inclusion to quality review and operational groups should stay with whole collective as mentioned in the governance documents. This needs no fix as it was already deliberated by community and governance documents capture the actual intent already.

[mbernst]

Thanks @shirishgoyal! Two thoughts. First, empty groups seem to me to be more a special case than the norm, thus the special treatment. Second, the practice in the Collective was not a public vote, it was a vote of the existing members. (That's why I posted the screenshot above.) I believe that was the intent...it may have shifted without my knowledge. But, so I disagree that the deliberation wound up capturing the intent.

Of course, one option here would be to leave the consensus broken and thus have the group vote on what they actually intended. The argument for in-group votes is that it ensures qualified people are reviewing the person's credentials, and is consistent with many open source efforts. The argument against in-group votes is that the group can be enforcing boundaries arbitrarily.

[dmorina]

@shirishgoyal it makes absolutely no sense to have people vote on stuff they have no clue about or that doesn't even affect them in any way, this proposed method has been in place for code reviewers since beginning and it was fine, no need to change that now with new gov, so this is just doing a rollback

[mbernst]

Swinging back with a screenshot of what I was mentioning above, evidence that the SCRC felt that votes should be restricted to people with sufficient "standing" to make the decision:

Plus there was the pre-established norm that code review group votes should be internal to code review group members. Again, my feeling is that changing this practice should have happened through an explicit decision. We would have needed to talk about questions like: what would prevent even poor nomination from going through, since generally our community is friendly and 👍 s nominations? (We haven't had a case of this yet, but I really want to solve this before we face it and the objection is in the context of a particular person.)

[shirishgoyal]

I fully agree that nomination rules should be made stronger than taking voting rights from the community.

[mbernst]

Could you please rephrase that? I don't fully understand yet. I think maybe you're saying that you support changing tightening nomination rules instead of changing voting rules?

[shirishgoyal]

Yes, you got that right. We should work on defining better nomination rules as compared to going for in-group voting.

[markwhiting]

@shirishgoyal, do you have a concrete suggestion about what you're thinking about as a better nomination rule?

Also, do you think that restricting nominations would increase or decrease the level of unconscious bias governing group membership? (I'm really not sure, but my hunch is that open nominations with some kind of improved voting might do a better job of that than closed nominations. Again, just a hunch, but I think its something worth considering in how we specify this kind of system)

[mbernst]

OK, got it. Here was my reasoning:

Suppose that we restrict nominations, for example to existing group members. Then, there will be a bias in the people who get nominated: people who look and think like existing members. This ingroup bias would be insidious in a closed nomination process because more diverse people wouldn't be nominated. It's done in the name of quality control, but there is evidence this practice really hurt major organizations like Wikipedia. I worry about this.

In contrast, if nominations are open but votes are closed, then the existing members of the group are forced to make public statements about anyone nominated. My hunch is that we would get more prosocial outcomes if nominations were open to anyone, and members had to vote publicly on these cases.

[shirishgoyal]

Strong nomination rules is not restricted by people who can nominate but who can be nominated based on contribution.

[mbernst]

Following discussion in a hangout, the refined proposal: let each operational group and quality review group define publicly-visible and measurable criteria before an individual can be nominated, then keep nomination and voting public. For example, for code review, criteria might be something like having N pull requests reviewed and merged, as well as having been active for a certain period of time.

[shirishgoyal]

I think before new operational groups or quality review groups are setup, we as a collective should come up with nomination criterion which is measurable and openly stated in the governance docs. Ofcourse these criterion can be updated as we move along if collective feels the need of it.

Since quality review groups grant more of assessment privileges, they should be based on contributions of similar nature of the work being assessed -- for code review, number of commits made or number of features pushed as compared to reviews done as that privilege doesn't even exist with the contributor till then. For UI/UX, it can be the number of evaluations or recommendations made that are deployed in Daemo based on proposals accepted.

For operational groups, people who are willing to overtake that responsibility can be inducted and then reviewed for their progress based on certain criterion to maintain their status in that group.

I think similar to the #21, if we are able to come up with a scale of active time period, it sorts out credit problems as well automatically with minimal effort.

[mbernst]

Closing this proposal: we are agreed that the approach moving forward will be through authoring specific and separate criteria required before nomination for each group.