SLEP019: Governance Update - Recognizing Contributions Beyond Code

jjerphan commented 2 years ago

Description

As discussed on the mailing list, this SLEP proposes updating the Governance to broaden the notion of contribution in scikit-learn.

The HTML output of the SLEP is rendered here: https://scikit-learn-enhancement-proposals--74.org.readthedocs.build/en/74/slep019/proposal.html

Associated implementation PRs: https://github.com/scikit-learn/scikit-learn/pull/24444

🗳️ The vote on this SLEP ended: the SLEP is not accepted as is.

The vote for SLEP019 has been cast on the internal mailing list on Friday, 14 October 2022 around 16:00:00 UTC+0

As specified in SLEP000:

The vote has ended in a month from the start, i.e. Friday, 11 November 2022 16:00 UTC+0.
the SLEP is not accepted as is as more than ⅔ of the votes are negative votes.

adrinjalali commented 2 years ago

Note that our governance is not very clear on whether we need a SLEP for changes to the governance itself, it only says the same decision making process is required.

For changes to the governance doc in the past, we haven't done a SLEP, we have only called a vote on the proposed changes.

jjerphan commented 2 years ago

I understand the Decision Making Process as indicating coming up with a SLEP and a vote on the internal mailing list for this SLEP prior to Governance changes:

Scikit-learn uses a “consensus seeking” process for making decisions. The group tries to find a resolution that has no open objections among core developers. At any point during the discussion, any core-developer can call for a vote, which will conclude one month from the call for the vote. Any vote must be backed by a SLEP. If no option can gather two thirds of the votes cast, the decision is escalated to the TC, which in turn will use consensus seeking with the fallback option of a simple majority vote if no consensus can be found within a month. This is what we hereafter may refer to as “the decision making process”.

Decisions (in addition to adding core developers and TC membership as above) are made according to the following rules:

[…]

Changes to the governance model use the same decision process outlined above.

reshamas commented 2 years ago

some notes:

adding some references: a) Amanda Casari on ACROSS and Measuring Contributions in OSS https://podcast.sustainoss.org/111 43-minutes audio

b) NumPy Newcomer's Hour: an Experiment on Community Building: Melissa Weber Mendonça https://youtu.be/c0XZQbu0xnw 10-minute video

lucyleeow commented 2 years ago

a) Amanda Casari on ACROSS and Measuring Contributions in OSS

I think she's also given a talk about contribution metrics (probably similar content to the podcast): https://www.youtube.com/watch?v=yrBqXfGzx-U

reshamas commented 1 year ago

Something to consider, as I sit in on a session on language translation at this meeting. And our recent attention and evaluation to the various types of (in/visible) contributions to the project.

To consider that Mariangela Petrizzo's initiative of translation of the scikit-learn docs is impactful. And to consider that type of contribution to the project...... that we would consider nominating her to the team. https://github.com/scikit-learn/scikit-learn/pull/20763

Reshama

On Tue, Sep 20, 2022 at 11:06 PM Lucy Liu @.***> wrote:

a) Amanda Casari on ACROSS and Measuring Contributions in OSS

I think she's also given a talk about contribution metrics (probably similar content to the podcast): https://www.youtube.com/watch?v=yrBqXfGzx-U

— Reply to this email directly, view it on GitHub https://github.com/scikit-learn/enhancement_proposals/pull/74#issuecomment-1253148984, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATEDYBKTLZ3IW4N6BJXEQDV7J3TVANCNFSM6AAAAAAQLOMBV4 . You are receiving this because you commented.Message ID: @.***>

jjerphan commented 1 year ago

@reshamas: please, let me know if the resources you suggested have been properly added and cited in 8ca1922d2acf3b361dbcbcb294c0a6a9fb253282.

As for translations of the documentation, I think we also need to recognize those contributions. Yet, I think this is a currently a bit orthogonal to the goal of this PR and I think recognising people contributions might better be done after we settle on the Governance update. What do you think?

@lucyleeow: thank you for suggesting this resource. The content covered is a bit different: it emphasis defining the community goals then coming up setting metrics accordingly. As this SLEP aims at recognising all contributions and team and mostly is conceptual, I think it might be preferable to include and use this resource at a latter stage, e.g. for the implementation of this SLEP. What do you think?

In both cases, I am glad people give their opinion and discuss the community structure and dynamic. This show that we probably have not been having room, nor time, nor place set to discuss those topics.

lucyleeow commented 1 year ago

@lucyleeow: thank you for suggesting this resource. The content covered is a bit different: it emphasis defining the community goals then coming up setting metrics accordingly. As this SLEP aims at recognising all contributions and team and mostly is conceptual, I think it might be preferable to include and use this resource at a latter stage, e.g. for the implementation of this SLEP. What do you think?

No problem, that makes sense - sorry it has been a while since I watched it and forgot!

jjerphan commented 1 year ago

No problem, that makes sense - sorry it has been a while since I watched it and forgot!

It's fine. With the lack of place for people to spontaneously discuss, I personally would rather have spontaneous suggestions and interjections in discussion on GitHub than people keep their views and ideas for themselves.

The only place scikit-learn has for people to discuss is the monthly "Core-Devs' meeting". Other projects like SciPy and NumPy have "Community Meeting".

I think this situation motives this SLEPs, as in the case we also probably need to change this semantic and come up with another organisation (e.g. have a general "Community Meeting" and than more scoped/focused "Teams' meetings").

lucyleeow commented 1 year ago

Other projects like SciPy and NumPy have "Community Meeting".

I think the community meetings are new and the result of work funded by: https://chanzuckerberg.com/eoss/proposals/advancing-an-inclusive-culture-in-the-scientific-python-ecosystem/ The work has included many things from improving contributing documentation, creating policies around inactive PRs, monitoring first time contributor PRs in numpy, standardising issue templates etc. I think there is collaboration with Scientific Python in the form of guides for maintainers, in order to share from their experience. This seems to be a WIP but they do have a guide on meetings. The work is being done by @noatamir @melissawm and @InessaPawson (sorry for noise, wanted to highlight your great work and you may be interested in this!)

adrinjalali commented 1 year ago

Thinking about the "Recurring Contributor", and I realized there are alternative models to what we're thinking here as well.

I also do agree with @thomasjpfan that a few of the people whom we would put in the recurring contributing team would be a good fit for the contributor experience team.

But as an alternative model, we can think of something more along the lines of a "scikit-learn fellows" which can be a team of people who keep a presence around the repository in different ways. We can think of people also nominating themselves or applying to be a member of that team, and instead of accept / reject, we can think of a path which they can take to be in that team. For instance, if we think they'd need more contributions, we can tell them what they can try to achieve first, and give them a few tasks / PRs / issues to handle. And as we talked in the last meeting, we can also require them to keep a presence of some sort to stay in that team. And if we clearly define what it means to be in that team, then it'd be a meaningful thing to say in someone's CV that I've been or I am in the scikit-learn fellows team. WDYT?

jjerphan commented 1 year ago

Thank you all for participating. Here is a summary of the last discussions and changes for this SLEP.

Supposedly consensual propositions

No-one discussed the following points:

Define "Contributions" explicitly
Define the notion of "Core Contributors" and the associated teams:
- Core Triagers Team
- Communication Team
- Core Developers Team
Extend voting rights to all Core Contributors

Hence, if no-one disagrees, those points can be considered as consensual.

:warning: Please, do object if you think those points are not consensual or if you further want to discuss them.

On-going discussions

### Adding references Several references have been added. 46044845f41f11f348fdb92df57919521769adc9 replaced a reference with an original resource material from the same author (see @reshamas, @cmarmo and @jjerphan in this thread https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r980296882). ### Regarding the Core Triagers Team The Core Triagers Team was initially proposed to have the "Write" permissions without misusing them. After inspection by @thomasjpfan, granting "Write" permissions to the Core Triagers Teams seems relevant and secure (@thomasjpfan in https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r990824122 and https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r990423243). Hence, giving "Write" permissions to the Core Triagers Team was introduced by 3e58334f109a2991a168dd456fb4ec7a9f914f4f. cc @cmarmo ### Regarding the Experts Team Experts are people whose reviews would be delegated to and trusted by Core-Developers. Yet, a few people think that having an Experts Team would just add complexity to the organisation (@lucyleeow, @adrinjalali, @jnothman, @Micky774 and @jjerphan in this https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r978194747). - Why would we need a Experts Team for? - Could and should operate with Experts without a Team, on per contribution basis? ### Regarding the Recurring Contributors Team It was originally stated that: > A Contributor is added to the "Recurring Contributor" team after championing by three Core contributors (notion of "Core contributors" as defined below). This part was considered loose, especially w.r.t the Experts Team whose members' promotion process was not specified. Hence this afore-cited part was removed in 3e58334f109a2991a168dd456fb4ec7a9f914f4f (@jjerphan in https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r991162333). Moreover, do we want the Recurring Contributors Team to grant responsibilities to people? If not, people who would like to get involved could simply join the Contributor Experience Team (@adrinjalali in https://github.com/scikit-learn/enhancement_proposals/pull/74#issuecomment-1273239135 and @cmarmo, @thomasjpfan and @jjerphan in this thread https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r990441515).

Proposal of actions

To meet the original goals and needs of the organisation stated in the SLEP, I am proposing:

to restrict the scope of this SLEP by displacing the discussions about the Experts Team and Recurring Contributors Team to another subsequent SLEP (this would mean removing their associated proposals from this SLEP).
to approve the changes regarding the Core Triagers Team
to approve the supposedly consensual propositions
to cast a vote for this SLEP
if this SLEP gets approved, to implement relevant changes. This includes:
- updating scikit-learn documentation (trimming and rewording https://github.com/scikit-learn/scikit-learn/pull/24444 based what has been agreed upon)
- creating Teams with on GitHub
- adapting Teams' permissions on GitHub

:warning: Please do :+1:-react to this message if you agree with this proposal of actions.

cmarmo commented 1 year ago

to restrict the scope of this SLEP by displacing the discussions about the Experts Team and Recurring Contributors Team to another subsequent SLEP (this would mean removing their associated proposals from this SLEP).

What about defining in this SLEP the process to discuss and add new roles, so a new SLEP will not be necessary in the future: new roles and teams will be subject to a vote and the governance updated accordingly.

Micky774 commented 1 year ago

Regarding the experts, I am +1 on codifying the idea that non-core-developers may contribute to the two approvals requirement. Of course, it still needs to be clarified what the requirements are for a non-core-dev approval to count towards this (i.e. how does one formally become an "expert"?).

If expertise is understood on a person-by-person basis, I think the complexity is not too bad. We should consider formally tracking which experts are available to ping for each code section (e.g. by module, estimator, or file). I'm imagining something like the current available reviewers page so that contributors not familiar with the experts may know who to ping.

Overall, I think:

An experts team makes a lot of sense for scikit-learn
Team membership should only signify that they are a trusted review for some subset of the library
The subset for which an expert is a trusted review should be formally specified (e.g. an "About Experts" page listing each expert and their area of expertise)
Contributors should have some simple reference for knowing which experts may be pinged to review their PR (e.g. available reviewers). Note that the aforementioned "About Experts" page may suffice.
A formal admission mechanism still needs to be specified, however I don't think we need formal criteria (expertise is case-by-case).

jjerphan commented 1 year ago

The vote for SLEP019 has been cast on the internal mailing list on Friday, 14 October 2022 around 16:00:00 UTC+0

As specified in SLEP000:

The vote will be closed in a month, i.e. Friday, 11 November 2022 16:00 UTC+0.
the status of this SLEP was changed from "Draft" to "Accepted" (by f103cb2025dcccb71baf7e154b8d734710df9a67)

To accept this SLEP, please do approve this PR.

adrinjalali commented 1 year ago

Note that it's handy to merge the PR introducing the SLEP so that it's rendered on the website and people can read there, and have a separate PR changing the status to Accepted where the actual vote happens. This is also how we've done it in the past.

jjerphan commented 1 year ago

Thanks for indicating this, @adrinjalali. I was not entirely sure on how to proceed.

In the end, I chose to have it in a single PR as the SLEP is rendered online and as its prose still need to be reworded (as proposed by https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r995974450 for instance).

reshamas commented 1 year ago

@jjerphan I can't find the text on Recurring Contributors. Can you link to it here?

reshamas commented 1 year ago

Other projects like SciPy and NumPy have "Community Meeting".

I think the community meetings are new and the result of work funded by: https://chanzuckerberg.com/eoss/proposals/advancing-an-inclusive-culture-in-the-scientific-python-ecosystem/

As an FYI: PyMC has Community Meetings as well (though I do not believe they have dedicated funding for this.) They like to do an annual conference and so created a Community Team to work on this. This team also covers sprints, events, answering questions on their Discourse, etc.

cmarmo commented 1 year ago

Two quick comments, if I may:

I don't like the "Core Triaging" wording. Just "Triaging" is enough to specify the role, no hierarchy is introduced between the triagers here, the SLEP aims only to guarantee that "contributors involved on GitHub / in triage and that aren't/don't want to be involved in maintenance have the proper permissions." (see comment)
without the introduction of a "Recurrent Contributors" (or whatever name you want to call it) group the SLEP does not solve the issue of smoother onboarding, which is a big issue in scikit-learn. In some months, with the acceptance of this SLEP there will be more roles in the project without people to fill them.

jjerphan commented 1 year ago

Two quick comments, if I may:

I don't like the "Core Triaging" wording. Just "Triaging" is enough to specify the role, no hierarchy is introduced between the triagers here, the SLEP aims only to guarantee that "contributors involved on GitHub / in triage and that aren't/don't want to be involved in maintenance have the proper permissions." (see comment)

Based on this comment, I have proposed to have this renaming in https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r997791926.

without the introduction of a "Recurrent Contributors" (or whatever name you want to call it) group the SLEP does not solve the issue of smoother onboarding, which is a big issue in scikit-learn. In some months, with the acceptance of this SLEP there will be more roles in the project without people to fill them.

That's right, but I think as far as we have not defined which roles scikit-learn we need, it might be rather hard to introduce "Recurrent Contributors". 😕

If everyone agrees on having changes to the Governance be made without SLEPs (which you propose and which was integrated in the this SLEP), I think we can define what we referred to as "Recurrent Contributors" then.

Do you think this is appropriate as an approach? Or do you think we must define further roles' or teams' in this SLEP?

jjerphan commented 1 year ago

@jjerphan I can't find the text on Recurring Contributors. Can you link to it here?

Most of the proposals for defining Recurring Contributors (as well as Contributions and Core Contributors) can be found on this DOC PR: https://github.com/scikit-learn/scikit-learn/pull/24444 (also linked in this PR description).

cmarmo commented 1 year ago

Based on this comment, I have proposed to have this renaming in https://github.com/scikit-learn/enhancement_proposals/pull/74#discussion_r997791926.

Thanks

In some months, with the acceptance of this SLEP there will be more roles in the project without people to fill them.

That's right, but I think as far as we have not defined which roles scikit-learn we need, it might be rather hard to introduce "Recurrent Contributors". 😕

If everyone agrees on having changes to the Governance be made without SLEPs (which you propose and which was integrated in the this SLEP), I think we can define what we referred to as "Recurrent Contributors" then.

Do you think this is appropriate as an approach? Or do you think we must define further roles' or teams' in this SLEP?

I guess you noticed already where my heart stands... 😁 I only partially agree with this approach, but it seems the only way to move forward... and also my vote doesn't count anyway... 😜 As long as friction can be reduced in the future, no need to add more details here.

noatamir commented 1 year ago

👋 I was tagged earlier and have been following the discussion. I had another suggestion for your consideration. CC @adrinjalali

How about renaming the Technical Committee (TC) to Steering Committee or Steering Council (SC)? My thinking is that they are sometimes called on to make decisions which aren't technical in nature, but related to governance or other project matters, e.g., if the vote on this SLEP wouldn't achieve consensus.

In addition, considering the changes you are proposing to the core members, would you agree to make it possible to nominate any Core Contributor to the SC as well? This would untangle it from technical requirements, or commit rights.

jjerphan commented 1 year ago

I integrated @noatamir's proposal (i.e. https://github.com/scikit-learn/enhancement_proposals/pull/74#issuecomment-1295078486) in 0ab025476bb78970ec9830b592ae9356c8017989.

@noatamir: please let me know if what I worded correctly phrases what you proposed or if it needs to be adapted.

noatamir commented 1 year ago

Thanks @jjerphan! I wonder if the last part is redundant? The text in Technical Committee doesn't actually specify that the committee will only make technical decisions. I believe that renaming TC->SC and core developers -> core contributors is sufficient in that paragraph. WDYT

jjerphan commented 1 year ago

You are right. The redundancy has been removed by https://github.com/scikit-learn/enhancement_proposals/commit/249b9f469d3ac21000b410769caaa5220d28e163.

cmarmo commented 1 year ago

There are two possible options:

1. A smaller SLEP that focuses on "Simplify subsequent changes to the Governance"

2. This document as is, which includes the Triage Team, renaming of the technical committee, extending voting rights and simplify governance  changes.

I would like to suggest a third path. The point is "Recognizing contributions beyond code" and the SLEP should be actionable in some way. The SLEP may define

The general recognition of new roles in contributing
The acceptance of the future creation of new roles as core contributors, for which no new SLEP will be necessary (no details about the roles for now)
The transformation of the Technical Committee in Steering Committee: the Steering Committee is composed of the current members of the Technical Committee, with the perspective of being open to members of the other core contributors groups. This will probably need more details about nomination or election to the SC with a specific time of service, other open source projects have already this kind of structure (well ... python... :) ), it should be a good idea to take inspiration from them.

Could this be an option?

jjerphan commented 1 year ago

I agree with what @cmarmo is proposing but I see no difference nor extra proposals with the 2. item @thomasjpfan has mentioned i.e. the current SLEP's scope with an explicit (long) exact wording; is this the case, @cmarmo?

We need to have actionable while not loosing everyone's time and energy: to me 1. is better for long term while 2. is better for what we aimed at initially but complex to be agreed upon in the current state.

Proposal for action: Assuming people who want to participate in those decisions are reactive and proactive enough in the phrasing of terms[^1], we can get 1. done so as to ease the implementation of 2. using independent, atomic and small modifications for each item which would be discussed and agreed upon in individual threads' (e.g. PRs).

What do you think?

Please do :+1:-react to this message if you want to proceed with this proposal for action. Please do :-1:-react to this message if you do not want to proceed with this proposal for action.

Please do suggest an alternative pathway if you have one.

[^1]: This is of personal preference.

adrinjalali commented 1 year ago

@jjerphan I can't really understand your comment, but I'm happy with what @cmarmo suggested.

cmarmo commented 1 year ago

I agree with what @cmarmo is proposing but I see no difference nor extra proposals with the 2. item @thomasjpfan has mentioned i.e. the current SLEP's scope with an explicit (long) exact wording; is this the case, @cmarmo?

Nope. I'm removing all the definitions of new roles from the SLEP. Accepting the definition of new roles outside SLEP proposals makes unnecessary defining them here. Enlarging the scope of the Technical Committee, which becomes a Steering Committee, opens the decision process to new roles already.

thomasjpfan commented 1 year ago

I'm happy with @cmarmo suggestion as well.

cmarmo commented 1 year ago

Dear all, two of the three negative votes seem to overall agree with my suggestion. Before starting to work on the text, may I ask @lorentzenchr to check this third option and let me know if this is acceptable? Also, as @GaelVaroquaux is listed as author of this SLEP, I would be happy to have his opinion about this suggestion. Thanks!

jjerphan commented 1 year ago

The vote period has ended about 2 hours ago.

More than ⅔ of the votes are negative votes so the SLEP is not accepted, at least in its current status.

lorentzenchr commented 1 year ago

@cmarmo the 3rd option sounds good. IIUC, it will make the text smaller and more precise.

But again: The induced discussion alone was worth this vote.

jjerphan commented 1 year ago

What do people think of merging this PR as a "Draft" and iterating to rephrase it in another PR?

adrinjalali commented 1 year ago

What do people think of merging this PR as a "Draft" and iterating to rephrase it in another PR?

We usually merge before the vote anyway ;)

jjerphan commented 1 year ago

I do not merge PRs I authored, hence I am looking forward to someone that would merge this first draft. :pray:

adrinjalali commented 1 year ago

You need to change the status to draft in this PR, then I'm happy to merge @jjerphan :)

adrinjalali commented 1 year ago

@jjerphan could you please add it also to index.rst under "Under Review" maybe?

jjerphan commented 1 year ago

https://github.com/scikit-learn/enhancement_proposals/pull/81 opened by @cmarmo follows-up with this first draft.

scikit-learn / enhancement_proposals