Governance: decision making on technical topics

[datapythonista]

Parent issue: #47694

Scope of the discussion here: How technical decisions about the pandas project are made

Other governance topics like decisions about funding, creation of pandas workgroups, who is an active core dev... should better be discussed separately, to keep things focused, even if they may affect technical decision making, and are currently not well defined.

Example of technical decision:

• What build system to use, setuptools, Meson, cmake...
• Should the implace argument be fully deprecated?
• Would it be better to move read_sas to a third-party project, and remove it from the core pandas?
• ...

Most often decision making in pandas worked well and fluently, with one person leading an effort, other people interested in the topic provides feedback, and we move forward with a solution that everybody seems happy with. Personally, I think this is just fine, and it may not make sense to overthink or add too much overheat to these cases.

But in some cases, we end up with different people having strong conflicting opinions on how to move forward with a topic. In my experience, this can end up being tiring and frustrating for all involved people, and the final outcome is either stay with the status quo for lack of consensus, or taking the decision of the person who has more energy and perseverance to invest in the discussion. From our current governance, we have Wes as the project BDFL who could mediate and make a decision, but in practice this doesn't happen as Wes is not active in pandas anymore.

With the introduction of PDEPs, these technical decisions are probably going to be required more often, with possibly more non-trivial decisions to make, and it'd be good to formalise the process on how decisions are made, as well as to clarify when a decision is assumed to be made.

I personally don't have a strong opinion on what governance model should be implemented. numpy's governance seems quite reasonable, and not far what we've been informally doing. I guess it can be a good starting point. Based on the experience on past pandas discussions I'd personally prefer to have some less ambiguity. For example, specify how much time is required since a proposal it's made, until it's considered that there are no vetos.

I don't want to bias the discussion much here, and please feel free to incorporate here any relevant topic not added by myself, but some of the questions that it may make sense to answer are:

• Do we want to keep the pandas BDFL role? If so, does it make sense that someone active in the project becomes it?
• Is a steering committee something useful to pandas? What the exact responsibilities of the committee would be?
• Are we happy with a veto model similar to numpy's? Should a single veto be enough, or maybe a proposal should move forward if it has enough endorsements and just a single veto?
• Would voting for the proposals instead of the veto model be more convenient?
• Do we want to have a minimum amount of core devs endorsing a proposal before it's considered accepted? If so, how many?
• For PDEPs, does PRs "Approve" and "Request changes" seem convenient ways to manage endorsements and vetos for a proposal?

Opening this as an issue, but maybe worth having a dedicated call to discuss this can also be useful. Let me know if there is interest.

CC: @pandas-dev/pandas-core

[mroeschke]

Just clarifying/seeing what issues this would solve:

1. Making the decision making process more explicit.
2. Avoid fading interest & lack on clarity on efforts due to unclear consensus on support.

One question I would be interested to see answered is when decisions are made e.g. "call for a vote". Related to 2) above, it seems like a lot of topics discussed on Github/mailing lists can have very thorough & active discussions during a period of time and then lose steam because the discussion stalled or there is lack of clarity on how to move forward.

+1 to have a dedicated meeting to discuss this topic.

[jorisvandenbossche]

I also think that a synchronous meeting to discuss this would be good to get this started (and to see how we further want to approach moving this forward).

---

Some random thoughts (not concrete proposals, but rather aspects that might be relevant in this discussion or that I find important):

• An important goal for this topic is indeed making the decision process more explicit (to make power structures visible and explicit, instead of having apparently almost no structure, cfr "tyranny of structurelessness"). This is also important for new contributors (so they know how a decision gets made, how a PR can be merged, etc)
• It might be good that we have a decision process that works both for "easier" or smaller decisions as for more controversial ones (so that we can kind of practice it and get used to it with the easier ones)
• Something that I would also hope to get from this is a way to discuss and agree as a project on a direction or vision for the project, like a high level roadmap (maybe discussing this doesn't need a governance, but deciding on outcomes of such a discussion does)
• A way to empower people to take initiatives, or feel ownership for certain tasks or projects
• Personally I think we should try to avoid that the core governance structure is focusing on the technical aspects, since the pandas community is much more than that. Of course, this issue might be focusing on that aspect, and for example the other issue about creating teams (#48462) is also addressing this. But just to mention this aspect, and we can discuss how we embed technical decision making in the general governance. Related to this is also the discussion about "membership" (#47706), which we should also broaden to not just be about commits to main pandas repo.

[datapythonista]

Thanks @jorisvandenbossche for the comments, I agree with all your points. And indeed, opening the issue for technical decisions only is just a way to keep the discussion focussed and address one topic at a time. Not sure if decisions for example about funding, or new maintainers should use the same procedure or not. But I guess it can make things easier postponing those discussions until later.

I added a point to today's agenda to see if people are happy to move forward with this. If there is interest, we can schedule the dedicated meeting for the governance.