2025 proposal selection process

[nairnandu]

Based on the retrospective and feedback from proposal authors, this is a draft of the 2025 proposal review process to be reviewed by the team

[jgraham]

Based on the feedback from last year, going forward I think we'll have more success if we're more successful at developing a shared understanding of value proposition of different proposals, and where we have room for discussion. Of course that won't always work, but I'd like the process to focus on finding those points of agreement and ensuring that as far as possible we end up with a shared consensus.

With that in mind, here's a different take on what the process could be:

1. After the proposal deadline each participating org shares a list of focus areas they'd like to champion. Those can correspond to one or more proposals. Where different orgs have similar focus area proposals they can work together out of band to develop one proposal, but in each case there should be one primary champion for later rounds. It's also OK to drop proposals at any stage if no one wants to champion it further.
2. Champions work on their focus areas, gathering evidence that the focus area represents an Interop priority (e.g. bug reports, developer feedback, positive platform impact on areas like a11y; we can discuss what people would like to see in advance). They are also responsible for ensuring the focus area has an identified set of tests. We do this in a shared space so that non-champions can provide feedback if desired (e.g. point to relevant standards-positions).
3. We have a fixed amount of f2f time (e.g. 15 minutes per participant over two meetings) to formally present the focus area proposals and explain why they're considered a priority for Interop.
4. We have some time for further async adjustments e.g. if there's feedback like "we like X, but can't support it with subfeature Y, please remove that".
5. Each participant now buckets the final proposals as P1, P2, P3 or "veto".
6. A successful outcome at this point looks like lots of proposals where orgs have given similar rankings, and few vetos. We sort by the buckets and pick focus area proposals with lots of P1/P2 rankings.

Some notes:

1. There is no formal elimination stage. It's up to participants to not champion proposals that don't meet the requirements to be accepted.
2. There's no limit on the number of proposals you can champion. It's up to individual orgs whether they'd prefer to put more time into fewer proposals or less time into more proposals. But at the presentation stage you get a fixed time to explain to others why you consider each proposal an Interop priority. This reflects the fact that time is the actual limited resource (both for decision making and implementation).

[dandclark]

1741ff1b1c5c3c78b02a8a2da3284336b1e576aa removed all the language about which parts of the process are confidential and which are not. I think it's still important that this all be defined beforehand.

Microsoft's position is still that the Interop process should follow the model of web standards development by doing things in the open such that positions and discussions are public. But at the very least, organizations should be able to share which proposals they are championing and why, since revealing that information won't give full visibility into how the final votes and vetoes stack up.

[foolip]

I agree it's important that any confidential parts are explicitly marked. Per our charter we operate in public except where a process document like this one says otherwise.

Google's preference is to operate in public, including our all or positions on specific proposals. If there is some confidentiality in the process, we think it's important to at the very least clarify that it only applies to sharing the positions of other organizations who want them to be confidential. This wording from the original revision of this PR would do the job:

However, organizations can choose to publish their own priorities, for the Interop proposals, outside of the Interop program.

[jgraham]

To be clear there's no intended change to the confidentiality compared to last year i.e. the details around proposal ranking and selection remain confidential to participants.

Of course there's not, and never has been, any restriction on commenting positively or negatively about specific web technologies in general.

I need to update the PR to make that clear.

However, the possibility of specifically allowing people to talk about the proposals they are championing is something I'd also considered, and would be worth discussing as a group.

[dandclark]

Of course there's not, and never has been, any restriction on commenting positively or negatively about specific web technologies in general.

I need to update the PR to make that clear.

However, the possibility of specifically allowing people to talk about the proposals they are championing is something I'd also considered, and would be worth discussing as a group.

Last meeting I believe we were discussing these points but ran out of time. To help focus the discussion for next meeting I've pushed a suggestion for language that further clarifies the first point and opens up sharing of championed proposals. Let's dig into this more Thursday.

[foolip]

Thank you @dandclark! I've included this in the agenda for tomorrow, https://github.com/web-platform-tests/interop/issues/674.

The wording you've suggested looks good to me, and is in line with Google's position/preference in https://github.com/web-platform-tests/interop/pull/657#issuecomment-2194230551.