Determining AC Consensus of Post-Review Changes

[fantasai]

@tantek, @frivoal, and I discussed some of the ambiguities in the Process around what happens when we follow the "adopt with substantive changes" track in https://www.w3.org/Consortium/Process/Drafts/#ACReviewAfter and how to fix them. We think three changes are necessary (two to Process, one to Guide):

1. Objections that block consensus of substantive changes during the consensus check, like those during the AC Review itself, get registered as Formal Objections.
2. These Formal Objections get batch processed with the FOs from the AC Review.
3. The consensus check on the revised document should use the same question structure as the AC Review itself, to ensure clarity on the responses; and ideally the same system (currently WBS).

Note that prior to Process 2023, consensus was defined as the lack of a Formal Objection. We revised this definition for various reasons in Process 2023, and I don't think we should revert; but the ability to block consensus without actually registering an FO leaves these cases in a weird limbo and makes it hard for a Council to properly address the W3C Decision at hand.

[plehegar]

Fyi, I'm experimenting with implementing how we assess the consensus check. See REVIEW by 2024-03-19/20: Proposed changes to the Federated Identity Working Group charter (Member-only link).

The WBS survey is available to all of the AC Reps, but only the AC Reps who responded to the original WBS survey were notified. No notification was sent to the public. Since the original comments were all supportive of the original proposed charter, the deadline for the new WBS survey is only one week. The proposed changes were discussed with the CG by the way since they were the ones who drafted the charter in the first place.

[frivoal]

One thing that I would add is that if the proposed substantive changes would expand the scope, then it need to go back to the whole AC, because some people who didn't cast a ballot might have if the scope had included something they cared about.

[css-meeting-bot]

The Revising W3C Process CG just discussed Clarifying AC consensus wrt substantive changes.

The full IRC log of that discussion

<fantasai> Subtopic: Clarifying AC consensus wrt substantive changes
<fantasai> github: https://github.com/w3c/w3process/issues/825
<fantasai> florian: This is a recently-introduced ambiguity because of how we tighted up some things but not others
<fantasai> ... in the past substantive changes were under Directory authority, and now not
<fantasai> ... there's a rule that you have to get consensus among the reviewers
<fantasai> ... in the meantime, we adjusted the definitions of consensus
<fantasai> ... in e.g. WG if you don't have consensus, you can make decision by vote
<fantasai> ... but problem here is if there's an objection (which blocks consensus) but not an FO, then it can't be resolved because not voting and can't escalate to Council
<fantasai> florian: Proposal is to require any objection blocking consensus of post-AC-Review changes to be an FO, and handled by same Council as the other FOs from that AC Review
<plh> q+
<fantasai> florian: Also to adjust the practice of checking consensus to make it clear
<fantasai> florian: One adjustment is, we should require substantive changes that expand scope, should go back to whole AC
<fantasai> plh: ???WG wanted to add a new deliverable to the charter
<fantasai> ... that would increase the WG scope substantially
<fantasai> ... here we went through full AC Review
<fantasai> ... Another case is WebAppSec charter
<fantasai> ... where the deliverable was listed, but the scope section wasn't compatible, so a reviewer commented that it needs to be adjusted
<fantasai> ... for that second case, would that also be a full AC Review?
<fantasai> florian: yes
<TallTed> Substantive change, whether broadening or narrowing scope or deliverables, should lead to a new review/vote — because initial votes might have been different based on the difference.
<fantasai> florian: if you add a deliverable maybe OK but changing scope, needs to go back to AC
<plh> ack plh
<TallTed> q+
<fantasai> ... reducing scope is ok, increasing needs to go back to AC
<fantasai> plh: what if deliverable was already listed?
<plh> ack fan
<fantasai> florian: that's a weird case
<florian> q+
<plh> ack ta
<fantasai> fantasai: if it'll require legal review, then really should go back to AC review
<fantasai> ... so increasing scope needs new review
<fantasai> TallTed: Whether removing or adding deliverable or removing or adding scope, anyone's vote could change
<plh> ack flo
<fantasai> fantasai: if you make a substantive change, then have to check with everyone who voted (including abstentions)
<fantasai> ... question here is whether to go back to the entire AC for a change
<fantasai> florian: OK, we're at time

[TallTed]

if the proposed substantive changes would expand the scope, then it need to go back to the whole AC

I think that return to entire AC should apply to both expansion and contraction, of both scope and deliverables.

It's just as possible that some element that's being removed would be considered a vital ingredient, as that they would consider some element to be too much reach, such that the revision — whether broadening or narrowing — would have had them vote differently (including whether they abstain or not).

[plehegar]

I concur that changes in scope should trigger new AC reviews but I'm worried that limiting the set of substantive changes that can be done on charters will add too many tape layers. The recent update of the FedID charter to update the out-of-scope section shouldn't trigger an entire new AC review imho. While adding Digital Credentials to it was certainly a major change that ought to (and will) trigger a full separate AC review. The case of the WebAppSec charter is a bit murkier since the deliverable was part of the charter submitted to the AC for review but it was noted that the scope section needed to include it as well.

One alternative here would be not to limit the review of the proposed changes to the set of the AC who provided feedback in the first review. Similar to our technical reports tracks, we should be able to accept comments on charters from anyone at any time. After that, it's a question of how long we should give the AC for its re-review and I would advocate that it should be context dependent.

[dwsinger]

Perhaps the team should make the considered call on whether a new AC Review is needed, and make a 'decision' that could be objected to?

[TallTed]

Perhaps the team should make the considered call on whether a new AC Review is needed, and make a 'decision' that could be objected to?

Presuming the Team would notify the full AC of this Decision with appropriate window for FO, that seems reasonable.

Still, I do wonder whether that would actually save AC time, vs notifying them of the proposed Charter DIFF, with a timeboxed link that would preload their previous vote&comment so they could quickly click to apply the same vote&comment to the revised Charter or almost as quickly revise and submit their previous vote&comment on the revision...

[dwsinger]

Yes, I am envisaging that the same email that documents what happened after the AC vote would do that. Something like "The AC vote recorded 7 objections and 3 formal objections, 28 yes and 4 abstains. Two of the FOs were withdrawn after a change was made to the charter; one was ruled against by Council. The objections and the two FOs resulted in changes to the charter, and we are now seeking consensus support from those who 42 who voted. The Team does not believe that these changes are sufficiently substantive to warrant a second full vote at the AC; however, this decision can be objected to."

[css-meeting-bot]

The Revising W3C Process CG just discussed Tighten the how subtantive changes are handled post AC-Review, and agreed to the following:

• RESOLVED: Accept PR with only in "may only" moved to "only if".

The full IRC log of that discussion

<fantasai> Subtopic: Tighten the how subtantive changes are handled post AC-Review
<fantasai> github: https://github.com/w3c/process/issues/825
<fantasai> PR: https://github.com/w3c/process/pull/910
<fantasai> florian: Practice isn't too far from this
<TallTed> q+
<fantasai> ... areas of difference are that if you increase the scope, you must go to AC
<fantasai> ... also bring clarity about when people disagree with changes that are proposed after AC Review intended to resolve an objection
<fantasai> ... most people happy with those changes, a few are not, then it's a weird situation
<fantasai> ... people who disagree with the proposed changes can disagree without making an FO, and then what?
<fantasai> ... what does that mean?
<fantasai> ... can we overrule their non-formal objection?
<fantasai> ... makes it clear that if you disagree, it counts as an FO, and existing Council gets to rule on it
<plh> ack TallTed
<plh> ack plh
<fantasai> TallTed: the "may only" phrasing, is usually restrictive
<fantasai> ... to entirely clear if that's intended
<fantasai> florian: I intended a restriction. If you don't have consensus, you cannot adopt.
<fantasai> TallTed: then suggest "only if" instead of "may only"
<fantasai> florian: sure
<fantasai> plh: wfm
<fantasai> fantasai: wfm
<cwilso> q+
<plh> ack cwilso
<fantasai> [haggling over wording]
<fantasai> fantasai: For the second one, agree that it reads "better" in an absolute sence with it moved, but the reason it's pulled forward is to emphasize that phrase since the sentence is about this particular timing of the FO.
<fantasai> RESOLVED: Accept PR with only in "may only" moved to "only if".