The appeals process doesn't actually describe how an appeal finishes

[dwsinger]

Section 7.2 ends by saying "If, within one week of the Team's announcement, 5% or more of the Advisory Committee support the appeal request, the Team must organize an appeal vote asking the Advisory Committee to approve or reject the decision."

What happens next? We get a formal vote to overturn the decision. But is there any requirement on how long the vote is open? What is the threshold (simple majority?)? What exactly is the question? (I.e. is it "Should the Chocolate IG be formed?" or "Should the Director's Decision to form the Chocolate IG be overturned"?). If the AC votes contrary to the Director, is the decision automatically overturned, or is it sent back to the Director to think harder?

We've not been down this road and hope we never will, but even so, this degree of uncertainty might be worrying.

[jeffjaffe]

It is true that the process document is not overly prescriptive about these questions. It is not obvious to me that we actually need process changes, so I would close this issue unless people felt that it is vital to be more prescriptive.

As a way of testing this, let me address dwsinger's questions above.

How long is the vote open? I believe that should be up to the Team who usually makes these decisions.

Threshold? Simple majority.

What is the question? Should the Director's Decision be overturned? There should be three choices. Yes. No. Abstain. In the calculation of the majority, the Abstains should not be counted.

What happens if the AC votes contrary to the Director? The decision is automatically overturned.

Is it sent back to the Director to think harder? I think the Director has several choices. He could move on and stop working on this decision. He can try to better explain his rationale - make the same decision - possibly be appealed again - and possibly be overruled again. Or he can (e.g.) go back to a WG to rework and improve the spec; then make a new decision and see if that gets appealed as well.

If reviewers agree that the above thoughts are common sense interpretations of the process document I would propose to the Chair that he close the issue. If they disagree, then we may indeed require being more perscriptive.

[wseltzer]

"What is the question?" I posit that it's up to the appellant to propose a question for appeal. Appellants may choose to appeal a part or the whole of a Director's Decision.

[jeffjaffe]

I don't support the level of micromanagement that Wendy proposes. Leaving the appeal question up to the appellant could result in an appellant appealing a particular normative reference or a particular field in a spec. In the US litigious society, attorneys love the flexibility to appeal any part of a verdict. In our collaborative community, after a WG has made progress and the Director has approved it, I believe that the intention of the appeal was an up/down vote on the Director decision. We should preserve that intent.

[michaelchampion]

I'm not sure it should be up to the appellant to pose the question given that we don't want to encourage push-polling, for example "Should W3C cave in to the junk food lobby and create a Chocolate IG". The team might be directed to phrase the question in a neutral way, or perhaps the Process should prescribe that the question be posed as "Do you support the Director's decision to X" yes/no/Abstain."

[dwsinger]

I agree with Mike; the reason I asked is we need to decide whether it's a) Do you support the Director's decision X? b) Do you support (for example) publishing ChocolateObjectModel as a Recommendation? c) Some other question, related to the subject.

(Where the second is the question that the Director was answering).

I think the AC should be asked "do you agree with the Director's decision?"

[wseltzer]

What if someone posing an appeal doesn't like the way the Director has framed a decision, and has a proposal that's more granular than "revert the whole thing"? Your interpretation would seem to leave no way for a member (+5% of membership) to put such a question before the membership.

[dwsinger]

I think that they should communicate to the AC why they think the Director's decision is wrong; request the AC to indicate that they don't support the decision; and when/if the decision is overturned, help frame the debate on what decision they'd like to see made.

[jeffjaffe]

Wendy, we should recognize that appeals are not the right mechanism to discuss items that should have been discussed years earlier. If someone doesn't like W3C, Working Group, or Director framing they should be engaged in discussions all along about how to do it right. If they failed to convince W3C to follow their path - an appeal is an emergency break to stop the train. At that point, it should be Yes/No on the decision.

[jeffjaffe]

Seeing from this thread that there is indeed controversy, I propose a rewrite of this last paragraph of 7.2.

Currently, the text says:

An Advisory Committee representative initiates an appeal by sending a request to the Team. Within one week the Team must announce the appeal process to the Advisory Committee and provide a mechanism for Advisory Committee representatives to respond with a statement of support (yes, no, or abstain) and comments, as desired. The archive of these comments must be Member-visible. If, within one week of the Team's announcement, 5% or more of the Advisory Committee support the appeal request, the Team must organize an appeal vote asking the Advisory Committee to approve or reject the decision.

I propose the following rewrite

An Advisory Committee representative initiates an appeal by sending a request to the Team. The request should say "I appeal this W3C Decision". Within one week the Team must announce the appeal process to the Advisory Committee and provide a mechanism for Advisory Committee representatives to respond with a statement of positive support for this appeal. The archive of these statements must be Member-visible. If, within one week of the Team's announcement, 5% or more of the Advisory Committee support the appeal request, the Team must organize an appeal vote asking the Advisory Committee "Do you support the W3C Decision" - together with links to the decision and the appeal. The ballot allows for three possible responses: "Yes, No, Abstain" together with Comments.

If a majority of those voting Yes or No vote "No", the decision is overturned. In that case, there are the following possible Next Steps:

• The activity concludes
• The Director provides a more compelling rationale for the Decision, makes the Decision again, leaving the possibility for another appeal
• The Director and relevant group modify the proposal based on the appeal and then the Director makes a new decision about the resulting document.

[jeffjaffe]

Ah, one more clarification.

s/The ballot allows for/The ballot is open for a duration to be chosen by the Team and allows for/

[dwsinger]

Looking good.

a) do we need a quorum (e.g. 5%) or was that satisfied in the original request for appeal? b) are we sure we don't need a supermajority to overturn the director? I would have thought so (but we can't do it for any imminent appeal). I would suggest 2/3rds.

[frivoal]

The Director provides a more compelling rationale for the Decision, makes the Decision again, leaving the possibility for another appeal

Infinite loops (or even long loops) are bad. If a majority has voted against the director, having the director say "I'll do it away" doesn't seem great. We could limit this alternative to the first appeal, and if the decision is appealed twice, the director cannot keep on insisting. But even that seems inappropriate. The number of appeals in the history of w3c so far (0?) makes me think this is not a procedure used lightly, and that an appeal would not succeed without serious motivated concerns from the AC. Dismissing them does not seem appropriate, and subsequent appeals may fail not merely because the director may have changed anyone's mind with a better explanation, but also because dismissing the first (or nth) appeal made people give up on the w3c process, which would be tragic.

The Director and relevant group modify the proposal based on the appeal and then the Director makes a new decision about the resulting document.

If the director approved the proposal at the end of an AC review in the first place, it stands to reason that there were also a substantial amount of people in its favor, even if the decision has been successfully appealed. It is doubtful that the initial supporters would be happy about a unilateral change against their preference, even if it is an informed unilateral change. A new negotiation between supporters and detractors seems in order, to be concluded with a new AC review.

---

I would propose that the director at this point should be able to make one of two decisions:

1. The proposal is rejected (i.e. the director thinks it's hopeless)
2. The proposal is returned for additional work, after which a new Call for Review is issued. (i.e. the director thinks consensus on different terms is possible)

[dwsinger]

I tend to agree that the Director going directly against the AC appeal vote is a logical possibility but likely to be a 'nuclear option' (kinda like the Queen refusing royal assent for a bill, it would only happen once, because there wouldn't be a Queen after that). Let's be careful to remember that any decision of the Director is subject to appeal (not just publishing Recs, but charters, and so on), so as much as possible we should keep this section independent of the nature of the decision. (For example, member presents a draft charter and the Director decides not to present it to the AC, and the AC appeals; this is neither document-based nor following a previous AC vote.)

[jeffjaffe]

Florian wrote,

"It is doubtful that the initial supporters would be happy about a unilateral change against their preference, even if it is an informed unilateral change. "

Florian, noone proposed any unilateral changes. My proposal suggested that the Director and the relevant group modify the proposal. In that sense it is identical to your proposal. But I'm happy to adopt your language as that is evidently more clear.

In terms of the option that the Director provide a more compelling rationale, I actually agree with you that this is an unlikely case. I was merely trying to provide flexibility for a case that the Director felt his/her pov was misunderstood. But if the AC would like to prohibit that possibility, I'm OK with dropping it.

[dwsinger]

I would like to consider (perhaps with the rather-thin Brexit margin in mind) whether we should require more than a simple majority to do something as significant as over-ruling the Director. Many bodies require a super-majority for such a step; should we? (E.g. 2/3rds of those voting vote in favor).

Most bodies also have a quorum, i.e. at least X% of the voting body votes in some way (even including an explicit 'abstain'). I am not sure whether the 5% threshold to start an appeal satisfies this common step. Should we require a quorum on the vote?

These are both questions: majority or super-majority? Any number of votes or quorum required?

[michaelchampion]

I agree it should be somewhat difficult to over-rule the Director, but feasible if he/she goes off the rails. A simple majority of a small group of voters should not be sufficient. Maybe: 2/3 majority? Simple majority with a quorum of 20% of the membership?

It might be useful to consider whether members have been active in the Consortium in the past year -- by attending at least one AC meeting, or having voted in some percentage of the AC ballots -- in setting the threshold or tallying votes.. Arguably members who actually participate in the work and follow the governance issues have a more informed opinion on what is at stake in an appeal. If they feel strongly enough to overturn the Director that's one thing; if an equal number of members who only showed up to vote because there was food fight, that's another.

[dwsinger]

Ah, another wrinkle from Michael. We now have three questions: a) Should we require more than a simple majority, i.e. a supermajority, and if so, how big? b) Should we have a quorum for the vote to be valid, e.g. 20% of the qualified electors filed any kind of vote? c) Should we have 'good standing' requirements, to decide what constitutes a qualified elector (e.g. has attended a recent AC meeting, has voted on other recent AC ballots...)?

[jeffjaffe]

I probably need to recuse myself on the discussion of supermajorities, etc. As a W3C Team Member, I might be viewed as biased in favor of the Director.

I would suggest, however, that if one is going to propose any restrictive ballot measures, they need a way to get buy-in from the community. The Process CG is too small to call anything we do here a consensus of the membership. If the membership sees such proposals for the first time when they see a draft of Process2018, I expect that the conversation would be very disruptive.

[dwsinger]

Yes, how to handle this conversation when, in all likelihood, we have had our first appeal, is certainly a tricky question. I suspect that the Process CG can't do much more than make sure that both the proposal and the set of related questions are 'complete'.

[frivoal]

In terms of the option that the Director provide a more compelling rationale, I actually agree with you that this is an unlikely case. I was merely trying to provide flexibility for a case that the Director felt his/her pov was misunderstood.

I am not overly worried that this situation would happen: an appeal is a heavy enough process that there should have been enough time for everybody to make themselves clear. That said, if this is a concern you have, how about giving the director an opportunity to better explain after 5% or more of the Advisory Committee support the appeal request, but before the appeal vote is held?

---

Let's be careful to remember that any decision of the Director is subject to appeal (not just publishing Recs, but charters, and so on).

Clarifying my earlier position: if the decision that gets appealed is one that follows a vote, there should be a new attempt at reaching consensus, followed by a new vote.

If the thing that gets appealed is a decision that did not need a vote in the first place, the answer is less obvious. Maybe letting the director make a new attempt (after discussion with relevant parties) is good enough. Or maybe the fact that it was appealed in the first place means that the AC has now taken over this question, and that it now needs to be resolved via a vote.

Florian, noone proposed any unilateral changes. My proposal suggested that the Director and the relevant group modify the proposal.

The director working with the relevant group to modify the proposal seems like the right way to go (modulo correctly defining "the relevant group"), but following that work, the decision should not be made by the director alone. That's what I called "informed unilateral change". If we're in a successful appeal situation after a vote, then clearly we were in a non-consensual situation and the director's proposed way forward as been rejected. If the conclusion of the new attempt is via a vote, I am hopeful that the discussion will focus on trying to find a consensual position. If the conclusion is a director's decision, I worry that the discussion will focus less on finding a consensus, and more on trying to sway the director.

---

@dwsinger @michaelchampion regarding the question of majority, super-majority, and quorum: I could be ok with some sort of quorum approach (either just a percentage (5% again?), or something involving good standing) to make sure that successful appeals without meaningful involvement of the AC don't happen. The 5% already deals with it to some degree though, so let's make sure we're only putting in safeguards, not extra hurdles.

When it comes to counting good standing, I would not hold the lack of participation to AC meetings against someone as long as they participate in other ways, such as AC reviews. AC meetings are expensive affairs that otherwise engaged AC reps may miss for budgetary reasons.

I am a bit more reluctant to call for a super-majority. If the result of the appeal was to take the opposite decision, then a super-majority would make sense, to avoid brexit-like situations, where it seems like holding the vote 3 weeks later may give the opposite result. However, a successful appeal here leads to a new attempt at finding consensus. The new attempt at consensus does not have to be the opposite outcome form the original one, and I think it is more likely to be somewhat similar but with some key provisions added to make it palatable to a greater number. Even if the AC vote were to fluctuate from 51% to 49%, we'd still be in a situation where we've had significant objections, where the director attempted to propose way forward which was rejected by a large proportion of the AC (both 51% or 49% are a large proportion), and where finding a more consensual outcome would be desirable.

At least that's the case for appeals of decisions that follow an AC vote. When it comes to appeals of decisions that do not follow a vote, it depends how we deal with them, but since there as well the result is a new attempt at consensus, not a 180 degrees turnaround, a simple majority seems sufficient.

---

in all likelihood, we have had our first appeal, is certainly a tricky question

I think we have a double question on our hands:

• What is the implication of the current somewhat vague text in the current process
• What should the next version of the process say

The process is silent on how long the vote is open, but as long as the team does not do anything silly (5 minute voting period, 3 year voting period…) I am not too worried about that. More problematic is the question of what to do if the Advisory Committee does reject the decision. I suggest that making the the consequence of a successful appeal to be a new attempt at consensus building followed by a vote is the least risky path. Detractors of the decision get a chance at reaching for a better compromise, and supporters need not be worried that everything will be thrown out, as mall adjustments to the existing proposal may be enough to turn enough detractors into supporters.

As for quorum or super-majorities, even if we want to introduce them in the next version, I do not think the current text leaves room for them. They may be desirable, but the current text does not mention them, so an appeal ending at 58% reject but thrown out because that's less than 67% would be likely to trigger a new appeal on the process, and I doubt we want to go down that path. And again, if we make the consequence of a successful appeal to be a new attempt at consensus building followed by a vote, I don't think the risk is that bad either way.

[Mitch77]

Whatever innovations in the appeals process this group considers for future appeals shouldn't be applied to the current appeal. To impose either a supermajority requirement or a quorum requirement beyond the 5% threshold on the current appeal would be to move the goalposts, because the existing Section 7.2 doesn't call for them.

If the vote on appeal comes down in favor of rejecting a decision, that rejection needs to be meaningful. That means that if the same issue is taken up again after a rejection, there should be a thorough process and opportunities for input. Simply asking the Director to revise the decision isn't sufficient. Nor should the goal be to flip a few votes by making small changes. Any process after a rejection should be thorough.

[dwsinger]

I don't think that appeals will always result in an attempt to find consensus. Should we publish this Rec.? It's a binary question, yes or no. If the AC overrides the Director, in either direction, what compromise is possible in a binary question? So I think the supermajority question needs addressing.

I completely agree that we should have designed this before an appeal happened, and we can't introduce new concepts for an appeal that is already imminent.

Good standing does worry me. The level of involvement of the vast majority of the AC is depressingly low and often zero. We struggle to get 5% participation on important questions. Few of the AC ever come to meetings. The number of AC who engage in email discussions on the mailing list is tiny. In any contentious vote -- and an appeal is almost by definition contentious -- proponents on both sides are going to see the hundreds of votes from non-involved AC reps as a mine for support, and yet there is reasonable anxiety that they have not followed the history, do not understand the question, and do not participate enough to understand the culture and customs.

[frivoal]

Should we publish this Rec.? It's a binary question, yes or no. If the AC overrides the Director, in either direction, what compromise is possible in a binary question?

I'm seeing this as "publish as is, yes or no", which is sort of binary, but under "no" there is still "publish once condition XYZ is met".

As for the lack of participation in the AC, I both agree it's terrible, and have no clear idea what to do about it. It is not only an issue with appeals, but also with all the votes we hold, where 2% of the AC cast an informed vote, and extra 4% cast a vote to support their friends, and the rest stays silent.

[jeffjaffe]

@frivoal Here is the use case for allowing the Director to try to influence the AC after a rejection and then have a re-vote.

After the Director issues a decision (such as the current case of EME) with a Disposition of Comments, imagine that 5% of the Membership wants to have an appeal vote. In that case, I would expect that the opponents of the Director's decision would lobby heavily to have it rejected, and the Director would not lobby at all - he would let the AC read the DoC and make their own decisions. If a rejection occurs, the Director might then want to communication his case more strongly.

But as I said, I don't feel strongly about this clause if the AC does not generally support it.

[jeffjaffe]

@Mitch77 Mitch, certainly any change in the appeal process would not apply to the current potential appeal request.

We are discussion possible changes to be introduced into Process 2018, which will not be approved without several rounds of AC balloting. The current potential appeal request times out in summer of 2017.

[dwsinger]

In terms of the formal process, I think we should admit the logical possible outcomes, which include the possibility that the AC votes to override and the Director nonetheless reaffirms the decision -- perhaps after dialogue, perhaps after explaining why he thinks he has no choice etc. Processes need to document the possible roads. In some sense, we're here because our predecessors didn't document the end of the appeal road.

[Mitch77]

@dwsinger Assume that the AC "rejects" the decision in the course of an appeal, and the Director "nonetheless reaffirms the decision." The only way that scenario is a "logical possible outcome" is if the appeal process is nothing more than an additional recommendation to the Director - in other words, if the appeals process has no real force or effect. Because there is an appeals process, that process has to allow for meaningful outcomes. Otherwise, why was the process included at all? To change it from a true appeal into a mere request for reconsideration after an appeal has already been announced would be, once again, moving the goalposts.

[dwsinger]

We think we should at least document the termination (see Jeff's Jul 13th proposal), and should have a discussion about super-majority, and other issues, there.

[jeffjaffe]

David assigned me to see where we are with this.

As starters, I think we should look at my 13 July proposal.

Looking at the three bullets at the bottom, Florian has raised a fair issue to delete the middle bullet. The W3C Process CG should debate that and decide what to propose.

For the other bullets, Florian's 15 July post provides better language than the original 13 July proposal.

David asked me to also look at other posts. My assessment is that most other topics (supermajority, qualifications to vote, quorum) could be discussed at the Process CG, but with limited time remaining, no clear consensus, and no clear text proposal that has garnered support - my feeling is that those issues should be pushed to Process 2019.

[michaelchampion]

I agree with @frivoal a successful appeal means at least another AC vote before a revised PR is sent back to the Director.

Let's not try to minimize what over ruling the Director means in W3C, where the Process relies on the Director's expertise and neutrality to resolve disputes. An appeal is a vote of No Confidence, and if successful probably means game over for a spec at W3C.

[paulbrucecotton]

An appeal is a vote of No Confidence, and if successful probably means game over for a spec at W3C. Especially if the PR was published under a liberal copyright license. ??

/paulc

Sent from my Windows 10 phone

From: Michael Championmailto:notifications@github.com Sent: August 22, 2017 15:42 To: w3c/w3processmailto:w3process@noreply.github.com Cc: Subscribedmailto:subscribed@noreply.github.com Subject: Re: [w3c/w3process] The appeals process doesn't actually describe how an appeal finishes (#48)

I agree with @frivoalhttps://github.com/frivoal a successful appeal means at least another AC vote before a revised PR is sent back to the Director.

Let's not try to minimize what over ruling the Director means in W3C, where the Process relies on the Director's expertise and neutrality to resolve disputes. An appeal is a vote of No Confidence, and if successful probably means game over for a spec at W3C.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/w3c/w3process/issues/48#issuecomment-324130997, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH6r-MUhx9ZwMalVQ_Aoqe3kSjQjysGZks5say8rgaJpZM4ORbNM.

[dwsinger]

Putting it all together, and changing the language to match the current appeal (which asks the AC to Approve or Reject the Director's Decision), I think Jeff and Florian are saying:

An Advisory Committee representative initiates an appeal by sending a request to the Team. The request should say "I appeal this Director's Decision". Within one week the Team must announce the appeal process to the Advisory Committee and provide a mechanism for Advisory Committee representatives to respond with a statement of positive support for this appeal. The archive of these statements must be Member-visible. If, within one week of the Team's announcement, 5% or more of the Advisory Committee support the appeal request, the Team must organize an appeal vote asking the Advisory Committee "Do you approve of the Director's Decision?" - together with links to the decision and the appeal. The ballot allows for three possible responses: "Approve, Reject, Abstain" together with Comments.

If a majority of those voting Approve or Reject vote "Reject", the decision is overturned. In that case, there are the following possible next steps:

1. The proposal is rejected (i.e. the director thinks it's hopeless)
2. The proposal is returned for additional work, after which a new Call for Review is issued. (i.e. the director thinks consensus on different terms is possible)

---

This matches what we happen to be doing...

I think we should take questions of super-majority and whether we ask people are qualified to vote to the AC. They are beyond the scope of the process CG, I fear.

[dwsinger]

One oddity about the current process is that the request for appeal merely has to garner 5% support; even if more people oppose, the appeal goes ahead. This is probably "there are enough people unhappy that we should hold the appeal", but it is a little odd. On the hand, making the request for appeal competitive rather than a mere sanity check would result in two competitive ballots (if the request passes). So, it's probably best left as it is, but it's an oddity worth knowing.

[michaelchampion]

I do find it a bit odd to have two ballots with different outcome criteria. I think it would be reasonable to specify that an appeal ballot is opened if 5% of the membership request it by email.

[jeffjaffe]

I thought we had agreed on text for this one.

Why did the editor add a label which said "Not agreed for merging"?

Why are other group participants adding new ideas after this is done? Shouldn't those be added as new issues for Process2019 at this stage?

[dwsinger]

we do have agreed text. I have no idea why the editor makes his choices. The discussion continues as we also have a desire to discuss at the AC meeting, and this is a convenient place to collect the discussion.

[dwsinger]

On the question of "qualified to vote in the appeal" two possibilities come to mind, neither ideal. (a) There were previous votes on the same topic (e.g. Rec approval) and the person in question voted; (b) in the previous 12 months, have you explicitly answered at least X% of the AC Ballots? I don't like the first because it's possible that the Appeal raises a question out of the ordinary. The second is more attractive; have you generally been paying attention to the work of the AC? But it might encourage uninformed voting on more ballots than just appeals, almost the reverse of what we want.

We could also see whether there is any traction on whether an AC Rep needs to have the standing to raise an appeal, I suppose (similar to legal standing, i.e. that you are one of the affected parties). I don't know how to determine standing easily, though.

[jeffjaffe]

I REALLY think that the "qualified to vote in the appeal" question deserves a separate issue and does not belong here. It was not part of the original issue and is not addressed in the pull request that addressed the issue which the Process CG has now agreed on.

On its merits, I like neither (a) nor (b). In most democratic societies or member organizations one derives authority to vote by being part of the society or a member of the organization; not by demonstrating that you have voted in the past.

[dwsinger]

I think there is plenty of precedent. Heck, we used to have 'good standing' here at the W3C. INCITS has a good standing rule to allow voting (you must attend 2 of the prior 3 official meetings or calls, and if you don't, you go into jeopardy and must attend the next one, or you lose voting rights.) So it's by no means unusual for standards bodies to determine voting eligibility, and I think the question needs to go to the AC. I doubt we'll resolve it this year, so we could split either or both of (a) supermajority and (b) voter qualification, into separate issues if you wish.

[michaelchampion]

I opened #76 for the good standing/supermajority question. Let's keep this issue focused on the proposed text for what happens at the end of an appeal.

[dwsinger]

Um, who is qualified to vote, and how the vote's result is determined ARE questions of what happens at the end of an appeal. They might not be resolvable in P2018, but we should try. They are not 'separate issues' unless we decide to carry on discussion in 2019 after a partial 2018 solution.