Consider making thresholds relative to active participation

[cwilso]

As per @tantek's suggestion in https://lists.w3.org/Archives/Member/w3c-ac-forum/2022AprJun/0168.html, it might be a good improvement to make the thresholds that are defined as a percentage of Membership relative to the active members - for which a reasonable stand-in (as Tantek suggests) might be the average number of Members who voted in the last two TAG+AB elections.

The bar of 5% of Membership is unfortunately quite high, as we have a large number of less-engaged-with-voting Members.

There are two places that use a percentage threshold:

• Section 5.9, on Appeals
• Section 6.3.12.4, on Process for Rescinding, Obsoleting, Superceding or Restoring a Recommendation

Both of these should probably be affected by this change.

[dwsinger]

I'd likely be supportive of a higher % of a more realistic number (e.g. the number of AC Reps who have voted at least once in the last year, including explicit abstains)

[chaals]

A majority, with at least 5% in support (i.e. in a contested case at least 10% care enough to vote) seems like it is about the right level.

In a case where an appeal is requested and supported, but the support doesn't meet the 5% threshold, I am still fine with that outcome - not enough W3C members are significantly exercised by the issue to make a formal requirement that the decision be revisited. Informally, there is a clear expression of sentiment that the decision-making procedure wasn't good enough. I trust that W3C team will see that, and take note, and that AC members likewise, so that either in the future such decisions will be handled better or the members' tolerance for it will be lower and the next appeal will have a higher response rate that makes the threshold.

[jeffjaffe]

I trust that W3C team will see that, and take note

We see it. We took note. We are working on improvements.

[frivoal]

There are various ways to determine what an "active" member is, for the purpose of determining participation levels:

1. We might, as David & Chris have suggested, only count those who have participated in enough recent elections (for some definition of "enough" and "recent")
2. We might want to exclude from the count those members who have not appointed an AC rep
3. We might want to exclude from the count those members who do have an AC rep in theory, when the registered email address has been bouncing
4. We might want to exclude members who are not up to date on paying their member dues (probably with some allowance for minor slip ups)

Ideally, 2, 3, and 4 would be negligible, but in practice, that's actually a non trivial number. Right now, there's 7 members with no AC rep, and 11 bouncing ones. The number of members behind on payments is not public, but it is known to be non-zero.

I'd suggest we at least exclude 2 and 3, because those are members who cannot take part in the ballot anyway, so counting them in the divisor skews things. I'd be inclined to exclude 4 as well, although we may want to take a broader look, and consider which rights, if any, ought to be suspended when the member is behind on payments.

[dwsinger]

I think we have always hoped that (2+3) is a 'small' number and could be negligible. But that's large enough to reduce by one the number of votes needed for an appeal (5% of 469 is 23.45, whereas 5% of 451 is only 22.55).

We've wondered for years about dues payments, but for that I am less sure.

[plehegar]

1. We might, as David & Chris have suggested, only count those who have participated in enough recent elections (for some definition of "enough" and "recent")

I would include Call for Reviews in this set. I also wonder if we should include participation in w3c-ac-forum.

2. We might want to exclude from the count those members who have not appointed an AC rep

ok.

3. We might want to exclude from the count those members who do have an AC rep in theory, when the registered email address has been bouncing

ok

4. We might want to exclude members who are not up to date on paying their member dues (probably with some allowance for minor slip ups)

I'd be fine with that, as long as we place a grace period once that new Process comes into place, eg 6 months or more.

In any case, for 1 & 4, I would suggest a period of one year. Eg, you're considered inactive after one year of lack of participation?

[jeffjaffe]

you're considered inactive after one year of lack of participation?

We should be careful about giving an AC rep the "label" of inactive.

I assume that even if an AC rep has not participated at all for years, they are always authorized to join the next survey if they choose.

[dwsinger]

I assume that even if an AC rep has not participated at all for years, they are always authorized to join the next survey if they choose.

I think by finding an answer to a different question, we can also find the answer to this one. The different question is "If I lose voting rights by failing to vote, how do I get them back if I cannot vote?".

When I mentioned above "voted at all in the previous year" I really meant "responded to a formal poll of any kind in the last year". So one way that this could work is:

1. Questions to the AC are formally divided into operational polls and governance polls. Operational polls include charter approval, and document progression. Governance polls include voting in AB/TAG/Board elections, responding to a support of an Appeal, and voting in an Appeal.
2. Those organizations that have responded in any way to X (or Y% of) polls over the preceding year are members of the Governance Advice group. Only members of this group can participate in Governance Polls, and the size of the GA sets the quotient for any percentage threshold (e.g. percentage approving a charter, percentage supporting an appeal). All AC reps can participate in operational polls.

Yes, this would mean you have to establish yourself after joining, before you can take part in Governance. No, I would not place all the members into the GA when this rule takes effect; I'd warn in advance when it's going to, and let the AC Reps choose whether to participate enough to qualify.

I think the dues payment question is different; either you're treated as a member, or you're not. We should have a warning/notice/suspension policy, I suspect.

[frivoal]

I just realized that the phrasing I used suggested something I didn't intend:

We might want to exclude…

In the case of 4, I did mean exclude from the right to vote.

But in the case of 1, 2, and 3, I merely meant to exclude them from quorum calculation. If they do turn up and vote after all, I didn't mean to suggest preventing them from voting.

[jeffjaffe]

I note that most who have posted here are fairly active. It would be worthwhile, specifically for this issue, to hear more from Members who are less active.

[css-meeting-bot]

The Revising W3C Process CG just discussed Making thresholds relative to actively-participating AC.

The full IRC log of that discussion

<fantasai> Subtopic: Making thresholds relative to actively-participating AC
<fantasai> github: https://github.com/w3c/w3process/issues/604
<plh> florian: one might need to wait until the LE is operating
<plh> ... since criteria may vary
<plh> ... (payed dues, email bouncing, etc.)
<plh> ... calculating the % based on active members
<plh> ... but number of definitions for "active members"
<plh> fantasai: let's take a strawpoll on whether we should do something about it
<wseltzer> q+
<plh> fantasai: should we modify the count on the AC for the percentage to be relative to a subset of members
<plh> wseltzer: I don't feel strongly but admin challenges in tracking activity may be greater than the benefit
<dsinger> q+ to ask the systeam for comment
<plh> fantasai: it will make a difference for appeals
<florian> q+
<plh> ack ws
<tink> q+
<plh> wseltzer: I recognize the argument but worried about cost
<wseltzer> s/cost/overhead/
<plh> dsinger: we should ask the system about what is easy to implement
<tink> ack me
<plh> ... like last voted date
<Zakim> dsinger, you wanted to ask the systeam for comment
<plh> ... if they can't, we should think again
<plh> q+
<plh> ack fl
<tink> q+
<plh> florian: +1 to ask systeam. we have email archive.
<plh> ... sent email, participating on votes
<plh> ... has an AC rep
<fantasai> ... systeam already knows which ACs are unassigned or have bouncing emails, so I think we should at least do this, and ask systeam whatever else we can do
<fantasai> scribe+
<fantasai> plh: I'm happy to ask the systeam
<florian> https://www.w3.org/Member/ACList already tracks who lacks AC reps and who's bouncing
<fantasai> ... but like to know what to ask exactly
<fantasai> ... they are tracking multiple factors
<fantasai> ... so what are the criteria we're looking for?
<fantasai> ... I should add them
<florian> q+
<plh> ... ack plh
<plh> ack plh
<plh> ack tink
<fantasai> ... can we think of all the criteria?
<plh> tink: I have a concern with using "when last voted"
<plh> ... some AC rep only vote in their area of interest
<plh> ... they have limit knowledge outside of their area
<plh> ... this would exclude some orgs
<plh> ack fl
<plh> florian: this is not to remove inactive ACs from the vote
<fantasai> s/the vote/the ability to vote/
<plh> ... this would exclude them for counting against the appeal, unless they vote
<dsinger> q+ to comment on abstain, maybe 'layers of votes' and to agree with Florian
<plh> ... if we reach 5% of the members who voted regularly, that should be enough
<plh> ... at this point, I would ask about voting records
<fantasai> ... what's the easiest way to ask, which proportion of the AC has voted in at least one thing in the past year or so?
<plh> ack fantasai
<Zakim> fantasai, you wanted to respond to tink
<plh> fantasai: if we need to make a change of the bylaws, and it needs 10% of Members saying "yes"
<plh> ... if a bunch of members don't pay attention
<wseltzer> q+ re governance vs Process
<plh> ... then the change shouldn't be 10% of everyone, but 10% who have put some amount of activity
<plh> ... otherwise we'll be stuck
<fantasai> tink: alright, convinced
<Zakim> dsinger, you wanted to comment on abstain, maybe 'layers of votes' and to agree with Florian
<florian> [Bylaws have legal requirements so we might not have a choice, but if you swap that for Process in fantasai's example, then I agree]
<plh> dsinger: also, consider abstain to be a vote
<plh> ... I did put in GH the idea of counting different types of votes
<plh> [the one vote that will be difficult to count will be the BoD votes]
<plh> dsinger: charters, seats, etc. any type of votes
<plh> ... so let's ask systeam
<plh> ack ws
<Zakim> wseltzer, you wanted to discuss governance vs Process
<plh> wseltzer: this is the process cg, not the gov tf. let's focus on the process....
<plh> wseltzer: I suggest people think how it changes the numbers and expectation if quorum gets reduced
<plh> ... if you get inactive to members to support something, you're lowering the threshold
<dsinger> q+ to say we should change the percentage if it's a change to the count
<wseltzer> s/... if you get inactive to members to support something, you're lowering the threshold//
<plh> ACTION: plh to ask systeam about counting active members
<plh> fantasai: let's skip thinking about ways to game the system
<dsinger> ack ds
<Zakim> dsinger, you wanted to say we should change the percentage if it's a change to the count

[fantasai]

The Process CG discussed this issue today. We're going to ask the systeam what kind of counts they can easily get, for example the number of ACs who have responded to any WBS poll (charter review, REC approvals, AB and TAG elections, etc.) in the last year or two. It is likely that we want exclude from the count at least those Member companies whose AC rep is unassigned or unreachable (bouncing email addresses).

[chaals]

Considering having an AC rep who is apparently contactable as a requirement seems OK by me. I trust that the team makes a reasonable effort to ensure AC reps don't fall through the gaps.

I am not sure about further requirements. Forcing people to answer a question on which they have no opinion or are happy to be guided by the majority, just so they retain "voting" rights, invites distortion of a different kind - especially given that the likelihood that each such AC rep will only participate in a couple of such events.

[dwsinger]

I am not sure about further requirements. Forcing people to answer a question on which they have no opinion or are happy to be guided by the majority, just so they retain "voting" rights, invites distortion of a different kind

which is why we propose that even filing an explicit "abstain" is good enough. Think of it as a way of saying "yes, I am still here, and I do differentiate questions on which I do or should have an opinion, from other questions – I'm paying attention". (There's a name for this device in various places, indicating that the operator is there and paying attention, but let's not use that name here)

[frivoal]

just so they retain "voting" rights

That's not the proposal. AC reps deemed inactive would continue to have voting rights.

The proposal is that when we count "did the number of people voting reach 5% of the membership", we s/membership/active membership/.

For instance, say that we have 400 AC reps, and that 197 of them voted on anything at all in the last year (or maybe two). Then somebody calls for an AC appeal, and 12 people support it, 9 of which are among the 197, and 3 aren't. Anyone can vote, so the 3 are now part of the active voters (they did vote on something within the last year, since they just did vote). So now we have 200 active members. And 12 is greater than 5% of 200, so the threshold for organizing an appeal vote is exceeded. It wouldn't be if we aimed for 5% of 400, as we do now.

Or maybe we could against 197 rather than 200, we haven't been specific about that. The point remains: the 3 can vote, and we count 5% against 197 or 200, but not against 400.

[dwsinger]

I suspect that if we had a more realistic base number, we might be able to raise the percentage above 5% too, but that takes data and thought. We're looking for the data, and thought is appreciated if anyone has any to spare.

[jeffjaffe]

I would like to propose a different, hopefully cleaner approach to this problem.

As I understand it, the problem is that some thresholds are too low. They would make sense if everyone is active. But given low activity level, they are too low.

The proposal, as I understand it, is to create some complex measure of which members are active so the thresholds can be dynamic.

But any such measure will be imperfect and contentious, and runs the risk of labeling members are being "bad" in some sense.

Here is a different simpler approach.

Let's assume that a certain threshold is 5%. Let's assume that roughly half of our members are active. Then we change the threshold to 2.5%.

That won't have the precision of being dynamically adjustable if suddenly 60% of our members are active. My guess is that whatever metric we use for "active" won't change that often. So the approximation I propose should be good enough.

[vivienlacourba]

@plehegar asked me to provide some stats, here there are:

New Members

• Between 2021-10-01 and 2022-08-12 -> 55 new Members (1 of them left)
• Between 2020-10-01 and 2022-08-12 -> 131 new Members (6 of them left)

Number of different Member organizations who voted for a WBS formal AC vote (= vote recorded on behalf of the organization)

• Between 2021-10-01 and 2022-08-12 -> 213 orgs voted on 35 questionnaires
• Between 2020-10-01 and 2022-08-12 -> 281 orgs voted on 65 questionnaires

Number of "active" AC Reps Members taking into account both recent Members and Members who voted (i.e. the intersection between the above populations)

• Between 2021-10-01 and 2022-08-12 -> 251 "active" AC Reps
• Between 2020-10-01 and 2022-08-12 -> 355 "active" AC Reps

[dwsinger]

@vivienlacourba Thank you, this is helpful. We actually had two questions; the first you answer above. The second is whether we could have automated tooling that would tell us, for any given vote, how many members are either in their 'honeymoon' period after joining, or have voted at all. Is that automatable?

[vivienlacourba]

@vivienlacourba Thank you, this is helpful. We actually had two questions; the first you answer above. The second is whether we could have automated tooling that would tell us, for any given vote, how many members are either in their 'honeymoon' period after joining, or have voted at all. Is that automatable?

Yes probably. My reserve is that it might not be as easy to obtain real time data once we operate as the LE as some of the Membership data (e.g. the date when a particular org became Member) is expected to transition to the platform of the administrative partner.

Also note that this number would be different if you use the opening or end date of the vote.

As @jeffjaffe noted I doubt that this "active" AC Reps percentage will change often so believe his suggested simple approach is worth considering.

[fantasai]

@vivienlacourba I think embedding changes of the percentage in the Process document as a constant is not really ideal: people will forget to update it, or we might be on a slow update cycle for various other reasons (as in this year), and eventually the task gets forgotten.

I don't know that we need the data real-time, either; probably just picking one day a year to do the count and using that number until that same date next year would be fine. It gives us a proportion (e.g. 125 of 467 members are “active” this year) and the Team can just multiply that by the number in the Process whenever necessary (125/467*5%). I'd rather have the Team do this than the Process CG, though. :)

[frivoal]

I wonder if this is still an issue that needs solving, or whether the clean-up of the membership that happened as part of the transition to the legal entity reduced the number of inactive members enough that this is no longer a pressing concern.

[plehegar]

I wonder if this is still an issue that needs solving, or whether the clean-up of the membership that happened as part of the transition to the legal entity reduced the number of inactive members enough that this is no longer a pressing concern.

We haven't used lately the 2 parts of the Process that do use the 5% metric. So it's hard to be definitive. My intuition tells me that the clean-up of the membership did not move the needle for this issue.