Open palemieux opened 4 years ago
@palemieux I think it's hard to say that they will never become Recommendations. It's entirely possible that some specs that are under active evolution for many years in CR eventually stabilize enough to shift into REC status. RECs are still malleable, there's just a bit more process around making changes to ensure they continuously meet the additional criteria.
And there's still some value in viewing a REC, which requires two interoperable implementations, as the end state for a standard to strive for, rather than settling for a CR, which has lesser qualifications.
It's entirely possible that some specs that are under active evolution for many years in CR eventually stabilize enough to shift into REC status
What about specifications that a WG indicate will never become a REC?
[edit: for clarity]
I'm a bit ambivalent about this. Yes, for some groups, CR will be the intended end state, which makes the name not really appropriate. At the same time, for some other groups, CR will continue to fulfill the role it's always had, and for them it's still right.
So "Candidate Recommendation" isn't perfectly descriptive of all the case it'll be used in. But I am not sure that fixing this is actually a good idea. There's a bunch of existing expectations and institutional knowledge about what it means for a spec to be at CR, and changing names would mess with that.
It's not like there's no precedent with odd names for standards that are more grounded in history than in current practices. "Request For Comments" is somewhat weird and doesn't really reflect what those documents actually are. But we all know what it means, so it's there to stay.
The AB felt that we do not have the time to give this proper consideration for P2020, and it's a naming issue, not functionality. Keep open and discuss.
I think it is more than merely a naming issue. The CR SotD states:
Publication as a Candidate Recommendation does not imply endorsement by the W3C Membership. [...] It is inappropriate to cite this document as other than work in progress.
Does this still apply to technical reports that remain CRs at perpetuity?
In other words, should external organizations not reference specifications that are intended to remain CRs at perpetuity?
The comment about maturity is more important than the naming issue and I think it does matter. A document that doesn't get proper W3C review, but is published by a Working Group on their own with no broader endorsement is a Note. A document that does have that is a Recommendation. CR is neither one nor the other.
Yes, the status question is important. We are getting closer to the IETF 'RFC' status; formally 'requests for comment' but actually what the internet runs on. CRs have more status than Notes, just as standards-track RFCs do — they are on the way to becoming standards. But they are not Recs, just as most RFCs are not Standards.
In other words, should external organizations not reference specifications that are intended to remain CRs at perpetuity?
@palemieux I agree that this is more than a naming issue.
I don't think that anyone on the AB or Process CG wants to confer the same "status" to a CR as we do to REC. There are reviews that are not yet done (e.g. AC Review) so it is impossible to state that these documents have the consensus and endorsement of the W3C Community. There was an ac-forum discussion about how there is continued value to the REC status.
Clearly, there will be groups that are more focused on having patent commitments and an updated view of the WGs consensus. Indeed, some may not proceed to REC. If they take that choice, they are choosing that their document doesn't need to have endorsement of W3C. That will apply in some cases.
Your question about whether perpetual CRs may be referenced is therefore more complicated. Our documents are referenced all the time; even CG documents are referenced. So the issue is not whether they are allowed to be referenced. The question is what does a reference "mean".
Hence, if a group wants to reference a W3C specification that is a stable reference and has the consensus of the W3C Community, they should not reference a perpetual CR. But in other cases, groups might want to reference a perpetual CR.
@palemieux are you concerned that there may be specs that you want to reference as RECs, but will never get there?
It seems to me, that if some in the community need a spec to get to the full REC level, that requirement should be surfaced to the WG (e.g. in the AC review of the Charter); and in that case most WGs would respect that requirement and not leave things in CR in perpetuity.
@jeffjaffe My concern is with lack of clarity with what it means for a Technical Report to remain at CR at perpetuity. This concern applies to both (a) external parties that wish to reference the Technical Report and (b) W3 groups that need to choose the right path for their Technical Report.
In particular:
To be sure, I do not object to adapting the requirements for publishing consensus documents within W3C. I am particularly excited by the idea of simplifying the maintenance of Recommendations.
In discussion in the Process CG we realize that the SOTD needs work, see separate issues on that. We're not confident of our ability to edit the Process in 2020 but leave this issue open to collect thoughts on a future revision.
It seems to me, that if some in the community need a spec to get to the full REC level, that requirement should be surfaced to the WG (e.g. in the AC review of the Charter); and in that case most WGs would respect that requirement and not leave things in CR in perpetuity.
@jeffjaffe this is a worrying statement! I've never seen a Charter say "these are Rec track documents but we never intend to get to Rec". The working assumption of anyone reviewing a Charter has got to be that it is the WG's intention to get to Rec, rather than the other way around. Otherwise what's the point of having the document on the Rec track at all?
Analogously, if someone says "I plan to go to the store to buy ice cream" then would you respond to say "ok go ahead, but do make sure you return with ice cream"?
Concretely, it is very difficult to imagine any change to a Charter that would somehow add a stronger requirement for getting to Rec. Do you have something specific in mind?
I'm a bit ambivalent about this. Yes, for some groups, CR will be the intended end state, which makes the name not really appropriate. At the same time, for some other groups, CR will continue to fulfill the role it's always had, and for them it's still right.
Sorry to be blunt @frivoal but CR being the intended end state should not be acceptable. This is like saying "the intended state is permanent lack of clarity about the status of this work in the industry". W3C should never support this as an end state for a Rec track document. Even in the "Rec snapshot of continually updated document" model there are at least fixed points of clarity.
@nigelmegitt I think your response to Florian answers your question to me.
In your response to Florian you state that CR should never be the intended end state. But Pierre points out that some groups might do exactly that. In some respects, that happens today - we unfortunately have many specs that have languished in CR for too long.
My simple response is that if a stakeholder sees that happening - a spec stuck in perpetual CR - and if they need (e.g. for referenceability) that the spec go through full W3C endorsement - they have an opportunity to drive for such a commitment in Charter review. For example, they can object unless a particular mature snapshot be taken immediately to PR.
@jeffjaffe sorry I don't follow: how would Charter review, when there may only be an FPWD, or a mere concept for a specification, be an effective time to press for a push to Rec?
There's some relief here due to the slightly reduced typical Charter period, which means that re-Charters have to happen typically every 2 years or less.
Let's imagine that during a re-Charter, an existing spec that has been in CR for a long time were highlighted and a request made to add some language to the Charter asking the WG to push ahead for CR exit. That still doesn't make anything happen, as far I can tell.
@nigelmegitt I agree that for a new spec that is in FPWD, an AC rep should not complain that it has been in perpetual CR for too long. That is not the right time to object.
You are correct, that the time to catch it is when a spec has been in perpetual CR for too many years (from the pov of the AC reviewer). You are also correct that an AC reviewer cannot force a perpetual CR into REC in P2020 any more so than they can today. But I believe that if the AC community expresses their view that this is needed, that the W3C would find a way to bring the spec to REC - much as we are bringing WHATWG Review Drafts of HTML to REC.
Also, as @fantasai has pointed out, while the process to maintain a "living standard" at REC is heavier than it is for a CR, P2020 makes that possible too. Groups that favor a living standards approach have no less interest than everybody else in high quality specifications (and the criteria about the spec itself for getting to CR are as high as for REC, the difference is in terms of implementation experience and AC review). So once a spec is good enough to go to REC, and the rate of change is low enough because the technology is mature, and there is demand for a REC, it's not completely out of the range of possibility that "living standards" oriented group could still go to REC, and continue maintaining the spec in that state. It just isn't their priority (and isn't expected to be for quite a while). But that path remains open.
Given that (a) you can maintain in Rec as well as in CR and (b) anyone can ask/push for a Rec transition if it's needed, I would surmise that the reason a document stays in CR semi-permanently is because the WG and the community are comfortable with it.
There are some documents where there is a gentle trickle of updates and no obvious updates that are 'more significant' and would by themselves trigger a Rec. transition. But anyone can ask, and the AC could (effectively) insist.
@frivoal wrote:
once a spec is good enough to go to REC
@dwsinger wrote:
the WG and the community are comfortable with it
The problem I think we have is that specs are staying in CR, and being normatively referenced, when they clearly are not verified as being good enough to go to REC, because the CR Exit Criteria have not been met, but that the WG is comfortable with the situation. That does not equate to the community being comfortable as well, but it's hard to gauge the impact of this. My guess is that folk will vote with their feet and the W3C will lose relevance. That's why it's a Process issue.
I agree with @palemieux at https://github.com/w3c/w3process/issues/402#issuecomment-634408246 and I don't think the solution is to promote CR to being the new acceptable end state.
@frivoal wrote:
once a spec is good enough to go to REC
@dwsinger wrote:
the WG and the community are comfortable with it
The problem I think we have is that specs are staying in CR, and being normatively referenced, when they clearly are not verified as being good enough to go to REC, because the CR Exit Criteria have not been met, but that the WG is comfortable with the situation.
Maybe the document is good enough to go to Rec but the WG chooses not to. We shouldn't guess. They may feel the time isn't right, or that there is no particular time that is more right.
My guess is that folk will vote with their feet and the W3C will lose relevance. That's why it's a Process issue.
But there are other ways the community can vote with their feet as well, and the W3C lose relevance. The high nail today is that we suck at staying current, and doing maintenance, and the community is voting with their feet and finding ugly workarounds.
I don't think the solution is to promote CR to being the new acceptable end state.
I don't think we're either promoting it or deprecating it. We've had long-lived CRs for years; we're not changing that. We've had the ability to ask for a Rec. transition to occur; we're not changing that either.
Maybe the document is good enough to go to Rec but the WG chooses not to.
Has the process CG documented the reasons WGs do not to proceed to REC?
We've had long-lived CRs for years; we're not changing that.
If a perpetual CR in an acceptable end state, then the process should make this explicit.
Has the process CG documented the reasons WGs do not to proceed to REC?
Not formally, but in my experience WGs don't go to REC because of one or more of:
Taking a spec to REC is a lot of work. The AC can stamp its feet and say anything it wants in the charter, but a WG isn't going to take the spec to REC unless someone puts in the work.
However, if some community wants that work to happen, I'm sure any WG would be happy to support such efforts. Everyone wants better quality specs, tests, and implementations.
Maybe the document is good enough to go to Rec but the WG chooses not to. We shouldn't guess. They may feel the time isn't right, or that there is no particular time that is more right.
@dwsinger The suggestion here is that the spec has passed all the exit criteria but the WG deliberately delays requesting transition to PR. I can buy that for a matter of weeks, but not months or years. Are there any examples?
Taking a spec to REC is a lot of work. The AC can stamp its feet and say anything it wants in the charter, but a WG isn't going to take the spec to REC unless someone puts in the work.
@fantasai exactly right. My proposal at https://github.com/w3c/w3process/issues/406#issuecomment-636092886 would change the dynamic by moving the cost or penalty of inaction to the WG, where currently it is a reputational cost to W3C as a whole.
The hope would be that this encourages (funding of) work to address the open issues or the QA work, since the alternative is to reduce the value of the existing investment.
There's known bugs / missing features in implementations that haven't yet been fixed, such that we don't have two interoperable ones
If this is genuinely holding up specs then it sounds like the WG is holding their spec up to a higher standard than is required for getting to Rec - my understanding is that CR exit criteria must minimally demonstrate implementability of each feature, and that the implementation report need not consider interoperability per se. This could be another point to address via a clarification in the Process.
if some community wants that work to happen, I'm sure any WG would be happy to support such efforts. Everyone wants better quality specs, tests, and implementations.
Sorry, I think this is unrealistic. There's barely any chance that a community wanting a spec to be a Rec, e.g. to get a clearer view of its status, will step in and start doing QA work in it, since they will always expect the community that owns the spec to do that.
@dwsinger The suggestion here is that the spec has passed all the exit criteria but the WG deliberately delays requesting transition to PR. I can buy that for a matter of weeks, but not months or years. Are there any examples?
I think we've had cases where the spec. as it is, actually could go to Rec. but that the WG feels it shouldn't until some feature or missing functionality is addressed. I don't know. It would take team/chair conversations to find examples.
The SOTD doesn't ask that external organizations NOT reference; it says it's inappropriate to reference other than as a work in progress. That's true for an evolving CR, aka Living Standard.
There is no such thing as "perpetual"; all we know is the current state and the past; the future is uncertain.
I don't see a problem with a slightly quaint term; no more quaint than the IETF 'requesting comment'. And it's not even wrong; it could be proposed to move to recommendation.
I don't think either suggestion is appropriate: changing the name CR would be moving away from a term we are known for an know well. Calling CRs Recommendations would be... wrong/strange.
But we should pay close attention to the SOTD boilerplate. I believe that the SOTD is the place to fix this.
The SOTD doesn't ask that external organizations NOT reference; it says it's inappropriate to reference other than as a work in progress. That's true for an evolving CR, aka Living Standard.
@dwsinger These two sentences seem to highlight a possible difference of world view: getting to alignment might really help us to move forward:
I don't know any other standards organisations that include normative references to unverified works in progress as a matter of course (I do know of SDOs that don't have any formal requirements but leave it to contributors though, like ETSI). Therefore, from a practical perspective, saying "inappropriate to reference other than as a work in progress" is effectively asking not to reference.
"evolving CR" != Living Standard: "living standards" hold each normative change up to a higher standard before accepting it than CR does. Specifically, tests are needed as well as either implementations that pass them, or commitment to implement. Whereas no testing at all is required to enter CR. This is a significant difference.
1. I don't know any other standards organisations that include normative references to unverified works in progress as a matter of course
We already support normative references to WHATWG HTML Living Standards when that is the most appropriate reference.
We already support normative references to
@jeffjaffe I'm talking about other organisations referencing W3C standards. But if you're trying to highlight existing weaknesses in W3C, then yes, that is a true statement.
(Note that this issue was resolved deferred on the 27 May 2020 telecon.)
Ah, I missed a resolution from the Call on 27 May 2020
RESOLUTION: we keep 402 without Process changes this year, but turn our attention to improving the SOTD
@dwsinger @fantasai re the resolution from the call on 27 May, I wonder if this group has a Decision Policy? Given that :
a) the resolution was not recorded on this GitHub issue and b) not all the contributors on this issue were present on the call, and c) the conversation continued here with apparent lack of consensus, and d) there are other process changes relating to CR in P2020
I would call on the Chair to re-assess whether the resolution from the 27th May should stand.
@nigelmegitt This issue is about renaming the stage. We're not going to do that at this point.
If you have other concerns with P2020, you should file them separately and clearly.
I do want to clarify the timeline for P2020, since Jeff made it seem like we just started in May.
I do want to clarify the timeline for P2020, since Jeff made it seem like we just started in May.
I'm not sure how I communicated that we just started in May.
In my view, the thinking about Living Standards (and hence the conceptual beginning of Process 2020) started well before TPAC 2019. It actually started with the raising of Issue #79 on September 8th 2017. We tried to get LS addressed in Process 2019 but it was too complex and we could not finish in time. So when P2019 was otherwise ready for Member review, we decided to roll our thinking about LS into P2020.
This issue is about renaming the stage. We're not going to do that at this point.
@fantasai I don't think it is about that, and from https://github.com/w3c/w3process/issues/402#issuecomment-632781320, it seems that neither does @palemieux .
I feel as though the issue is being mis-represented to make it easier to close off or postpone. I hope I'm wrong.
If you have other concerns with P2020, you should file them separately and clearly.
I think there have already been at least 2 lengthy issue or pull request threads about dealing with issues around perpetual CRs. I think you're asking me to open a 3rd. It doesn't seem efficient to keep recycling issues and risking repetitive or confusing threads. Wouldn't it be better to deal with the issue instead?
By the way, I think @jeffjaffe communicated about the P2020 timeline at https://github.com/w3c/w3process/issues/406#issuecomment-636893783 - a great example of the confusion of threads on this topic!
@dwsinger @fantasai re the resolution from the call on 27 May, I wonder if this group has a Decision Policy? Given that :
a) the resolution was not recorded on this GitHub issue and b) not all the contributors on this issue were present on the call, and c) the conversation continued here with apparent lack of consensus, and d) there are other process changes relating to CR in P2020
I would call on the Chair to re-assess whether the resolution from the 27th May should stand.
The issue remains open for discussion. The decision on May 27th was not "close" but "defer, not for P2020". I asked again today if anyone wanted any discussion or had proposed action, and no-one did.
There is a narrow question here: should we have a different name for long-lived CRs? And a broad one: do we want to be doing Living Standards at all, and if so, in a way that's integrated with the current process and stages, or as a separate track?
We definitely answered the broad question: yes, we need better maintenance and evolution, yes, we want something like LS, and yes, we want it integrated.
The group feels that though the name is not ideal, it's good enough for now, and introducing a new term/concept at this late stage is more than cleaning up and implementing decisions. We also don't have specific proposed changes here. Personally, I feel it's no 'worse' than RFC at the IETF; slightly odd, but a recognized name.
I would urge everyone to make sure that the question that they want addressed is within the scope of the issue that is commented on; this and #406 both risk wide-ranging conversations, and splitting wide-ranging conversations into a set of focused ones generally helps focus the conversation and increase the likelihood of convergence.
So, overall, though there is consensus that we should not make changes in P2020, there is consensus that the issue should remain open for discussion.
That's my read of the situation.
By the way, I think @jeffjaffe communicated about the P2020 timeline at #406 (comment) - a great example of the confusion of threads on this topic!
Nigel, in #406 you had asked about the P2020 deadlines, so I responded with the going forward schedule. That in no way was an implication about when we started.
That in no way was an implication about when we started.
@jeffjaffe indeed, and I didn't make that inference.
We have such a strong 'brand name' around CR, I think it's similar to the IETF Request for Comments. I think I need a strong reason to lose the branding.
closing no action
Serious consideration should be put into revisiting this issue.
The arguments by @palemieux in the issue description and in his other comments are objectively compelling, and specifically the statement “"Candidate Recommendation" is no longer an accurate term” is objectively correct.
Organizational inertia seems to be the only thing collectively making us (me too) want to keep clinging to the Candidate Recommendation term despite the strong evidence we have for the confusion and potential for confusion it creates.
In the interest of offering at least one really solid idea that may have a chance of actually resolving this to the satisfaction of all — and considering: we actually have two terms we need to replace: CR Snapshot and CR Draft, and that in replacing those, we might also optionally want to tweak/qualify the Working Draft term a tiny bit — here’s a concrete proposal:
CR Snapshot | CR Draft | Working Draft |
---|---|---|
Working Group Specification Snapshot | Working Group Specification | Working Group Draft |
Changing (further qualifying) WD to Working Group Draft wouldn’t be strictly necessary; but I can imagine some of us might end up feeling that for clarity and consistency/congruency with our names for the other stages/states, it’d look/read better.
I’ve opened https://github.com/w3c/process/pull/890 with a patch that does the above. You can view the results here:
Although i’m not as involved as i used to be, i’ve seen a number of specs languish at CR because
None of these seem to be reasons to refer to a CR or to legitimize the state.
I’ve also seen a CR go back to WD when it was found not possible to implement.
A CR does not specify a Working Group (I suppose that might be a charter), so Working Group Specification doesn’t seem helpful. How about, “Incomplete Recommendation”? Or, “Draft Pending Implementation”?
Removing CR altogether and combining the requirements with Last Call Working Draft makes more sense to me. Then it’d be at Last Call for Comments and Implementations, and people outside W3C would understand better what was going on.
I appreciate the input @sideshowbarker - did you consider that "Working Group Specification" could be misinterpreted as a being a Charter, i.e. the specification for a working group rather than a specification by a working group? Similarly, "Working Group Draft" sounds like some non-final version of a Working Group, rather than of a specification.
@liamquin although LCCI represents what CRs should be in my view, I don't think it reflects how they are actually used:
did you consider that "Working Group Specification" could be misinterpreted
In general, I think any possible replacement we might consider could be misinterpreted in some way.
But I think Candidate Recommendation is relatively more prone to being misinterpreted than Working Group Specification.
misinterpreted as a being a Charter, i.e. the specification for a working group rather than a specification by a working group?
That’s not a possible misinterpretation that had occurred to me, no. But to me at least, it anyway doesn’t seem like a misinterpretation that would happen very commonly.
And even when it did happen that some person initially misinterpreted it like that — it’d be a one-time misinterpretation that would get dispelled very quickly, in a moment when they Aha! realized what it actually meant.
Contrast that to Candidate Recommendation being prone to a very different kind of ongoing misinterpretation — which goes on creating problems and confusion that can’t be cleared up in someone’s mind in a single Aha! moment of realization.
Similarly, "Working Group Draft" sounds like some non-final version of a Working Group, rather than of a specification.
That seems like a bit of stretch. I at least find it very unlikely that anyone in practice would imagine a “draft” Working Group…
How about, “Incomplete Recommendation”?
Not good, because an implicit goal of the proposal — based on trying to get some resolution among at least the set of people who have commented here — is: completely stop using Recommendation to refer to stages/states that aren’t actually Recommendations (that is, things that aren’t formally/fully endorsed by the W3C organizationally — the W3C membership).
Or, “Draft Pending Implementation”?
Something relatively-more arcane/esoteric like that doesn’t seem to me at least like it’d be a choice that’s likely to reduce confusion. That feels like it’d increase confusion — it would increase the amount of time/documentation we’d need to spend explaining what we mean.
Working Group Specification, on the other hand, is something that we and others already use informally when referring, well, to the specifications that our working groups produce. We don’t need to explain to people what it means — it doesn’t increase confusion, and doesn’t have the problem of being inaccurate at all.
Removing CR altogether and combining the requirements with Last Call Working Draft makes more sense to me. Then it’d be at Last Call for Comments and Implementations, and people outside W3C would understand better what was going on.
I definitely see your point there. However it seems like Last Call for Comments and Implementations (or similar) would be a bit unwieldy. But regardless, it’d be a non-starter as far as trying to resolve to issue-author/commenter satisfaction the problem in the issue description here — and which is further amplified on in some of the discussion comments: Which is, we need something that’s also accurate in the face of the “living standard” or “evergreen” option that W3C now provides.
And for that, Last Call for Comments and Implementations seems at least as inaccurate as Candidate Recommendation was.
need something that’s also accurate in the face of the “living standard” or “evergreen” option that W3C now provides
That’s actually not the most-precise way to describe what we need.
A perhaps-more-precise way to describe what we need is:
The W3C charter template has the following text that can optionally be in any charter’s Deliverables section:
The Working Group does not intend to advance their documents to Recommendation.
We already have at least 5 WGs chartered with the above language in their charters: Web Application Security, Browser Testing and Tools, ARIA WG, Service Workers WG, and RDF/JS WG.
Lack of that language explicitly being in their charter does not prevent them from making that choice.
And so, given all that, we need something that’s also accurate in the face of that “does not intend to advance their documents to Recommendation” option that the W3C allows as a choice for WGs.
So, the term Candidate Recommendation objectively is not accurate in the face of the facts outlined in https://github.com/w3c/process/issues/402#issuecomment-2178822524.
And beyond being not accurate in that case, Candidate Recommendation is actually actively misleading — and arguably even actively harmful — in that, for example and to quote verbatim some words from a related discussion elsewhere, it can (mis)leading people into assuming wrong conclusions such as “the document is heading towards a W3C-wide consensus”.
But it is impossible for “does not intend to advance their documents to Recommendation” specifications to at the same time also ”heading towards a W3C-wide consensus” specifications.
So rather than a Candidate Recommendation, a more-accurate name for those cases would be something like: “Not Candidate Recommendation” or “Not Headed for Recommendation”.
But a name like Not Headed for Recommendation — or whatever actual less-silly name we might try to find to accurately name that case — would of course not all accurately cover the other case — the case of specifications which really are “heading towards a W3C-wide consensus” or “heading towards Recommendation”.
Hence, we really need a single name that can at least not inaccurately describe either case — and that can at least not be actively misleading and at least not arguably even actively harmful.
And Working Group Specification seems to satisfy those requirements.
I agree with the intent to separate the 'document headed to REC', namely a Candidate Recommendation, from a 'living something until whenever we decide to move on', which has no specific name. But IMHO it's not just a naming issue. The expectations need to be clearer in terms of wide review of substantive changes and also implementation requirements (currently basically no requirement at all after publishing a CRS for the first time).
'living something until whenever we decide to move on',
To be clear about one thing: In many or most of the actual WG cases I’m aware of, there’s no “until whenever we decide to move on” — that is, it’s just “living something”, with intentionally nothing else conceived to ever move on to, but instead a plan to just indefinitely continue maintaining that “living something”, at the stage with whatever name we label that for them in our process.
The term Candidate Recommendation accurately and literally describes the state of a technical report that is expected to become a Recommendation.
Process 2020 explicitly contemplates technical reports remaining Candidate Recommendation at perpetuity, but being regularly revised (as a Candidate Recommendation Snapshot or Candidate Recommendation Draft).
The term Candidate Recommendation hardly describes these specifications, which will never become Recommendations.
Suggest either: