The 2021 "Revising a Recommendation Process" is painful for WGs

[msporny]

TL;DR: The work and tooling required to execute upon the new "Revising a Recommendation Process" in Process 2021 is most likely far more manual and painful than the Process CG intended. spec-prod is struggling with finding a solution that isn't going to require significant education across every WG maintaining a REC. A proposal to address the situation is outlined below.

@msporny wrote:

As an Editor, I had to manually mark up the revised REC substantive changes over the weekend (we had been merging editorial and candidate changes over the past two years before the 2021 Process existed) and it was not an experience I would wish on anyone. link to thread

@marcoscaceres wrote:

I feel pretty strongly that Editors shouldn't be put in a position where they have to manually include before/after changes. I'm hearing a lot of negative feedback from folks in multiple working groups about how annoying/frustrating this process is going to be, which is really concerning to me as and editor, tool maintainer, staff, and as a chair. link to thread

@msporny wrote:

The simplest fix that I can think of is changing to this mode of operation:

1. The changelog in the specification lists "editorial changes" and "substantive" changes since the last release, like we do here: https://www.w3.org/TR/vc-data-model/#revision-history

2. You either link manually to the PRs for substantive changes and/or we upgrade rs-changelog to make that easier for folks.

3. We get rid of all the proposed correction markup and point AC Reviewers to the Changelog at the bottom of each spec (which has links to the PRs if they really want to dig in).

4. We provide a diff-marked version as a part of the review content to the AC.

If we do those things, and IIUC, we don't disrupt the regular Github workflow that most WGs are using. We just require "editorial" or "substantive" labels on PRs done when revising a REC (which we already require when generating the new errata.html pages). link to thread

[frivoal]

(Small note: those provisions were introduced in Process 2020, not Process 2021, though of course they're still in this newest one as well).

As far as I can tell, the Process which you are proposing is already available (and has always been), by cycling through CR. In that case, the various kinds of inline markup and overhead that you're unhappy about aren't needed, and a changelog+diff is indeed sufficient.

The process that you're unhappy about is intended to allow piecewise incremental update of a REC. If you have a large amount of batched changes that have never been put to TR, and you want to not only add them all at once, but also get formal W3C backing for all of them at once, yes, it's going to be painful, but it is not intended for that.

It is intended to allow a WG to add changes to a REC one by one, as the WG runs into them, without batching, nor without administrative gatekeeping or review. Everytime the WG wants to change something in the REC, you can highlight in place the change you want to make, republish daily if you want, without needing AC review or patent review or director permission or anything beyond group consensus, and move on. Then, once in a while, for those changes that have stood the test of time, you issue a call for review to make them official, and if they are indeed accepted, you fold them in. Since you can be adding more changes on an ongoing basis, not all may be equally mature, so there's an allowance for only making some of them official each time you ask for review while keeping the rest as a work-in-progress, in order to make sure that the WG does not need to stash the less mature changes somewhere else in the meanwhile, and can indeed put all its latest work in the same place.

There's a tradeoff: a REC is a document that has the backing of more than just a Working Group. We can disallow the WG from making any changes to it unless it gets permission, or we can allow the WG to make provisional changes in place without permission, provided that they're highlighted as such until they do undergo formal approval by the rest of the consortium. Both approaches are available in Process 2021. But publishing a document as a REC, indicating it has the formal backing of the whole consortium after having undergone various types of reviews and approval, and then letting the WG change it's content without control while still calling it a REC doesn't seem reasonable.

Yes, there is some overhead in doing that, but (1) you can skip that by batching them and going through CR, (2) RECs are supposed to be reasonably stable, so the amount of churn ought to be limited, and (3), in time, I'd expect tooling to appear to help editors generate these annotations from good old fashioned diffs.

Also note that all of that is just for substantive changes. For editorial ones, you can just republish your REC, without permission or additional markup, every time the WG wants to make one. (Or you can, if you prefer, batch your editorial changes with your substantive ones, and put them through a CR. But you don't have to).

For your situation, what I would have suggested, since you already had a large batch of changes (both normative and editorial) folded into the ED after multiple years of work and since you wanted seek approval for all of them at once, would have been to issue a CR, get that approved and brought back to REC, and only then, start using these new process provisions to deal with any future change on an incremental basis.

Splitting an existing large batch of changes into individual annotated pieces, only to ask approval for them in a single batch anyway does indeed seem like a lot of unpleasant work.

I guess I am a little bit confused: as far as I can tell, the pre-existing path of cycling through a CR+PR appears to do exactly what you wanted: ask for mass approval for a batch of changes documented via changelog. Which features of the new part of process where you interested in? It doesn't appear to be the ability to do changes one at a time (since you batched them all), nor the ability to publish changes before they're all mature enough to go to AC+Patent Review (since you went straight for Proposed changes without going through Candidate changes first for any of them).

[marcoscaceres]

The above all sounds great in principle, but it's not really matching reality of what I'm seeing from HTML in Aria for instance: https://github.com/w3c/html-aria/pulls

Yes, there is some overhead in doing that, but (1) you can skip that by batching them and going through CR,

This defeats the purpose, no? We want to incrementally update RECs.

(2) RECs are supposed to be reasonably stable, so the amount of churn ought to be limited, and

Again, see https://github.com/w3c/html-aria/pulls ... "amount of churn ought to be limited" doesn't seem to be a good working assumption... or at least, I don't see how that will hold in practice.

(3), in time, I'd expect tooling to appear to help editors generate these annotations from good old fashioned diffs.

The problem is that this also assumes limited churn... sections could move... and correction could be made to corrections.

I kinda wish we had worked out the tooling first (or had gotten funding to figure out how we do the tooling to actually support this).

For your situation, what I would have suggested, since you already had a large batch of changes (both normative and editorial) folded into the ED after multiple years of work and since you wanted seek approval for all of them at once, would have been to issue a CR, get that approved and brought back to REC, and only then, start using these new process provisions to deal with any future change on an incremental basis.

Again, I think this is dismissing the reality of the the situation: The reality is that specs will be updated just like Manu's spec was (again, see HTML in Aria)...

As another case in point - or more proof: I completely rewrote the Geolocation API Recommendation from scratch to republish it as a REC (like, I think 99% of it). If we'd been operating under this regime, the assumption made above wouldn't have held at all. Our ways of editing and our tooling constantly changes. Imagine if we had to update SVG or some other old REC... we would totally be rewriting it completely. It's a bit fantasy/wishful thinking to assume that RECs getting updates won't include massive amounts of changes (Editors should be free to rewrite RECs if they feel the REC would benefit, and not have to go back to CR). This should be the default working mode.

I guess I am a little bit confused: as far as I can tell, the pre-existing path of cycling through a CR+PR appears to do exactly what you wanted: ask for mass approval for a batch of changes documented via changelog. Which features of the new part of process where you interested in? It doesn't appear to be the ability to do changes one at a time (since you batched them all), nor the ability to publish changes before they're all mature enough to go to AC+Patent Review (since you went straight for Proposed changes without going through Candidate changes first for any of them).

I think ideally we want the equivalent of the WHATWG "no nonsense" publication model... or something close to it that is not painful.

Bar that, we want something simple, like:

• If there is a deletion, them just delete (will show up in htmldiff and in changelog).
• If it's a "correction", add <span class="proposed correction"> and simply correct the text, deleting whatever text was there.
• If it's an "addition", then wrap with <div class="proposed addition"> or <section class="proposed addition"> for the addition... or use any container element. If it's an inline, use a <span class="proposed addition">.
• Finally, add a Changelog section that just shows the pull requests links.

[frivoal]

This defeats the purpose, no? We want to incrementally update RECs.

Well, sure, but if you have 2 years worth of changes that you want to push through at once and get AC+Patent review on all the changes at once, that doesn't look very incremental to me. I wasn't suggesting sticking to CRs forever for that spec, just to evacuate this large batch, and then do the incremental thing after that.

I think ideally we want the equivalent of the WHATWG "no nonsense" publication model...

The WHATWG model has merits, but what it doesn't get you is the official backing from the whole consortium. If you want WG agreement and Patent protection, you (now) get that at CR. REC means it's been vetted and endorsed by more people than that. You cannot get the endorsement from a large body without going through the steps to validate it with that body. Before Process 2020, we had the ability to get the whole document re-vetted by going back to CR and to REC again. Process 2020 added two things:

• Patent protection at CR rather than REC, so that you get Patent protection parity with the WHATWG at a level of endorsement that is similar to that of the WHATWG
• Ability to make incremental updates to a REC with the same degree of vetting that you get when getting to REC the old fashioned way, so that the meaning of REC doesn't change and they continue to carry the endorsement of the consortium as a whole.

Lowering the amount of vetting that the normative text of a REC gets, and thereby lowering the strength of its endorsement was an explicit non-goal for P2020. The goal was making it easier to update a REC, not changing what it means to be REC.

I completely rewrote the Geolocation API Recommendation from scratch to republish it as a REC (like, I think 99% of it)

I am confused. As far as I can tell, Geolocation API never was a REC, and still isn't a REC. A working draft only has the endorsement of it's working group, and not even as a final output, so of course you can rewrite the whole thing in place if the WG wants it that way. If it had been a REC, it'd be a little more curious as to why a rewrite from scratch would be necesssary, and even more so about why you'd want to do this through incremental patching-in-place of a REC. If the whole spec has been rewritten, would it not make more sense to take it back to REC the old fashioned way?

Editors should be free to rewrite RECs if they feel the REC would benefit, and not have to go back to CR

I question this. There can be benefits to rewriting a spec from scratch, sure. But I don't see why we'd need ability to rewrite a spec from scratch while pretending it is continuously at the most stable maturity stage of spec development and endorsed by the whole consortium.

(Also WGs own specs, not editors, but that's kind of orthogonal).

[marcoscaceres]

Please see: https://www.w3.org/standards/history/geolocation-API

We had to mint a new short name to work around the process: https://www.w3.org/standards/history/geolocation

the reason for rewriting it was that it didn’t match implementations, now uses infra, Permission API, etc. it needed a lot of modernization.

I question this.

I’m wondering if that your personal experience (which is really valuable, particularly in CSS!) might be different to what happens in other working groups? It’s good to question, but Manu and I are presenting hopefully clear evidence that might require to re-check some assumptions about RECs and how other folks are updating them… or would like to update them.

[marcoscaceres]

Just to be clear, I fully expect to rewrite significant parts of all specs I work on every few years. So much changes in DOM, HTML, WebIDL, Infra… and tooling that it would be unrealistic and/or irresponsible not to as part of general maintenance.

I think if we looked at any DOM API spec that’s about a decade old we could see that it would be in need of a rewrite.

[msporny]

@frivoal wrote:

I guess I am a little bit confused

@frivoal, thank you for taking the time to write that detailed response. It (once again) changed my understanding of the intent of these new processes. My understanding changes every time it's explained to me by the Process/Staff folks -- which probably signals some sort of problem but I can't quite put my finger on what the problem is. Let me try and provide some background on how our WG tried to apply the new process just to throw some new data into the mix.

When we re-chartered the VCWG into "maintenance mode", the WG participants expected people showing up to the maintenance calls to drop by ~50% (maintenance work is horribly unsexy, and the DID WG was where most of the participants wanted to put their limited time -- so we had an exodus of VCWG members to the DIDWG). When this happened, the group was asked "do you want to be able to add new features in REC" and while most of the group wanted to do that, there were a few individuals that did not and thus the group couldn't reach consensus on the "add new features" bit. At the time, I was thinking flexibility was good, and that the AC Review would stop abuse of the new "add new features" option, but two WG members were skeptical that the remaining WG members wouldn't abuse the option. It was a level of paranoia (or misunderstanding of the 202x W3C Process) that was surprising to me, but oh well, that's where the WG ended up. So, the VCWG was rechartered w/o the "add new features" option, but they wanted to be able to fix editorial issues and bugs that required substantive changes. That was in Jan 2020, before the 2020 Process was put in place (but we knew it was coming at the time).

That put us in the position where all we could do was Editorial changes to the REC with substantive bug fixes if necessary. So, fairly constrained charter.

We split the workstreams up into v1.1 (editorial changes only) and v1.2 (substantive changes). We planned to publish v1.1 as we went using the new update-in-place REC process. We planned to publish v1.2 six months before our charter expired. Well, those plans are great and all, but that's not what happened.

We did keep the two work streams separate, but the publication to REC never happened because we were waiting on Process 2020 to be put in place, which took longer than we had hoped, then we tried to understand how we could auto-publish to REC, but that was turned off, and so our Staff contact didn't want to be put in the position of having to manually turn the crank and possibly make a mistake... so we decided to hold off on publishing v1.1 because the manual process of putting the REC into place wasn't worth it to just get editorial changes out. We did that for over a year, until we heard about the 2021 Process... which had some bits (I can't remember which) that would allow us to do something we wanted (maybe it was the proposed correction stuff? -- can't remember)... so we started holding off for Process 2021, which took longer than we had hoped.

So, at this point, v1.1 was jamming up into v1.2 -- we had gone a fully 1.5 years without publishing either to REC -- partially because we were waiting for Process 2021, and partially because it wasn't clear exactly how to put what we wanted into REC-space at that point, and partially because the WG was split about what we wanted to do and kept debating the best path forward from a Process perspective. Add to this the lack of tooling and understanding about how to do proposed corrections (only now, after you explained the above, do I think I understand the intent)... and we are left with six months left on our charter to get anything out the door, which felt ridiculous that we hadn't been able to get anything out at that point.

The group then said "FINE! Forget it, we're not going to do an editorial release, let's just merge v1.1 and v1.2, rename it to v1.1, which will have all the editorial and substantive changes in it, and get it into REC space. That's when I hit the Editorial World of Pain (aside: great album from the 90s, btw) alluded to in this comment.

It was then that I found out that you needed some sort of nebulous markup around your changes, about all the variations of (candidate|proposed) (change|addition|correction) terminology, and Last Call process (which I thought we had gotten rid of, except now it's back!)... just lots of details that were surprising to me (but I know probably made sense because the Process folks had put a lot of time into them). So, it took about 3-4 weeks to do what felt like a random walk with no supporting tooling (and this is with over a decade of W3C spec writing experience under my belt already).

It was only in reading your comments above, @frivoal, that I now understand (I think!?) that we're expected to publish both editorial and proposed corrections in a continuous fashion to REC. The problem is that there is no automated tooling for the Staff to make this easy (perhaps there never will be? sounds dangerous?), and it's not clear at all that we can publish "proposed corrections" to a REC whenever we want. If I had known that, we might have done shorter publication cycles... maybe.

I think what we have above is the simplest case, and our WG thoroughly failed to navigate even that. @marcoscaceres goals of "WHATWG no nonsense publishing" are probably where some of us want to get to, and from what I understand, that requires a CR/PR/REC cycle. I'll note that our WG will probably vote against being able to do CRs while in maintenance mode (again, due to paranoia about someone abusing the process)... and that presumes a re-chartering, which CSS WG is all but guaranteed... but VCWG and DIDWG is absolutely not guaranteed. There is definitely a Process disconnect between the people that work within perma-WGs and the majority of the rest of us that don't (but that's probably a different topic).

So, where do we go from here? I like what @marcoscaceres proposed here. Please keep it simple for the WG and Editors.

[msporny]

@frivoal wrote:

I guess I am a little bit confused: as far as I can tell, the pre-existing path of cycling through a CR+PR appears to do exactly what you wanted: ask for mass approval for a batch of changes documented via changelog.

CR+PR was far more heavy-weight from a Process perspective than what we wanted to do (Amended REC). We had always wanted to just do an Amended REC, it was just way more difficult because of 1) evolving W3C Process document "that's just around the corner", 2) lack of people that understood the new 202x W3C Process, 3) lack of understanding of the sequence of operations, and 4) lack of tooling.

Which features of the new part of process where you interested in?

The Amended REC stuff.

It doesn't appear to be the ability to do changes one at a time (since you batched them all), nor the ability to publish changes before they're all mature enough to go to AC+Patent Review (since you went straight for Proposed changes without going through Candidate changes first for any of them).

We didn't want to do Candidate anything. We wanted to update the REC with editorial fixes and substantive bugfix changes (which we've done now), but it was not clear that we could mix those and publish both the editorial changes and the marked-up proposed changes to REC at any time. I'm still unsure if that's what you're saying?

One of the lessons we learned was to not have two REC spec streams active at any given moment... it just made everything complicated (for WG participants, Staff, and Editors). We're probably just going to keep one spec stream active now (the immediate next point release if we're in maintenance mode, or the next major version if we're in a "new spec/features mode").

[marcoscaceres]

Ok, back on a real keyboard... want to take up a few more points:

The WHATWG model has merits, but what it doesn't get you is the official backing from the whole consortium.

My understanding is that the WHATWG is a consortium in its own right. And the changes do have the official backing from the whole WHATWG consortium so long as the changes made followed the WHATWG Working Mode.

Now, we (W3C) need to fundamentally change our thinking here: if a Recommendation is marked as "updatable" from a Process perspective, and them membership said that was OK during the transition to REC, then it should be possible to update the Recommendation without all the extra work. We should just publish an "PER" without all the purple boxes (or people can include them if they want, but it shouldn't be mandatory... just polite). Then every few months take a snapshot, give the Consortium members/lawyers a chance to review the changes, and republish as a REC. Rinse and repeat.

When giving the consortium a "chance to review" and updated draft, it shouldn't be the Editors' job to provide diffs: If the membership wants to review a diff, they can go to GitHub or use HTML-Diff to view the differences (like they already do with all other drafts). We already have tooling for that (e.g., .pr_preview), so adding the changes to the actual specs is just "busy work" and needless clutter.

Lowering the amount of vetting that the normative text of a REC gets, and thereby lowering the strength of its endorsement was an explicit non-goal for P2020. The goal was making it easier to update a REC, not changing what it means to be REC.

I think this is giving too much credibility to RECs. But no one is changing what a REC means.

If CR are the equivalent of WHATWG "Living Standards" from an IPR perspective, we should rename "CR" to "standard" and keep "REC" as a purely ceremonial designation (i.e., sure, it's nice to have the endorsement from the whole membership, but in practice and once a spec is widely implemented and deployed... does it do much beyond marketing purposes? 🤷‍♂️).

Well, sure, but if you have 2 years worth of changes that you want to push through at once and get AC+Patent review on all the changes at once, that doesn't look very incremental to me.

That's what we do already when we go to REC the first time... in fact, most specs take 5+ years to develop so it's totally fine - AC and Patent review happens and no one complains (or asks for inline purple boxes showing changes).

I wasn't suggesting sticking to CRs forever for that spec, just to evacuate this large batch, and then do the incremental thing after that.

It's unfair for us doing the actual standardization work to get put into this situation: the Process should work for us, not make us do some random song and dance because it can't cope with how specs change in reality over time.

My sense is that this process has been designed to appease reviewers by shifting the "what's changed?" problem to Editors. That should be the reviewers problem to figure out (i.e., by generating a diff, looking at commit history, or seeing a change log).

[fantasai]

@marcoscaceres The entire contents of a Recommendation needs to adhere to the following criteria:

• has the endorsement of the WG
• has passed wide review
• has passed through a Patent Exclusion period
• has the endorsement of the AC
• has demonstrated interoperability in the real world

Organizations outside W3C (ISO, national standards ministries, legislatures, etc.) point to W3C Recommendations expecting this level of vetting for the document they are pointing at.

For Process 2020 the goal was to allow incremental changes, but to not change the requirements on the contents of a Recommendation: the contents of a Recommendation before P2020 and the contents of a Recommendation after P2020 would continue to guarantee the same level of vetting.

That means we can't actually change the normative contents of a Recommendation unless the new content has already fulfilled all of those criteria. That is why you cannot just publish your changes in place and call it a W3C Recommendation.

WHATWG does not require wide review or AC Review, so changes can sit in a PR known only to a handful of browser vendor employees until they pass their (much more limited) requirements for publication into a Living Standard. But W3C requires the contents of a REC to have passed review by the broader community, not just a few browser vendors, and that requires proposed changes to be published where they will be visible to the broadest possible community (ideally, at minimum, all readers of the current spec).

Which is why proposed updates need to be published before being folded in. This can be done by changing the status of the document to something that has fewer requirements and publishing the changes inline; or by annotating them into the REC as (non-normative) inline errata. But it cannot be done by publishing an unreviewed updated REC, because such a REC would violate the criteria for being a REC.

[marcoscaceres]

But it cannot be done by publishing an unreviewed updated REC, because such a REC would violate the criteria for being a REC.

We are in agreement about that, and all of the above. And this is not about diminishing the rigor of the review process or what a W3C REC means.

My recommendation was to publish "PER" -> "REC" (and have people review the "PER" in detail... without the need to do boxes).

[frivoal]

My recommendation was to publish "PER" -> "REC" (and have people review the "PER" in detail... without the need to do boxes).

Assuming you can iterate on the PER depending on the feedback you get, how's that different from publishing a CR?

[marcoscaceres]

Assuming you can iterate on the PER depending on the feedback you get, how's that different from publishing a CR?

Mechanically it's not any different. But for the reasons we just discussed: It's a "W3C Recommendation"™️, and that means something to a lot of people and to industry (i.e., the comparison to "ISO standard" that @fantasai made above).

As an institution, we should be wanting to have as many W3C Recommendations as possible, but also maintaining them as effectively as possible.

(the current process is too painful and will drive people towards perpetual CR or into FPWD->CR->PR->REC loops… what we should be aiming for “updatable” RECs to stay RECs with minimal effort… having to include diffs and purple boxes is an unreasonable request, because it doesn’t match the reality of how specs are updated… no one should need to do that unless they actually want to)

[LJWatson]

The process introduced in 2020 not withstanding, a specification can be in one of six states: ED, FPWD, WD, CR, PR, and Rec. That's a lot of stages even before you add CR Drafts, CR Snapshots, Candidate Editions etc.

I agree with @marcoscaceres that we could do with some fundamental rethinking, so here goes...

Based on the incubation process, when a spec moves into a WG and onto /TR, it could immediately be published as an FPWD, triggering the patent exclusion and wide/horizontal review.

With some WG already auto-publishing their ED straight to WD, the WD could become the "work in progress" version of the spec and we could do away with ED altogether.

We'd need to keep FPWD and CR because they both trigger patent exclusions and wide/horizontal reviews, but could we combine CR and PR into a single stage?

Implementation experience would need to be a requirement to enter CR instead of exit it, but again this is a direction some WG are already moving in.

If the AC was included in the wide review done in preparation of advancement to CR, it would give them an opportunity to raise any issues at a time where they could still be dealt with without generating more admin overhead. As with issues raised during wide and horizontal review already, the WG could need to provide evidence that they'd been dealt with appropriately.

Then, assuming all other transition requirements are met, the WG consents to transition to CR, and the Director (or whatever Director-free solution we arrive at) approves, the CR is published.

This triggers the patent exclusion and the final AC review (though the latter may be delayed to allow the patent exclusion period time to run down first), and assuming they both go OK, the Rec is published.

Probably lots of reasons why this would not work, but...

[frivoal]

a specification can be in one of six states: ED, FPWD, WD, CR, PR, and Rec.

The process does not recognize ED as a state of earlier maturity to FPWD. See at the end of 6.3.1: “Editor’s drafts (ED) have no official standing whatsoever”

the WD could become the "work in progress" version of the spec and we could do away with ED altogether.

That's already the view of the Process. ED is not an earlier maturity, it's just a semi-private scratch pad. Process-wise, we already want to use WD as the work in progress version. It just happens that lots of WGs and Editors are often not taking the time to keep their WD up to date (for tooling or historical reasons), but the Process does expect us to.

could we combine CR and PR into a single stage?

PR is just a transitional stage. It's what we call a CR when the AC's been called to vote on it in order to make it a REC. We could do away with the name, but as long as we keep CR and REC distinct, and that there's supposed to be an AC review in between, there's not much simplification to be had there.

Implementation experience would need to be a requirement to enter CR instead of exit it

That's challenging, for two reasons:

• You get patent protection from CR. People can implement before that if they want to, but requiring implementation before you get patent protection is problematic
• If you require wide review and implementation experience and AC review before you get to CR, you've just made CR the same as REC. Then why even have CR?

If the AC was included in the wide review done in preparation of advancement to CR

Wide review is supposed to be wide. AC review is a formal opportunity to make sure nothing is missed, but feedback from everyone, including the AC, is not only welcome, but should be sought before CR.

This triggers the patent exclusion and the final AC review (though the latter may be delayed to allow the patent exclusion period time to run down first), and assuming they both go OK, the Rec is published.

All in all, I think that your suggestion would indeed simplify the number of states and state transition that we have in the process, but that's at the expense of effectively eliminating everything between WD and REC, without making REC any easier to reach. Sure, it's less Process, but I don't really see how it helps anyone. Effectively, if you want that, you can do it today, by withholding the transition to CR until you've achieved all the CR exit criteria, and then oneshot the CR/PR/REC transition. But that doesn't seem to be that popular a move.

[marcoscaceres]

Thanks @LJWatson and @frivoal for the further points and clarifications.

Maybe we should spin off the discussion around all the different publication statuses to a different issue?

For this issue, I'd like to keep the focus exclusively on amended recommendations and the requirements for actually amending them.

Can we agree to not require the purpose boxes and just do something simple as @msporny wrote in https://github.com/w3c/w3process/issues/589#issue-1060328934 or that I proposed in https://github.com/w3c/w3process/issues/589#issuecomment-976294121?

Namely:

• Just provide a change log to substantive changes and corrections.
• For review purpose, provide an "out of band" link to on a diff document (HTML diff).

The change log can provide links the various pull request.

The change log looks like this: https://www.w3.org/TR/geolocation/#change-log

Happy to improve the above if needed, so long as it's something the GitHub API provides.

[chaals]

@marcoscaceres proposes that we make the changelog requirement a link to github diffs.

These are often very difficult to interpret for someone who isn't very close to the document - they make purple boxes look like a wonderful tool designed to ensure deep and careful review.

It is true that asking for human-readable change logs written by the group is extra work, but

• compared to the work required to justify a change to the group (which in many cases provides text taht can be copy-pasted to form a real changelog entry) it is generally a small amount
• asking reviewers to base their understanding on an unannotated source diff is unrealistic and will lead to fewer in-depth reviews, which is not a goal for W3C.

[marcoscaceres]

@chaals, you are right on both points... the problem is maybe how we are expecting or assuming reviews are done. If diff are too hard to read by reviewers, and adding the purple boxes are too much work for Editors, then we need to think of entirely new way of doing this. I think it's worth thinking about.

[frivoal]

@marcoscaceres I am still confused about the difference between what you're proposing and cycling through CR.

The way I see it, one of three things have to be true

• All changes have been approved (Wide review, AC Review, Patent review… all the usual), and the document can be called a REC
• Some or all changes have not yet been fully approved, and the document cannot be called a REC
• Those changes that haven't been fully approved yet are marked as non-normative tentative changes in a document that is otherwise a REC

If you do not want to distinguish in the spec all the changes that are being proposed for review from the text that has already been approved, then you can take the whole thing to CR and get approval on the whole thing, and call it REC again when you have received that approval.

If you want to include in the spec changes which haven't got approval yet, while continuing to call the whole spec a REC, then readers need to be able, in the spec, to see the difference between what is actually part of the REC and what is merely a yet-to-be approved suggestion of a change. And as some of those changes get approved, you need to be able, in the spec, to see that they're not longer mere suggestions, and are not normative parts of the REC.

We can quibble about the precise way to express that distinction (at I do think that tooling can be created to make it easier to annotate the documents), but short of changing the definition of what a REC is, I don't quite see any other way around. Applying all the changes and pointing at a changelog isn't good enough, because then the text that claims to be the normative part of the REC wouldn't actually be the one that has the full approval.

[marcoscaceres]

If you do not want to distinguish in the spec all the changes that are being proposed

That's an incorrect presupposition: I just want to do it differently. I want to do it in the SoTD, as list of changes that link to pull requests.

for review from the text that has already been approved, then you can take the whole thing to CR and get approval on the whole thing, and call it REC again when you have received that approval.

No. You may be are misunderstanding the problem I'm pointing out: What I understand in what you are saying above is, "if it's too much work to put in the purpose boxes + diffs into your spec, then just do another CR".

Respectfully, that seems wrong: Dropping to CR defeats the purpose of "updatable REC". Why should Editors be put into a position to do another CR round when it's the mechanics being requested that are broken/too much work? I know that in an ideal word Editors would do the right thing and keep a perfect change history, or underlying specs wouldn't change, etc. providing clean simple changes and purple boxes.

But that's just not realistic for most groups/Editors. The reality is that making changes to specs is very messy, even after they have gone to REC.

What I'm proposing is simpler: the status of the document can link to the pull requests and rationale for changes. Optionally, and where it makes sense, purple boxes appear.

If you want to include in the spec changes which haven't got approval yet, while continuing to call the whole spec a REC, then readers need to be able, in the spec, to see the difference between what is actually part of the REC and what is merely a yet-to-be approved suggestion of a change.

We agree. But different means to the same goal.

We can quibble about the precise way to express that distinction (at I do think that tooling can be created to make it easier to annotate the documents),

Ok, PR welcome for ReSpec and Bikeshed? 🙏

but short of changing the definition of what a REC is, I don't quite see any other way around. Applying all the changes and pointing at a changelog isn't good enough, because then the text that claims to be the normative part of the REC wouldn't actually be the one that has the full approval.

I disagree. It will have to be good enough because that's what reviewers should expect to get.

Otherwise, short of someone making all the tooling, we should ditch "updatable REC" as unrealistic because it's too much work and way too error prone.

[jyasskin]

Just a note from someone who reviewed Manu's updated REC without having been involved in the process that produced it:

• This was a small set of just 8 changed paragraphs, with only 3 underlying changes. If it was burdensome, something's wrong. But I'm not sure why this small set of changes was so difficult — maybe just that the changes happened without anticipating the process, and so Manu had to go back and convert them to the new markup? If that's the case, maybe future instances won't be as bad as this one, and so nothing significant in the process needs to change.
• Seeing the PRs and mailing list discussion that produced the changes would have been easier to review than the diffs alone. But it's possible I'm weird for thinking that — too much software engineering background.

[msporny]

• But I'm not sure why this small set of changes was so difficult — maybe just that the changes happened without anticipating the process, and so Manu had to go back and convert them to the new markup? If that's the case, maybe future instances won't be as bad as this one, and so nothing significant in the process needs to change.

It wasn't only that, that was the easy part. As far as my experience w/ this new process was concerned:

• There was no clear guidance on what needed to be produced during the lifetime of the WG and then we found out that we were supposed to be tracking all of this in a very different way than just Github issues, labels, and PRs. We were treating it like any other CR1 -> CR2 transition, where (in the old days) you'd just diff-mark the changes and be done with it... but we were told that we needed to do this new fancy markup for the changes, show a diff/current/future state, explain the change to people that didn't have time to go into the issue tracker or look at a PR, etc... which then puts the Editors in a position where they thinking "Well, use plain 'ol English without technical jargon" except the change you're making is highly technical... and might be the result of months of discussion.
• Turns out there was documentation on what was supposed to be done, but we didn't see that documentation until /after/ we had already done a random walk through snippets of mailing list discussion on what the new process expected... and then there was disagreement on some of the documentation (this issue).
• It would have been easy to just link to the issues and PRs, but that was cited as "not enough" and we had to do this new markup approach that Editors have not had to do before at W3C (AFAIK).
• It was not clear to what extent that we should explain each PROPOSED CORRECTION in the spec.
• There was little to no tooling in ReSpec to make the process easy (automatic numbering of proposed corrections, automatic diffing between old text and new text, preservation of proposed rec anchors), so much of what we depend on ReSpec to do (so humans don't mess it up) ended up becoming a painstaking hand-edited chunk of complex HTML... and we really didn't change much, so I can't imagine what an involved REC update would look like.

• Seeing the PRs and mailing list discussion that produced the changes would have been easier to review than the diffs alone. But it's possible I'm weird for thinking that — too much software engineering background.

We tried suggesting that as an alternative, but the counter-suggestion was that we needed to make it easy to review the proposed corrections... and based on your review comments, we failed to do that. :)

What we could do in the future is link to the issues and PRs that led to the proposed corrections for those that want to really dig into why we made the changes (as those then link to the meeting minutes where there was discussion about each issue). Would that have helped, @jyasskin?

[marcoscaceres]

There was little to no tooling in ReSpec to make the process easy

In our defense, there was ZERO coordination with the ReSpec (or Bikeshed, afaik) when this process was put together (which is really disappointing: it's akin to "we don't write specs without getting implementer support/commitment". This should have gotten input from tool developers much earlier).

We are not in a position to just come up with our own thing/markup/solution - so long as it meets whatever is in the Process, it should be fine.

What we could do in the future is link to the issues and PRs that led to the proposed corrections for those that want to really dig into why we made the changes (as those then link to the meeting minutes where there was discussion about each issue).

That's the plan for ReSpec. Just provide the list of PRs in the SoTD and any other links can be provided manually there too.

[msporny]

In our defense, there was ZERO coordination with the ReSpec (or Bikeshed, afaik) when this process was put together (which is really disappointing: it's akin to "we don't write specs without getting implementer support/commitment". This should have gotten input from tool developers much earlier).

... and to be clear, I'm not faulting ReSpec (or the fine people that maintain ReSpec) for the issue... and I'm certainly not trying to point the finger anywhere else. I'm merely stating -- there was miscommunication, everyone was acting in good faith, and it resulted in a sub-par experience for at least one person that was expected to follow the new process.

Having been through it once, if nothing changes, the next time should be easier... but I'm still worried about having to do anything manual ('cause I have a high degree of certainty that I'll mess it up)... and I'm worried about the other Editors that have not had contact with the problem yet.

That's the plan for ReSpec. Just provide the list of PRs in the SoTD and any other links can be provided manually there too.

That feels workable. The only other remaining bit is "How are we supposed to mark up the PROPOSED CORRECTION sections?" -- feels like there should be diff-marking tooling that works with "the old text" vs. "the new text"... and that might be a bit of a tall ask (and it doesn't seem like everyone is on the same page there).

[plehegar]

A note on this front: @dontcallmedom is working on an alternative approach to identify candidate/proposed changes in ongoing W3C Recommendations. The goal being that it's easier to manage than the current one, from the editors point of view. So, this is ongoing.

Update (March 25, 2022): @dontcallmedom got feedback on an early proposal and keeps iterating.

[plehegar]

[[ florian: Tzviya earlier raised complex areas that are ripe for simplification. This is one of those areas. It's far from trivial to see how we can simplify at a glance; we can't just simplify the grammar or removing half the rules, because they are there for reasons. … we don't want to make it possible to change a REC in ways that could not have originally been published as REC in the first place through the update process.

plh: over the summer, Ralph asked me to put more resources on ReSpec and Bikeshed, which should help in maintaining RECs

plh: it's sometimes difficult to determine whether a change is substantive or editorial. … for example, WCAG 2.1

https://www.w3.org/2023/Process-20230612/#correction-classes tzviya: I think this goes hand-in-hand with issue 700 that Wendy raised on how to determine which of the two paths to use. … [w3c/w3process#700](https://github.com/w3c/w3process/issues/700) … since there's no explicitly different name for the "living standard" version, it's very confusing. … there needs to be some kind of guide for chairs et al to help understand how to choose between the paths. … Wendy wrote some text about this. plh: hopefully we're not suggesting simplifying by writing more documents. tzviya: I'm saying as co-chair of EPUB that the experience of understanding this is... hard. … it's not clear what maintenance vs stable rec means florian: I think I know what you're talking about - the choice between recs that can gain new features or not tzviya: yes. florian: most groups should choose the type that can gain new features tzviya: but you have the knowledge of having written the process. Chairs do not (necessarily) I think we need more flow charts tzviya: there's "the default of what the W3C has done historically" as the preferred process. It would be good to have some textual explanation. plh: #700 feels like a guide issue, not a process issue. florian: possibly … we should still examine and see if we can make it simpler, as I agree it would be desirable, but I expect this to be challenging to simplify. plh: tzviya, can you work with Wendy to get that primed? tzviya: yes. plh: sounds like this should be assigned to myself and Florian. … I think this is still in P2024 issue to look into. ]] https://www.w3.org/2023/09/27-w3process-minutes.html#t04