Closed foolip closed 7 months ago
I agree that it would be valuable information to have.
Of course, the question becomes: to avoid having to maintain the information manually, can we compute it automatically? Some ways to do that, on top of my head:
auto-publish.yml
file. But then that's just a naming convention. Some groups use other job names (e.g. WebGPU/WGSL). Or use their own logic (e.g., WebAssembly). Also, there are cases where the spec-prod
action is used to deploy the spec automatically to GitHub Pages, but not to /TR (as done for WebTransport, WebAuthn, or MathML Core), which could create false positives.revision
that appears in a <meta>
tag in the /TR version with the latest repo commit. Now, that's not enough in itself to claim that the spec is published automatically to /TR. Also, commits may have touched other parts of the repo and not lead to a new published version of the spec. Or publication may still be pending when we check specs. Or publication may have failed temporarily, as happens from time to time.<meta>
tag to the spec published on /TR to make it explicit that the spec was published automatically. That would not capture cases where a group decided to stop automatic publications, but that should remain fairly exceptional.Now, it's not clear to me that "most don't" is still a valid statement: a growing number of W3C working groups have switched to automatic publication in practice (including Device and Sensors, Immersive Web, Media, Pointer Events, Second Screen, WebApps, WebAppSec, WebAssembly, Web Editing). The main exception is going to be the CSS WG.
Perhaps this could be a good opportunity to assess whether linking to the Editor's Draft is still needed, and to take an opposite approach: instead of identifying which specs can be referenced using the /TR version, identify those that should not.
I'll try to create a first draft split of /TR specs (auto-published, non auto-published, don't know) to inform the discussion.
My claim of "most don't" wasn't researched, just based on my hunch and past experience. It's great if most groups are autopublishing now, I didn't know it was such a success 😄
If Echidna is the system that has to be involved for autopublishing, then that seems like the place to get this information.
I have to say that flipping the default makes me a bit nervous. What happens when a group starts working a new version or level of spec, do they always do it in-place or does it sometimes move to a github.io URL while /TR/ goes stale?
there remains at least a clear situation where TR can't keep up with the editors draft: once a spec reaches PR/Rec; even if the Process now allows for easier updates to Rec once published, that isn't a seamless or automated process yet.
Oh, that's very good to know. In that case I'm quite hesitant to use autopublished /TR/ URLs in places like MDN, if they will one day become snapshots that don't reflect the latest work.
Discussing with @dontcallmedom, it seems clear that "is the spec autopublished?" is not a very useful question to answer to determine which URL to use. That status changes over time depending on where the spec is on the Recommendation track.
It may be better to know whether the /TR version of a spec is up-to-date or outdated. This information is not available in browser-specs, but it's available in Webref. See below for an analysis. As with an hypothetical autopublished
flag, that information is not going to be useful to take a decision: it may change over time. In some cases, that's just temporary (WebGPU is a couple of days late due to some publication hiccup last time the ED was updated). In other cases, because the group changes its workflow, or because the spec reaches a status where updates are rarer but not automatable.
Anyway, looking at Webref data, I see 313 specs published to /TR and tracked in browser-specs (112 from the CSS WG, 201 from other WGs):
This confirms that, CSS excluded, a majority of the /TR specs tracked in browser-specs are now up-to-date with their ED. It also confirms that /TR versions of some CSS specs are very old:
thanks for collecting that data, that's already quite enlightening!
Just noted a couple of random bugs I spotted, in case they're the sign of another issue (in the data or in the analysis):
I put the script I used in a gist. Note there are a couple of additional cases that I excluded (the CSS 2.1 spec and WOFF) as the dates in the crawl data are wrong (they're not even dates).
it shows the AV1 spec as a (up-to-date) TR spec (which it isn't)
Yes, the code was selecting specs based on the presence of release
, which is also set for the AV1 spec. I adjusted the code to only consider W3C specs. That was the only spec with that issue.
it lists WebNN as 2 days out of date, which it doesn't seem to be (maybe linked to the different frequency in TR data collection?)
Soooo picky! ;) That's indeed due to the different frequency for data collection: the /TR version of WebNN was last crawled 12 hours ago, when it was still the 25 March 2024 CRD. The new version is still not in Webref. I guess that illustrates how easy it is to create wrong signals: there's always a slight delay between the publication of the ED and publication of the /TR version, and crawl introduces another delay. In practice, anything that is less than one week late can probably be considered up-to-date!
I think the conclusion is that auto-published is not sufficiently stable of a state to be usefully trackable and publishable in browser-specs, so closing this issue.
Some W3C specs are published to /TR/ for every commit, while most don't and require github.io URLs to lead the latest.
In https://github.com/mdn/browser-compat-data/pull/22694 I learned of one such case – WebNN – and it looks like https://github.com/webmachinelearning/webnn/blob/main/.github/workflows/auto-publish.yml is how that works.
I would be great if browser-specs could provide the information needed for other projects like BCD or web-features to know when a /TR/ is autopublished and not. This would avoid the need to keep track of which WGs prefer these URLs, and can lint for most up-to-date and preferred spec URLs more easily.
One possible implementation is to use /TR/ as nightly URL if /TR/ is autopublished, and put the github.io URLs in as alternatives, not the canonical.