Retiring tr.rdf - Githubissues

tobie / specref

An open-source, community-maintained database of Web standards & related references.

http://www.specref.org/

Apache License 2.0

165 stars 142 forks source link

Retiring tr.rdf #612

Open xfq opened 4 years ago

xfq commented 4 years ago

Background: https://lists.w3.org/Archives/Public/spec-prod/2020AprJun/0006.html

In the past, the only way to get data related to the specifications was tr.rdf. A few years ago, W3C released the W3C API to provide more complete data. Since W3C is now focusing the effort to improve the API, it would be great if w3c.js can be updated to use the W3C API.

A valid W3C API key[1] is required to query the W3C API, so we need a way to configure the API key somewhere in Specref (for example, it can be an environment variable queried by update-all).

PS: The W3C API is currently a plain REST/JSON API, but in the future it might be migrated to API Platform.

[1] That is, in order to use the API, you need to create a W3C account and a W3C API key.

tobie commented 4 years ago

Thanks. What's your timeline for this and would you have resources to implement those changes in Specref?

tobie commented 3 years ago

I looked at this a bit, and it's a significant amount of work as W3C's API doesn't appear to be designed for this use case. I don't have the resources to get this work done. So this won't happen unless someone steps up to do the work or is ready to pay for the implementation.

If tr.rdf is retired, Specref will just no longer be automatically updated for W3C specs. Other organizations (such as IETF or WHATWG) will continue to be automatically updated. So will WICG specs.

frivoal commented 3 years ago

@xfq In that case, either W3C should not retire tr.rdf until find the time/money to do the work to integrate the new API (which possibly includes improving the new API, given that @tobie says it's a poor fit for the use case).

Specref no longer being updated automatically just doesn't seem like an acceptable outcome.

deniak commented 3 years ago

@tobie @frivoal we'll see if we can have someone update specref to switch to the W3C API instead of tr.rdf. Rest assure we won't retire tr.rdf without that switch. If you know other tools relying on tr.rdf, please let me know.

tobie commented 3 years ago

Things to think about:

Updates are hourly, we risk going above the threshold allowed by the API.
We need to go through all specs every hour to see if they have been updated or if there are new ones.
This means n/20 calls to the API listing specs and then n calls to the API for specs, and again n calls to the API for different versions.
We also need to check aliases, and possibly obsoleted specs.
We need a migration story from the previous solution that preserves the existing specs and versions (possibly with a script that helps us do that).
There might be specific issues with referencing working groups name, I haven’t looked into that yet.

caribouW3 commented 3 years ago

For the record https://www.w3.org/2002/01/tr-automation/ has a number of tools working from tr.rdf input, so we should probably keep it, at least until we have a proper replacement for each of those.

tidoust commented 1 month ago

PR #828 would make Specref use the W3C API instead of tr.rdf. See the PR description for details on the logic implemented to address the points raised in https://github.com/tobie/specref/issues/612#issuecomment-750114550.