Adding more specs and a UI

[marcoscaceres]

I'm wondering what we need to do to broaden the number of specs in the dataset? I'd like to make sure ReSpec is spitting out the right markup so that spec's dfns can end up in the data.

Additionally, we've been playing around with the idea of providing a web app that would allow editors to look up what terms are available in specs (this is just a rough prototype):

https://specxref-beta.herokuapp.com/xrefs

But it would be nice to be able to go to something like the above and query:

• What specs are in the data set?
• In a spec, what terms are defined (and dfn type).
• What specs define a term (e.g. "string" is ambiguous - defined in like 5 specs).

cc @sidvishnoi

[marcoscaceres]

We have the API to query the data already implemented, so #1 would really help. But would be great to discuss what features would be useful in a UI.

[tabatkins]

So the biggest limiter here is that the underlying tool gathering the data is Shepherd, from @plinss, and its database of specs is tracked via a SQL db that has to be manually maintained.

I'd love to have a simple JSON structure describing the specs being spidered, which could be PR'd/etc, but I don't have easy access to Shepherd's code to make that happen, and @plinss has limited time to work on this sort of thing.

---

Additionally, we've been playing around with the idea of providing a web app that would allow editors to look up what terms are available in specs (this is just a rough prototype):

Note Bikeshed's existing tooling for doing this, in the bikeshed refs command. I've got a few useful variations beyond what y'all are exposing currently. It also uses Bikeshed's actual logic for matching autolinks, which I don't know if y'all are reproducing exactly.

[plinss]

The Shepherd specification module code is all here: http://hg.csswg.org/dev/specification/file/tip

I'd be happy to give you push access or set up a mirror on GitHub if you want to add code to import/sync spec data from a json file (and help deploy the code on the csswg.org server).

There's PHP code to read/write spec data in: http://hg.csswg.org/dev/specification/file/tip/Specification.php and Python 2 code to read spec data in: http://hg.csswg.org/dev/specification/file/tip/python/specification/specifications.py (haven't had a need to write to the specification table in python yet, just anchor data, but wouldn't be hard to add).

[plinss]

It also probably wouldn't be hard to add code to allow writing to the specification table in Shepherd's API: http://hg.csswg.org/dev/specification/file/tip/pages/api/SpecificationAPI.php then you can update from whatever (so long as you can authenticate).

[marcoscaceres]

Having the stuff on GitHub would be a great first step. At least would give all this more visibility about how it works.

[marcoscaceres]

Closing as we now have http://respec.org/xref/ and the data is coming from https://github.com/w3c/webref/