Publicly documenting authoritative sources for bib data

[ajeanmahoney]

Description

It would be good to capture somewhere publicly where the bib data comes from. Somewhere, relaton-data-rfcs, relaton-data-ieee, etc., should be explained--maybe a link off the About page, or maybe there's developer's documentation in GitHub that can expanded.

For instance, it has been mentioned in issue discussions that the authoritative data for RFCs comes from rfc-index.xml provided by rfc-editor.org, but I can't find that documented in the code base or any readmes or About pages...

This file is basically an API and it should be tracked. The RPC will be making changes to their database in the next few years, and such existing APIs should be identified and managed.

Thanks!

Code of Conduct

• [X] I agree to follow the IETF's Code of Conduct

[ajeanmahoney]

I may have made an assumption about how the bibmxl-service pulls RFC bib data. Thus documentation of where the data comes from is even more important.

[rjsparks]

Digging through older tickets, I relocated this: https://github.com/ietf-tools/bibxml-service/wiki/Data-sources. Is the table still accurate? The discussion of sync from tools.ietf.org needs to be updated.

[ronaldtse]

@rjsparks yes that page is accurate.

For W3C, we now have an extra repo that contains historic W3C documents since 2002 from the same endpoint (https://github.com/relaton/w3c-tr-archive). Would you be able to help update that row? Thanks!

[rjsparks]

I note also that links in the documentation have become broken. https://bib.ietf.org/static/docs/topics/architecture.html has a link to https://github.com/ietf-ribose/bibxml-service/wiki/Relaton-sources. (which redirects to ietf-tools, but then finds no wiki page - I suspect it was intended to point to Data-sources instead)?

[rjsparks]

Would you be able to help update that row? Thanks!

Sure - but we should make it so that you could too. Anton and Vladamir already have the ability to do so, and we'll make sure you do to (I want to confirm with Kesara when his timezone comes online how he's started this config before adding changes).

[ronaldtse]

Thanks @rjsparks , will wait for @kesara

[kesara]

bibxml-service wiki is configured so that it's only editable by people who have push (write) access to this repository. At the moment Anton and Vladamir have that access. These settings were inherited from ietf-ribose.

[ronaldtse]

Thanks @kesara , I confirm to have received access to this repo. I believe when the repository was transferred, the permissions given to "Teams" originally assigned were removed, but permissions for individuals remained.

[strogonoff]

I see the link in documentation was fixed to point to https://github.com/ietf-tools/bibxml-service/wiki/Data-sources thanks to Ronald. Combining docs and wiki, is documentation lacking?

[rjsparks]

https://github.com/ietf-tools/bibxml-service/wiki/Data-sources#xml2rfc-mirror-archive is stale.

Would it be hard to add a column to the table pointing to the code that ingests each source? (Labelled "as of version nnn"). Or if not a direct link to the code, at least a sentence that would help someone responsible for one of these sources find the right code to see what the impact of a change to the source will have?

[ajeanmahoney]

It would also be nice to capture how often data is fetched from each data source.

[ronaldtse]

@rjsparks @ajeanmahoney I've fixed the stale link, added links to the sources of the software, and added a frequency column.

[ronaldtse]

@ajeanmahoney @rjsparks is the documentation now sufficient to close this issue? Thanks.

[rjsparks]

@ronaldtse : Close.

There are still some /ietf-ribose/ repositories called out that have moved already to /ietf-tools/, and we need to address https://github.com/ietf-ribose/bibxml-data-misc which has not moved yet?

[kesara]

bibxml-data-misc repository has been transferred to ietf-tools: https://github.com/ietf-tools/bibxml-data-misc