Burndowns for BCD, MDN, and Caniuse

[ddbeck]

I've created burndown lists for web platform features known elsewhere that aren't yet reflected in web-features. These burndown lists can help us focus our attention on adding the most-interesting features first. Sources for features include:

• mdn/browser-compat-data (BCD) keys (excluding WebDriver and webextensions data)
• BCD keys cited by mdn/content frontmatter
• Can I use…? features (via caniuse-lite)

For more on how these data sources are processed (and how I hope to generate progress reports), you can see my scratchpad repo.

How to view the burndown lists

The burndown lists contain a lot of entries (over 14,000), so you'll probably want to use Filter views.

The BCD and MDN burndown list has the following filter views:

• Baseline 2023+ candidates — BCD keys with Baseline dates possible from 2023 and later (dates are computed, not reviewed)
• Baseline 2016 and later candidates — BCD keys with Baseline dates possible from 2016 and later (dates are computed, not reviewed)
• MDN top 1k — BCD keys cited by the top 1000 visited pages on MDN
• Unresolved Baseline dates — BCD keys where a Baseline date could not be computed (usually because of incomplete data in BCD)
• Done — BCD keys that have been cited by web-features

The caniuse burndown list has the following filter views:

• Shown on caniuse, newest first
• Done — Caniuse IDs that have been cited by web-features

If you think there should be other views, please propose them (or add them and document them here).

How to use the burndown lists

1. Find some feature or features that are interesting (e.g., Baseline 2016 and later candidate).
2. Create feature PRs.
3. When the PRs merge, mark the checkbox for cited in web-features (or wait for me to run an update against the spreadsheet).

[tidoust]

The group discussed visualizations as a burndown chart (see #538). That chart would track the evolution of the number of BCD keys that still need to be reflected in web-features over time. The chart could help track progress, but also promote the work of the group.

Ideally, we'd be able to generate that chart from the lists. Thinking out loud, three possible ways to make that possible:

1. Track, for each BCD key, the date at which it starts to reference a feature through a web-features tag. That date might be interesting to have, but it strikes me as painful to compute and track. A git blame approach seems brittle for example.
2. Generate and store useful indicators (number of BCD keys in total, number of BCD keys that are refelected in web-features) each time the lists are updated. That's easy but means we cannot go back in time to add new indicators.
3. Version and preserve the lists themselves so that we can process previous versions at any time, to compute useful numbers. This would have the advantage of enabling future processing over past data (for instance if we later realize that we would like to track the evolution of a new indicator). Perhaps that's already doable as Google spreadsheets track changes made to the spreadsheet? I'm not sure that's easy to process automatically though. Otherwise, that would mean putting the lists in a Git repository somewhere.

[ddbeck]

Very helpful comment, @tidoust. Some thoughts:

1. Track, for each BCD key, the date at which it starts to reference a feature through a web-features tag. That date might be interesting to have, but it strikes me as painful to compute and track. A git blame approach seems brittle for example.

Right now, we don't actually pull in the keys by BCD tags. We'll have to do that at some point, so I think the answer here is to orient ourselves towards the slightly broader view of "when was the key first cited by web-features". That's relatively straight forward to figure out: you just find which keys first appeared in which release. We don't have to fuss with git blame, just the set difference of keys in each successive web-features JSON file.

(This will necessarily miss renames, but fixing that would be very difficult. I don't think we should worry about it.)

(I've given myself this task over in https://github.com/ddbeck/web-features-burndown-tools/issues/1)

2. Generate and store useful indicators (number of BCD keys in total, number of BCD keys that are refelected in web-features) each time the lists are updated. That's easy but means we cannot go back in time to add new indicators.

We discussed this a little yesterday on the Baseline call. I've got some tooling to figure this out retrospectively (at least, to the point the schema last changed), but if you have any additional indicators to request, the sooner the better. 😄

3. Version and preserve the lists themselves so that we can process previous versions at any time, to compute useful numbers. […] Perhaps that's already doable as Google spreadsheets track changes made to the spreadsheet? I'm not sure that's easy to process automatically though. Otherwise, that would mean putting the lists in a Git repository somewhere.

I think a belts-and-suspenders approach is probably good here. We'll have the Sheets history, but as you say, that's not very easy to process. But there's nothing stopping me from doing the Git approach too. Filed that away here. https://github.com/ddbeck/web-features-burndown-tools/issues/2

[ddbeck]

On the March 7 Baseline call, someone noted that the filter views hide completed entries. I've now added "Done" filter views (and updated the issue description) for each sheet. Now you can see which BCD keys and caniuse IDs have been cited by the project.

[ddbeck]

A sample of a progress report:

	Before (2024-03-21)	After (2024-03-28)	Net	Change (%)
features in web-features	171	189	+18	+10.53%
compat keys mapped to web-features	1361	1483	+122	+8.96%
caniuse IDs in web-features	84	89	+5	+5.95%

Coverage	Before	Before %	After	After %	Change (points)
caniuse IDs	84 of 562	14.95%	89 of 562	15.84%	+0.89%
browser-compat-data keys	1361 of 14366	9.47%	1483 of 14382	10.31%	+0.84%

Edited (2024-04-01): corrected table Edited (2024-03-28): With ✨ charts ✨

[tidoust]

@ddbeck Is the 16 value (and +0.16%) incorrect in the "compat keys mapped to web-features" row? Coz' 1487-1365 = 122. If it's correct, what does it mean?

[ddbeck]

@tidoust good catch! I've fixed the error. Thank you!