Proposal: BCD dashboard and query language

[foolip]

Summary: one giant compat table that can be filtered with a simple query language

BCD is treasure trove of browser compatibility data, over 11k entries, but it's difficult to understand high level trends or to filter it to specific interesting subsets. The current best and most canonical way of visualizing it is the rendering on MDN pages, e.g., the big table for HTMLElement.

Being able to render all of BCD as a single giant compat table that can be sorted and filtered would enable the following use cases:

• As proposed by @gsnedders in https://github.com/mdn/browser-compat-data/issues/10533, diffing two browser releases which should largely match, such as the desktop and mobile versions of a browser, or two Chromium-based browsers.
• Looking at all entries matching "fullscreen" while reviewing a PR touching one of the entries, in order to see if the proposed change is consistent with existing data.
• List any null or true data, to help visualize https://github.com/mdn/browser-compat-data/issues/3555 + https://github.com/mdn/browser-compat-data/issues/6369
• List any experimental entries that have been shipped for more than two years, similar to https://github.com/mdn/browser-compat-data/issues/10961
• More generally, filter to "burndown lists" for improving data quality

Are there more important use cases? Please comment!

Prior art:

• https://github.com/connorshea/mdn-compat-data-explorer (archived)
• https://caniuse.com/ (includes BCD)
• https://web-confluence.appspot.com/ (not based on BCD, see methodolgy)
• http://kangax.github.io/compat-table/es6/ (JavaScript features)
• https://wpt.fyi/ (platform-tests test results)

P.S. One thing I'd consider important is that the same queries should be possible to run on the command line, maybe as part of BCD itself, so that burndown lists don't just live on a website frontend but could also be consumed by scripts, etc.

[foolip]

In https://github.com/foolip/browser-compat-data/pull/10 I wrote some code to take screenshots of the BCD tables on all MDN pages, and stitched them together. The result is a 720✗819060 image. I downscaled it to 10% horizontally and 0.1% vertically to produce something that looks like a minimap of BCD:

A fun visualization IMHO. Seeing this get more green (and longer) over time would be great :)

[Elchi3]

Prior art for querying / BCD API: https://github.com/Xelzs/mdn-browser-compat-api

[foolip]

More prior art for the query syntax: https://github.com/GoogleChromeLabs/confluence#querying-the-catalog

[foolip]

To give a few examples of what a query language for this could look like:

• chrome:91 chrome_android:false or chrome:91 -chrome_android:91 to find things in Chrome but not in Chrome for Android
• chrome:version_removed:<76 edge:version_removed:<79 firefox:version_removed:<69 safari:removed:<13 to list features that weren't in these two-year old browser releases and thus can be removed from BCD (sort of terrible?)
• version_added:null for any feature with null data (also terrible?)

It's harder than I thought to come up with something that looks OK :)

[Elchi3]

Prior art: https://github.com/browserslist/browserslist#full-list

[foolip]

@bkardell, @PaulKinlan and I had a chat about https://the-web-of.glitch.me/ and our conversation earlier this week. On "where to start", Paul suggests starting from questions we want to answer, or use cases, which makes a lot of sense to me.

So, for anyone on this thread, if you could post specific question you'd like to get answered, that would be great.

[gsnedders]

So, for anyone on this thread, if you could post specific question you'd like to get answered, that would be great.

• Differences between aligned releases (per OP)
• Features not in one browser but in two others (but including beta/preview, unlike the-web-of)

[foolip]

Here are my use cases, including/restating ones in the issue description:

• Data quality checking: compare data for browser releases which should largely match
• Data quality checking: show all experimental or deprecated features, skimming for ones that shouldn't be
• Reviewing: show all the compat data for a feature such as Fullscreen API, to see if it matches a PR in review
• Curiosity: show all the compat data for a particular point in time
• Metrics idea: count/list all APIs that are in all browsers in the past N years

[foolip]

Features not in one browser but in two others (but including beta/preview, unlike the-web-of)

+1 to this one. @gsnedders I guess you're looking for stuff that will soon be a single-engine omission, so future releases are interesting but stuff behind flags is less so?

I might add that I would generally like the ability to inspect BCD with flag data deleted, since stuff behind flags doesn't impact web developers yet.

[gsnedders]

+1 to this one. @gsnedders I guess you're looking for stuff that will soon be a single-engine omission, so future releases are interesting but stuff behind flags is less so?

With/without flags is both interesting. Both tend to be indicative to some degree.