search

web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
5.02k stars 3.12k forks source link

wpt run should be able to run only tests in Interop #48603

Open gsnedders opened 1 month ago

gsnedders commented 1 month ago

We should add a number of command line arguments to wpt run to make it possible to select the subset of tests in Interop.

There's a few potential variants here:

The data is basically split between https://github.com/web-platform-tests/results-analysis/blob/main/interop-scoring/category-data.json and https://github.com/web-platform-tests/wpt.fyi/blob/main/webapp/static/interop-data.json, and we can get all the labels from https://wpt.fyi/api/metadata?includeTestLevel=true&product=chrome rather than manually re-parsing all of https://github.com/web-platform-tests/wpt-metadata.

(Arguably this can already be done by writing a script to enumerate all the tests in one of those variants and put them in a file and then just run with --include-file, but it would be easier to just have this all integrated into wpt run.)

WeizhongX commented 3 weeks ago

I would prefer to check in the list of tests from interop then use --include-file. I think Wptrunner's command line needs some cleanup, instead of adding more, :).