search

ErichDonGubler / moz-webgpu-cts

An extremely fast (but opinionated) tool for working with WPT metadata while developing an implementation of WebGPU in a web browser.
Apache License 2.0
0 stars 2 forks source link

`update-backlog`: triage not-not `PASS`ing tests #109

Open ErichDonGubler opened 1 month ago

ErichDonGubler commented 1 month ago

There are quite a few tests that expect PASS, TIMEOUT, and NOTRUN outcomes (or some subset) at the test and subtest level, but not FAIL or CRASH outcomes. The key observation here is we haven't actually observed these to not PASS; I'll call these "not-not PASSing tests". My hypothesis is that many of these will simply be PASSing when we run those tests in tier 2's broader timeout window. We should triage not-not PASSing tests with a new heuristic for update-backlog.

ErichDonGubler commented 1 month ago

Estimate: S (~1.5 days; ~0.5 days to implement, and 1 to triage).

ErichDonGubler commented 1 month ago

Expected to yield at least a dozen promoted tests.