Currently, if one is running a WebGPU CTS test such that there's not
a deterministic place it will time out (like the tests with thousands of
tiny subtest cases that simply don't fit in the time budget wptrunner
gives them), it can take dozens to hundreds of runs to empirically
observe every single potential point of concretely timing out. This is
a problem when the goal is to converge expected outcomes in as few runs
as possible!
Resolve this as follows: If over the course of processing we encounter
any reconciled subtest outcomes that contain a TIMEOUT or NOTRUN
value, ensure bothTIMEOUT and NOTRUN are present in the
reconciled subtest outcomes.
Currently, if one is running a WebGPU CTS test such that there's not a deterministic place it will time out (like the tests with thousands of tiny subtest cases that simply don't fit in the time budget
wptrunnergives them), it can take dozens to hundreds of runs to empirically observe every single potential point of concretely timing out. This is a problem when the goal is to converge expected outcomes in as few runs as possible!Resolve this as follows: If over the course of processing we encounter any reconciled subtest outcomes that contain a
TIMEOUTorNOTRUNvalue, ensure bothTIMEOUTandNOTRUNare present in the reconciled subtest outcomes.