Closed nt1m closed 8 months ago
cc @past @DanielRyanSmith @jgraham Do we know why this is getting computed wrong?
It looks like it's wrong in the interop view: the 8 tests that end in ?av1
show up as "0 of 1 subtests" in the interop view but are shown as green and it seems like no points are deducted. (Or maybe something is trying to exclude those tests but doing it strangely?)
The dashboard is handling PRECONDITION_FAILED
incorrectly (counting it as passing all the tests).
As previously discussed it doesn't make sense to have tests in Interop that pass in the absence of interoperability, so if Safari isn't going to implement AV1 support these tests should be removed.
Ah :( This same issue again. I thought we fixed the AV1 tests, but we only did so in STP apparently. I'll file a wpt-metadata PR to get the AV1 tests removed.
I'm not sure what's going on but https://wpt.fyi/interop-2023?stable shows "87.3%" even though clicking on the focus area link shows 98%: https://wpt.fyi/results/webcodecs?product=chrome&product=firefox&product=safari&aligned=&view=interop&q=label%3Ainterop-2023-webcodecs&label=master&label=stable
I don't really get where this number comes from.