search

web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
4.89k stars 3.06k forks source link

[eme] Harden tests and utils against test bugs #3764

Open ddorwin opened 7 years ago

ddorwin commented 7 years ago

As demonstrated by #3763, it is easy for a simple bug or regression in the tests to cause tests to follow the wrong path and/or hide real failures. As it is, we cannot trust the results.

To help prevent this, we should be more careful to check inputs, "use strict", and perhaps refactor some tests or utils.

As an example, in the case of #3763, isInitDataTypeSupported() should check that its parameters are valid strings. Better yet, it should check that they are one of the known Key Systems and Initialization Data Types, respectively. (We can add helper isKnownFoo() functions then assert_true(isKnownFoo(foo)) in each relevant function.)

/cc @mwatson2 @jdsmith3000 @paulbrucecotton

jdsmith3000 commented 7 years ago

I agree with making this VNext.