web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
5.03k stars 3.12k forks source link

Deflake and speed up LoAF tests #49237

Closed chromium-wpt-export-bot closed 1 week ago

chromium-wpt-export-bot commented 1 week ago

The main cause for flakiness in long-animation-frame tests was the fact that long animation frames can be caused by arbitrary system operations, unrelated to the ones created by the test, and those would generate noise that would make the tests believe that the entries are incorrect.

The previous mitigation was to generate really long LoAFs (360ms), filter only really-long LoAFs, and also retry 10 times. However, this brittle approach had several issues:

The new approach, is to mark the time during calls to busy_wait, and filter only the LoAFs that contain the actual busy_wait call. This filters out any LoAF that we didn't generate ourselves in the test. This also allows us to reduce the busy_wait time from 360ms to 120ms.

Refactored some tests to call the utils busy_wait instead of rolling their own.

Bug: 40263438 Bug: 41484738 Bug: 369956892 Bug: 336691003 Bug: 335003887 Change-Id: Idf3505aef54837f7d7200b0dc57dffdec7c748c6 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6022841 Reviewed-by: Scott Haseley \shaseley@chromium.org Commit-Queue: Noam Rosenthal \nrosenthal@chromium.org Cr-Commit-Position: refs/heads/main@{#1384560}