web-platform-tests / wpt

Test suites for Web platform specs — including WHATWG, W3C, and others
https://web-platform-tests.org/
Other
5k stars 3.1k forks source link

Some User Timing tests are flaky #14434

Open foolip opened 5 years ago

foolip commented 5 years ago

https://github.com/web-platform-tests/wpt/pull/14409#issuecomment-445753408 lists a number of tests in user-timing/ that were flaky in Firefox:

Test Subtest Results Messages
/user-timing/measure_associated_with_navigation_timing.html Second measure of current mark to navigationStart should be negative value. FAIL: 1/10, PASS: 9/10 assert_true: Second measure of current mark to navigationStart should be negative value. expected true got false
/user-timing/measure_associated_with_navigation_timing.html Measure from domComplete event to most recent mark "a" should have longer duration. FAIL: 1/10, PASS: 9/10 assert_true: Measure from domComplete event to most recent mark "a" should have longer duration. expected true got false
/user-timing/measure_associated_with_navigation_timing.html Measure from most recent mark to navigationStart should have longer duration. FAIL: 1/10, PASS: 9/10 assert_true: Measure from most recent mark to navigationStart should have longer duration. expected true got false

Suggested method to investigate:

  1. Add a trailing whitespace or other no-op to all tests in user-timing/ in a test PR
  2. Check which were flaky on Taskcluster
  3. Fix them :)
foolip commented 5 years ago

Notifying suggested reviewers for user-timing: @plehegar @igrigorik @toddreifsteck