search

catapult-project / catapult

Deprecated Catapult GitHub. Please instead use http://crbug.com "Speed>Benchmarks" component for bugs and https://chromium.googlesource.com/catapult for downloading and editing source code..
https://chromium.googlesource.com/catapult
BSD 3-Clause "New" or "Revised" License
1.91k stars 562 forks source link

[traceviewer] Test failures when `--enable-skia-benchmarking` set #4640

Open hsheth2 opened 4 years ago

hsheth2 commented 4 years ago

When Chrome or Chromium is started with the --enable-skia-benchmarking flag set, two of the tests fail.

tracing.ui.extras.chrome.cc.picture_ops_list_view_test.instantiate  failed (1.04ms) 
TypeError: Cannot read property 'objects' of undefined
    at TestCase.testFn_ (http://127.0.0.1:8003/tracing/ui/extras/chrome/cc/picture_ops_list_view_test.html:28:24)
    at TestCase.run (http://127.0.0.1:8003/tracing/base/unittest/test_case.html:84:19)
    at TestRunner.<anonymous> (http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:235:48)
    at new Promise (<anonymous>)
    at TestRunner.runCurrentTestCase_ (http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:232:14)
    at TestRunner.runOneTestCase_ (http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:180:12)
    at http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:137:43

tracing.ui.extras.chrome.cc.picture_ops_list_view_test.selection  failed (0.67ms) 
TypeError: Cannot read property 'objects' of undefined
    at TestCase.testFn_ (http://127.0.0.1:8003/tracing/ui/extras/chrome/cc/picture_ops_list_view_test.html:42:24)
    at TestCase.run (http://127.0.0.1:8003/tracing/base/unittest/test_case.html:84:19)
    at TestRunner.<anonymous> (http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:235:48)
    at new Promise (<anonymous>)
    at TestRunner.runCurrentTestCase_ (http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:232:14)
    at TestRunner.runOneTestCase_ (http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:180:12)
    at http://127.0.0.1:8003/tracing/base/unittest/test_runner.html:137:43

My suspicion is that the tests effectively are not run unless --enable-skia-benchmarking is set, due to an early check and return: https://github.com/catapult-project/catapult/blob/53913cecb11a3ef993f6496b9110964e2e2aeec3/tracing/tracing/ui/extras/chrome/cc/picture_ops_list_view_test.html#L23

There is also be an issue in the picture_test. It appears that the undefined check is not enough to distinguish between the getOps being unavailable and incorrectly returning a falsy value. If we replaced that if statement with if (!tr.e.cc.PictureSnapshot.CanGetOps()) return;, then that test also fails when --enable-skia-benchmarking is set. https://github.com/catapult-project/catapult/blob/53913cecb11a3ef993f6496b9110964e2e2aeec3/tracing/tracing/extras/chrome/cc/picture_test.html#L40

Tested in Chrome Version 75.0.3770.142 (Official Build) (64-bit) and Chromium Version 75.0.3770.0 (Developer Build) custom (64-bit).