search

catapult-project / catapult

Deprecated Catapult GitHub. Please instead use http://crbug.com "Speed>Benchmarks" component for bugs and https://chromium.googlesource.com/catapult for downloading and editing source code..
https://chromium.googlesource.com/catapult
BSD 3-Clause "New" or "Revised" License
1.91k stars 562 forks source link

ChromeOS telemetry perf test results are 80% empty #4614

Open quisquous opened 5 years ago

quisquous commented 5 years ago

(This is similar to #4613.)

I'm trying to run telemetry perf tests on a ChromeOS device via a Linux command line like this: tools/perf/run_benchmark run rendering.desktop --story-tag-filter=repaint_desktop --browser=cros-chrome-guest --remote=coral --use-live-sites

However, the major issue is that 80% of the results in the results.html are (empty), even though they appear to run visually on the device. Telemetry also appears to save what appears to be a valid trace for those stories with (empty) results in the tools/perf/ directory.

quisquous commented 5 years ago

When I run this locally, I get:

[  PASSED  ] 9 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ]  rendering.desktop/repaint_amazon_2018

The amazon test failure is due to #4613.

Here's the results from running this (all but cnn and reddit are empty): results.zip

Here's a trace from one site that had a result and one site that didn't: repaint_reddit_2018_2019-01-07_15-34-50_45564.zip repaint_google_search_2018_2019-01-07_15-33-40_61866.zip

benshayden commented 5 years ago

@CalebRouleau @achuith PTAL?