search

catapult-project / catapult

Deprecated Catapult GitHub. Please instead use http://crbug.com "Speed>Benchmarks" component for bugs and https://chromium.googlesource.com/catapult for downloading and editing source code..
https://chromium.googlesource.com/catapult
BSD 3-Clause "New" or "Revised" License
1.93k stars 564 forks source link

Trace viewer needs more perf tests #3434

Open benshayden opened 7 years ago

benshayden commented 7 years ago

What operations specifically have regressed? Does the perf test harness need any new features?

@natduca @lpy

nedn commented 7 years ago

@zeptonaut already added tracing import as a metric: https://chromeperf.appspot.com/report?sid=acf61875e7af890a421954ed1d301bf395252938d5b2d70ab7104558d3206d5d

zeptonaut commented 7 years ago

Note that the metric is currently unmonitored, which almost certainly isn't good. There should maybe be a tracing-alerts@ alias, and tracing perf sheriff rotation that sends alerts to that alias which is monitored by the tracing owners?

benshayden commented 7 years ago

I think that the perf tests that we were discussing were also about interacting with the timeline-view, not just import. Zoom/pan, expanding rows, selecting events, etc.

nedn commented 7 years ago

Writing automatic perf test & make them stable is not easy. In fact, benchmarks are usually very expensive to maintain. Can you just manually trace the trace-viewer to fix the perf issues as they pop up? How often do we have perf regressions with those?

zeptonaut commented 7 years ago

I think I'm with Ned - it seems like, given the same amount of time, we'd be much more effective in improving trace viewer performance by using that time to fix performance problems as opposed to writing performance tests. I'm all for writing performance tests when we find common problems that can easily be tested, but we know how hard it can be to set up a real performance testing infrastructure for a complicated product...

anniesullie commented 7 years ago

I wonder if there's a quick manual spot check we could do once in a while, or ask people to do for large changes?