search

catapult-project / catapult

Deprecated Catapult GitHub. Please instead use http://crbug.com "Speed>Benchmarks" component for bugs and https://chromium.googlesource.com/catapult for downloading and editing source code..
https://chromium.googlesource.com/catapult
BSD 3-Clause "New" or "Revised" License
1.91k stars 563 forks source link

pinpoint: results OOMs for rendering.mobile #4571

Open sadrulhc opened 5 years ago

sadrulhc commented 5 years ago

If the full rendering.mobile benchmark is run, the results page runs out of memory, and the iframe containing the results gets killed. From chrome's task-viewer, it looks like the memory usage gradually keeps increasing, and it gets killed when it reaches ~2GB. Before being killed, the results page is in 'Parsing HTML...' step.

Sample run: https://pinpoint-dot-chromeperf.appspot.com/job/13c35246640000

@nedn @benshayden

nedn commented 5 years ago

@simonhatch

nedn commented 5 years ago

My suggestion for now is trying to distill a subset of stories that are most valuable in term of catching regressions. Then users can only select those subset to avoid running all 500 rendering stories

@perezju has tried this in https://chromium-review.googlesource.com/c/chromium/src/+/1177396 and can help with the analysis tooling

simonhatch commented 5 years ago

I doubt results2 was ever meant for that scale, @benshayden would have ideas on what might be doable in the short term but I'd follow @nedn suggestion and just run the stories you need. If being able to view enormous results files like this is a highly requested feature, we can prioritize it appropriately in a coming Q.

sadrulhc commented 5 years ago

Would it be possible to make it easier to specify tags when triggering pinpoint jobs? e.g. perhaps we could include that in the benchmark name? rendering.mobile.key_silk etc?

nedn commented 5 years ago

@dave-2 for tags support in pinpoint

perezju commented 5 years ago

Another alternative would be to allow pinpoint users to download the results data (either as histograms.json or some plain csv) to process locally themselves (e.g. on colabs or what not). Agree results.html is not really ideal for exploring such big data sets, so maybe not attempt to display it at all.

sadrulhc commented 5 years ago

Here is a rendering.desktop run from end of June: https://pinpoint-dot-chromeperf.appspot.com/job/137927c0a40000 The results load OK.

Here is a rendering.desktop run from mid-August (yesterday): https://pinpoint-dot-chromeperf.appspot.com/job/13c35246640000 The results OOMs.

I don't believe the pagesets have grown significantly enough during this time. @amyqiu Can you confirm?

amyqiu commented 5 years ago

Yes, the size of rendering.desktop should not have changed much since end of June.

amyqiu commented 5 years ago

I'm not sure if this helps, but I ran rendering.desktop yesterday and the results loaded okay for me: https://pinpoint-dot-chromeperf.appspot.com/job/128ce47a640000

dave-2 commented 5 years ago

Filed issue 874940 for tags support.

chiniforooshan commented 5 years ago

Doesn't adding new metrics (more data per test) change the size of results.html significantly? If it does, then we have actually added new metrics since the end of June.

sadrulhc commented 5 years ago

Oh good point, yes! We did add a few new metrics recently. Perhaps we can sunset some of the thread_times metrics?

chiniforooshan commented 5 years ago

I agree; we should start retiring legacy smoothness and thread_times metrics. I'll assign this to myself, for now.

simonhatch commented 5 years ago

For reference:

https://pinpoint-dot-chromeperf.appspot.com/job/137927c0a40000 -> 305 mb

https://pinpoint-dot-chromeperf.appspot.com/job/13c35246640000 -> 640 mb