search

catapult-project / catapult

Deprecated Catapult GitHub. Please instead use http://crbug.com "Speed>Benchmarks" component for bugs and https://chromium.googlesource.com/catapult for downloading and editing source code..
https://chromium.googlesource.com/catapult
BSD 3-Clause "New" or "Revised" License
1.92k stars 564 forks source link

[smoothness] Reported frame_times regression may be actually progression and vice versa #4426

Open wangxianzhu opened 6 years ago

wangxianzhu commented 6 years ago

crbug.com/822865 is an example. The measured frame_times dropped because a big frame time is measured for a period of idleness during an animation. Perhaps we should change how we measure frame_time either to exclude the idle period or measure the idle period as 60fps.

It might not be a big issue when a progression is reported as a regression, but it's an issue in the reverse direction because we may miss some important real regressions.

wangxianzhu commented 6 years ago

See traces in https://pinpoint-dot-chromeperf.appspot.com/job/11c14a35c40000, story http___jsfiddle_net_bNp2h_3 for details.