search

flutter / devtools

Performance tools for Flutter
https://flutter.dev/docs/development/tools/devtools/
BSD 3-Clause "New" or "Revised" License
1.58k stars 326 forks source link

What makes a frame janky? #451

Closed kenzieschmoll closed 3 days ago

kenzieschmoll commented 5 years ago

Starting an issue to track this discussion: what makes a frame janky?

Right now, we consider a frame to be janky when the sum of UI and GPU durations exceed the target time limit per frame (currently hard coded to ~16ms). We do not give each thread a 16ms threshold because we are also concerned with latency.

Question: To determine whether a frame is janky, should we A) add the durations of UI work and GPU work (current implementation), or B) use the duration of the frame?

Case 1: When both the sum of durations and frame duration meet the target, it does not matter which way we determine jank. Screen Shot 2019-03-20 at 2 44 37 PM

Case 2 (problem case): The sum of durations exceeds the target, but the frame duration does not. Screen Shot 2019-03-20 at 2 44 46 PM

Case 3 (problem case): The frame duration exceeds the target, but the sum of durations does not. Screen Shot 2019-03-20 at 2 44 53 PM

kenzieschmoll commented 5 years ago

@chinmaygarde @liyuqian @Hixie

liyuqian commented 5 years ago

Good question. I think a clear definition would help us and our developers a lot.

My personal understanding is that "jank" is the opposite of "smooth". Latency is another issue. It's kind like when you buy a gaming monitor, you'll check 2 separate specs:

  1. Refresh rate, which is about the smoothness. Often monitors have 60fps but 144hz is now quite popular for first-person shooting gamers.
  2. Input latency. It could have a 240hz refresh rate but a high latency (say 10ms), or a 60hz refresh rate with low latency (say 1ms).

I think the separation of smoothness and latency also applies to Flutter. It's best to measure, track, and optimize them separately.

For the "jank" or smoothness specifically, it's just about how continuous your frames are. If you have two frames that differ from one another significantly in an unexpected way, users will detect such "jank" or non-smoothness.

In my experience, there are 2 ways to have such non-smoothness:

  1. There's too much a time gap between the first frame and the second frame. It's usually caused by the first frame taking up too much time which creates an discontinuity in time, and thus a discontinuity in pixels (assuming that each pixel is a smooth function in time). Be aware that the the time gap may not be equal to the total time that 1st frame takes when there's concurrency. For example, in Flutter where 2 threads exist, 1st frame could take 32ms to render while we only maintain a 16ms time gap between 2 frames.
  2. The pixel itself is unexpectedly non-smooth in time. This is usually caused by a bug in the application or the framework which unexpectedly turns a smooth physical process into a non-smooth animation. In this case, no matter how fast a frame renders, the users will still detect the "jank". See https://github.com/flutter/flutter/issues/813#issuecomment-403946266 and https://github.com/flutter/flutter/issues/19222 for an example.

Now let's turn back to the latency. When there's a high latency, users may still see a very smooth animation, but they'll feel that the animation is lagged behind their input. Note that "an input event" is required for a nontrivial latency issue. If there's no user input and you see a lagged but smooth animation, you can simply shift your time function a little to synchronize everything. So if we're going to measure the latency, it's better to measure from the start of the input event. Measuring from the start of rendering is Ok (just like the monitors only measure latency from the input signal to the output pixels). But be aware that doesn't tell the full story of user-experienced latency.

xuning0 commented 3 years ago

@kenzieschmoll I have some doubts about the case 2 diagram. In my understanding, after the cpu calculation is completed, data is submitted to the gpu for rendering. Why does the gpu start rendering before the cpu ends in the diagram? Thank you for your answer.

TheVinhLuongFossil commented 2 years ago

Let's say I have two subsequent frames A and A+1 in chronological order. Is it possible for frame A is still in the middle of rasterizing while frame A+1 is in building phase (raster time of A overlaps with frame A+1's build time)?

kenzieschmoll commented 2 years ago

Is it possible for frame A is still in the middle of rasterizing while frame A+1 is in building phase (raster time of A overlaps with frame A+1's build time)?

Yes this is possible, and most of the time expected. Build and Raster happen on two different threads and can happen in parallel.

kenzieschmoll commented 3 days ago

Closing since there is no action to be taken here.