App time variance

[UkoeHB]

Framepacing adds a delay between the previous frame's presentation and the beginning of the current frame's update cycle. It calculates that delay based on the duration of the previous frame's update cycle.

If the current frame's update cycle takes longer than the previous frame, then the current frame's duration (from previous frame's presentation to current frame's presentation), will be longer than the target duration. This is a problem when vsync is enabled, because the current frame will miss the frame buffer refresh that comes after the previous frame's presentation step (the previous frame was 'just in time' for the buffer swap, but the current frame is 'just too late'). As a result, the previous frame will be presented twice. Depending on the vsync buffering strategy and how many frames are buffered, the current frame may be stuck on the frame presentation step all the way until the next buffer refresh.

Even very low variance in update cycle times can therefore lead to frequent frame drops on machines with unbuffered vsync. On machines with buffered vsync I think you will instead see occasional stutter based on desync between frame presentation and the time stamps of the beginning of update cycles.

Frame loss for unbuffered vsync can be solved by adding a 'buffer' to the framepace sleep based on the variance of app times (e.g. 2 standard deviations).

Since vsync is not always on, you'd need an additional framerate limiter just prior to the frame presentation step, which can absorb any remaining time to reach the target frametime. @aevyrie has observed that on machines with buffered vsync you get better results by targeting a slower framerate than the monitor refresh period -> a framerate limiter would allow compile-time configuration to achieve the best strategy for any given machine.

[aevyrie]

Since vsync is not always on, you'd need an additional framerate limiter just prior to the frame presentation step, which can absorb any remaining time to reach the target frametime.

FWIW, this was actually the initial design of this plugin.

0ms -----------------------
            Estimate how long the next frame will take, minus a small margin to give us space if it takes longer
            Sleep for this duration. (Forward estimation)
8ms         Start of event loop. Get input. Do stuff.
11.8ms      See how close our estimate was to the requested frame time, sleep if required to get the frametime just right
12ms        Send to GPU, present frame (input is 4ms out of date)
16ms ----------------------

That was taken from my comments in the initial discussion that spawned this plugin: https://github.com/bevyengine/bevy/issues/3317

However, in practice I had a lot of trouble getting good results from it. This may have predated my use of spin_sleep, so it may be worth revisiting that approach.

[UkoeHB]

However, in practice I had a lot of trouble getting good results from it. This may have predated my use of spin_sleep, so it may be worth revisiting that approach.

Yeah you'd essentially need to duplicate the framepace PID controller for the framerate limiter. You need very accurate sleep times.

[aevyrie]

Yup, I did play around with writing a PID controller, but I may have been limited by platform sleep time accuracy/variance. Definitely worth revisiting this, I appreciate your deep-dive.

[MatthewScholefield]

Appreciate the investigation into this. Would be great to get this improved. I can replicate frequent stutters on my machine under Wayland.

[UkoeHB]

@MatthewScholefield a comprehensive rework of this crate is blocked by bevy's render code, which doesn't currently expose timing information for presenting frames.

[aevyrie]

Unfortunately that is the case. I can take another look at the PID impl if I get the time, but it's low on my priority list as I expect improvements will be minor until we can properly measure vblanks.