framebuffer session: atomic blitting and panning

[nfeske]

The current GUI and framebuffer session interfaces do not provide a satisfactory way for clients to display animations without tearing artifacts. Even though a few full-screen applications like scout implement atomic back/front-buffer flipping using the GUI session's Offset command, this approach operates on the GUI session - not the framebuffer session - and thereby involves the participation of intermediaries of the GUI stack like the window manager. For non-full-screen applications, the current best practice is the client-side back-to-front copying, which hides intermediate drawing states but cannot rule out tearing artifacts. After all, the capturing of the front buffer (by the nitpicker GUI server on behalf of a display driver) happens concurrently with the client-side back-to-front copy operation. Most applications including Qt apps fall into this category.

Long story short, tearing artifacts are hardly avoidable with the current interfaces. To overcome this situation, the framebuffer session should be extended by two operations:

1. A Framebuffer::Session::blit operation instructs the GUI server to copy a rectangular part of the client's virtual framebuffer to anther position. This enables a client to place both its back buffer and its front buffer inside the virtual framebuffer and let the GUI server perform the back-to-front blitting. Thereby, blitting operations become implicitly serialized with the capture clients (i.e., display driver). Consequently, the driver always observes a consistent picture.

2. A Framebuffer::Session::panning operation allows a client to specify the position within its virtual framebuffer that corresponds to the upper-left corner of the displayed picture. It is similar to the existing Gui::Session::Command::Offset but operates on the framebuffer-session interface (taking the wm from the critical path). The panning position can be used to implement the atomic back/front-buffer flipping. It may also be useful for the efficient implementation of scrolling.

Both operations imply the Framebuffer::Session::refresh of the affected pixels. So the number of RPC calls stays the same.

[nfeske]

The patch series above implements this change and adjusts several GUI clients like window decorators, menu_view, gui_fader, and demo applications. It thereby removes tearing artifacts on account of client-side double buffering by design.

Note that the new mechanisms do not address tearing artifacts on account of the scanout of the display driver. This residual effect needs to be addressed by synchronizing the operation of the display driver with the vsync of the hardware, which is a matter local to the respective driver.

[alex-ab]

@nfeske: i added a prototype commit which uses the drm vblank feature for the Intel display driver. Technically it should work now, but it probably needs more tuning or understanding of that matter.

[nfeske]

Thanks @alex-ab for the prototype. One can nicely observe the effect when assigning origin="pointer" to the leitzentrale domain in the nitpicker config, and slowly moving the mouse. My observation is as follows:

1. There is a tearing artifact visible at a rather static vertical position. This means, the vsync synchronization is in effect! Otherwise, the artifact would sporadically appear at different vertical positions.
2. The vertical position of the artifact depends on the position of the mouse, which is logical because the costs of capturing/compositing depends on the horizontal mouse position. When on the left border, the whole screen (with the leitzentrale as mouse cursor) changes. When nearby the right border, only a narrow column needs to be updated. Interestingly, the artifact moves almost to the bottom of the screen in the worst case. So the capturing/compositing can be rather expensive (up to 16 ms).

From the observation, I take the following:

• Vsync signalling works, which is nice!
• The order of capturing and back-to-front blitting in the driver should better be changed. Right now, it is: vsync, capture-current, blit-current. The sum of capture-current and blit-current exceeds the vblank time. We need to "move" this procedure in time relative to the vsync such that only the blit phase falls into the vblank time window. This could in principle be achieved by setting up a mid-frame timer (so the position of the time window could be adjusted freely, at the expense of an additional timeout per frame). Alternatively, we could try to change the sequence of the operations to: vsync, blit-previous, capture-current. So the blitting comes directly when observing the vblank.

As a note of caution: right now, nitpicker delivers sync signals to each client on any capture call. When using two displays, the amount of capture calls doubles. Right now, this will cause the doubling of the frame rate as observed by nitpicker's GUI clients. So clients like menu_view that render on sync will become twice as busy. It goes without saying that the sync signal distribution by nitpicker will be fixed (delivering sync signals by only one sync source per GUI client) but I wanted to share this known limitation.

[nfeske]

Fixed in master.