Add DRM overlay planes API

ghost commented 6 years ago

Separate from #193, this issue will track status of using overlay planes.

wlroots has migrated to gitlab.freedesktop.org. This issue has been moved to:

https://gitlab.freedesktop.org/wlroots/wlroots/-/issues/859

ascent12 commented 6 years ago

There is already some half-assed support for it in the DRM backend (because we use primary and cursor planes), but it really needs to be fleshed out a bit more and actually exposed in a useful way. I'll get around to them in my https://github.com/ascent12/drm_doc series, but for now I'll give a quick overview:

An overlay/cursor plane is a framebuffer that can be laid on top of the CRTC (primary plane) and used to composite images together. Think exactly how we draw a window normally, but this all happens in hardware, so it's much more efficient, and we don't need to damage/redraw whatever is under it. Overlay and cursor planes are pretty similar, but are used for different purposes.

Why they're good

It allows us to perform a lot of operations without needing to wake the GPU up. Less GPU means better battery life.

For example, we could perform an interactive drag for a floating window without needing to touch GL at all; we just move the plane. We could implement things like layer-shell surfaces over the top of normal content without needing to damage content beneath it.

Why they're so annoying to use

Due to the wide variation of hardware support, there is a lot of shit you have to deal with.

There is no guarantee for the number of planes present (0 is quite common)
They may only work with particular CRTCs (think wlr_output)
They may or may not support hardware rotation
They may or may not support Z-ordering

The "legacy" API for planes is complete crap and is unsynchronised with the pageflip, which would lead to horrible tearing. Weston has some weird shit they do to get around this, but I'd just limit this API to the atomic backend.

This is also going to make our damage tracking even more complicated.

The API

I can't think of sane way to "wrap" this API for library users without removing a lot of the power and usefulness of planes. Perhaps we just expose them very close to how they are, and just let the user deal with the complexity. It's a purely optional thing, so if they don't want to deal with it, they don't have to.

There is also going to be some interaction with #774.

ddevault commented 6 years ago

I can't think of sane way to "wrap" this API for library users without removing a lot of the power and usefulness of planes. Perhaps we just expose them very close to how they are, and just let the user deal with the complexity.

This idea is bourne of ignorance and probably stupid but I imagined having wlr_output learn the ability to manage planes (perhaps on wayland taking advantage of subsurfaces), via some functions to attempt to make new planes and some other functions to make them current and suchwhat.

emersion commented 6 years ago

Yeah, we'll probably want to add a new wlr_output_plane struct with the usual make_current, swap_buffers, … functions. These would be backed by DRM planes in the DRM backend and subsurfaces in the Wayland backend.

I'll have a look at Weston's API, since they now support it.

ascent12 commented 6 years ago

Hmm, I never thought about wayland subsurfaces when it comes to this. I could sort of see how that may work. Maybe if you took it far enough, you could pass off the real compositing to the parent compositor, and need to actually draw very little ourselves, but it would make input more complicated.

Basically, there would be a function wlr_output_get_plane which would get a plane if it can, and fail otherwise.

X11 and Headless will always return NULL
Non-atomic DRM will always return NULL (unless we choose to implement the same Weston hacks)
Atomic DRM will return planes as long as it can (possibly none)
Wayland may be able to return a valid plane indefinitely

Some extra restrictions on Atomic DRM

If Z-ordering is not supported, we will only return a max of 1 plane per output, even if more are supported. The unspecified plane ordering is just not useful.
By assigning a plane to a wlr_output, we may take away the ability for another wlr_output to have it. As such, getting the max number of planes for an output is not obvious, and could be extremely misleading. So we shouldn't implement a function for that; users just keep trying to take new planes until it fails.
For the same reason above, planes shouldn't be considered a long-lived object that would last the entire lifetime of the wlr_output. They should be gotten and released frequently, as needed.
When we hotplug an output, we may reshuffle the CRTCs used by each output, so we can get a modeset that works. We try as hard as we can not to, but sometimes it's unavoidable. This can change the set of supported planes for each wlr_output. So we may need to support the ability to revoke planes at any time.

ddevault commented 6 years ago

Should we do soft planes like we do soft cursors if the hardware/backend/whatever isn't up to snuff?

ascent12 commented 6 years ago

That kind of defeats the entire purpose of planes. It'll just punt all of the hard compositing problems into the renderer, instead of the user managing their windows themselves.

ddevault commented 6 years ago

Alright, fair enough. Those problems will be hard, though.

ascent12 commented 6 years ago

Also, just to bring things into context: Typical hardware (at least as far as I've seen) supports very few overlay planes. Out of the 5 DRM devices I have access to, only 2 support overlay planes (both intel iGPU laptops), and they only support 1 overlay plane per CRTC. We don't want to exclude any unknown hardware or hardware that may come, so we should be prepared for many more planes to be supported.

However, don't think of this like we can have every client sitting on its own plane.

ddevault commented 6 years ago

I imagine it will only be practical to do fullscreen surfaces this way.

ascent12 commented 6 years ago

Yeah, that would be the easiest one. There are a few extra little optimisations that would be possible even with a single overlay plane. We can just move clients between the main framebuffer (primary plane) to the overlay plane. So we could still get the efficient interactive drag, which is probably one of the most inefficient operations we can do when it comes to using the GPU.

emersion commented 6 years ago

Weston decided not to allow overlapping planes because the order of planes is ill-defined.

emersion commented 5 years ago

Related: support for underlay planes

In general,the difference for underlay plane compared to overlay plane is that the video data has to be "underlaid" through a black transparent quad rendered on the top plane(primary/RGB). In other words, underlay plane has no individual existence w/o a primary plane being rendered on top with black data or data size smaller than that rendered on underlay plane.

More info about underlay planes: https://gitlab.freedesktop.org/wayland/weston/issues/9

emersion commented 5 years ago

Note that Weston makes no difference between a cursor plane and an overlay plane (apart from z-index). This means a window can be placed on a cursor plane. After all, a cursor plane is just like an overlay plane with additional constraints. I think we could use the same approach to de-duplicate the code.

emersion commented 5 years ago

i915 patch to expose the zpos property: https://patchwork.freedesktop.org/series/58761/

emersion commented 5 years ago

For example, we could perform an interactive drag for a floating window without needing to touch GL at all; we just move the plane. We could implement things like layer-shell surfaces over the top of normal content without needing to damage content beneath it.

BTW, planes are useful in a lot more situations. For instance we could start by trying to put the focused view on a plane. Then each time the view updates its contents we can skip compositing completely.

I'll have a look at Weston's API, since they now support it.

Turns out the DRM backend has access to views, so it's not really relevant.

Tentative API:

struct wlr_plane {
    const struct wlr_plane_impl *impl;
    int z;
};

// Returns NULL if no planes are available anymore
struct wlr_plane *wlr_output_get_plane(struct wlr_output *output);
void wlr_plane_destroy(struct wlr_plane *plane);
bool wlr_plane_attach_dmabuf(struct wlr_plane *plane, struct wlr_dmabuf_attributes *attribs);
void wlr_plane_set_position(struct wlr_plane *plane, int x, int y);
bool wlr_plane_commit(struct wlr_plane *plane);

The only issue with this API is: what does a compositor do when wlr_plane_attach_dmabuf returns false? It means the buffer can't be attached to this particular plane (likely because of a format or size restriction), but maybe it'll work on another one? Should it try to create another plane until there are no more planes or the buffer gets accepted?

How does this play with the Wayland backend?

swaywm / wlroots