layer-shell: Exclusive zone while sized by the compositor

[dos1]

zwlr_layer_surface_v1's set_size(width, height) states:

        If you pass 0 for either value, the compositor will assign it and
        inform you of the assignment in the configure event. You must set your
        anchor to opposite edges in the dimensions you omit; not doing so is a
        protocol error. Both values are 0 by default.

Which seems fair - if the surface is going to span the entire available height, it probably makes sense for it to be anchored to both top and bottom edge.

Consider a surface that's meant to work as an unfoldable panel. In its folded state, it takes some vertical space at the bottom of the screen which is meant to not be obstructed by running applications:

When unfolded, it covers the whole available screen space:

To achieve that, the surface can be anchored to all edges and set size of (0,0), hiding its unused portion behind the screen edge by using negative margin:

However, this interacts poorly with how exclusive zones work:

        A positive value is only meaningful if the surface is anchored to one
        edge or an edge and both perpendicular edges. If the surface is not
        anchored, anchored to only two perpendicular edges (a corner), anchored
        to only two parallel edges or anchored to all edges, a positive value
        will be treated the same as zero.

By setting the anchor to two parallel edges we're losing the information on where to place the exclusive zone.

---

Above describes the way phosh currently works. Turns out that, when implementing unfolding, we have carelessly violated the protocol by setting the surface's size to (0,0) while not anchoring it to the top edge - which actually turned out to work in expected way on all implementations I tried it on back in the day. The surface covered the whole available space, while the exclusive zone remained anchored to the bottom edge of the screen.

In recent versions, wlroots started to enforce the rule as specified in the protocol, which made me look at how to reimplement this behavior in a conforming way. Unfortunately, I have found no other solutions - managing the surface size by the client has made it impossible for other exclusive zones to influence the surface's size, which is apparent in case of the on-screen keyboard on a higher layer - the whole surface gets "pushed" upwards by the size taken by the keyboard, not given a chance to adjust its size:

On the other hand, letting the compositor manage the surface size makes it impossible for the surface to define its own exclusive zone, because there's no way to specify the screen edge it should be anchored to. This would make all maximized surfaces uselessly change their size every time the panel gets unfolded since there would be no effective exclusive zone anymore, which is disastrous for performance, especially with many surfaces being mapped.

Therefore, here comes the question:

• should the restriction on anchored edges when dimension equals 0 be lifted from the protocol, making wlroots work the same way it did before commit https://github.com/swaywm/wlroots/commit/8dec751a6d84335fb04288b8efab6dd5c90288d3?
• should exclusive zone interface be extended to allow being explicit about the screen edge it should apply to?
• should it stay as is, making it a thing that should be resolved by using other/custom protocols?

[zzag]

By setting the anchor to two parallel edges we're losing the information on where to place the exclusive zone.

Could you please explain why the overview surface needs to have a positive exclusive zone? At quick glance, the overview surface needs to be anchored to all four screen edges and the exclusive zone has to have a value of -1 or 0.

we have carelessly violated the protocol by setting the surface's size to (0,0) while not anchoring it to the top edge

It's probably an abuse of the protocol, but you could set the top and the bottom anchors. In order to move the overview surface to the bottom, you will need to set the top margin to screen_height and the bottom margin to -screen_height.

[ifreund]

It's probably an abuse of the protocol, but you could set the top and the bottom anchors. In order to move the overview surface to the bottom, you will need to set the top margin to screen_height and the bottom margin to -screen_height.

This doesn't work if there are other exclusive zones present. A "better" hack would be to use a separate, invisible layer surface to enforce the exclusive zone and let the compositor manage the size of the actually rendered surface.

[dos1]

Could you please explain why the overview surface needs to have a positive exclusive zone?

There's no separate surface for the overview - the "panel" is the overview, it's the same surface. Currently the unfolding is animated by the client, but in the future it's going to be swipeable by the user.

While we could remove the exclusive zone (set it to 0) when unfolding, this would resize all xdg-shell surfaces (this is a shell for phones, so windows are usually maximized) every time the overview is (un)folded, which is slow, resource consuming, visually glitchy and, well, should be avoided.

A "better" hack would be to use a separate, invisible layer surface to enforce the exclusive zone and let the compositor manage the size of the actually rendered surface.

I guess that could somehow work, but this raises another issue - at least in sway's and phoc's implementations, exclusive zones between surfaces on the same layer depend on inner-layer surface ordering, which is implementation defined (and that ordering already differs between phoc and sway - personally I believe the one used by phoc makes much more sense, but the protocol explicitly says it's up to the implementation, so...). This means that sizing of the actually rendered surface will still be tricky - it would have to somehow take other exclusive zones into account, but not the hacky invisible one, and you have no way to tell whether the compositor took that one into account or not.

[dcz-purism]

I was not sure if I understood the exclusive zone problem, so let me rephrase in different terms to check if I'm getting this right.

There are two modes for the surface in question to be in: folded and full-screen. In folded mode, the "fold" part visible on the screen is the exclusive zone. In full-screen mode, the surface must keep the exclusive zone at the same value, because the compositor will otherwise try to resize all surfaces to the new exclusive zone. If that wasn't the case, the full-screen mode could have been achieved without any exclusive zone.

If my understanding here is correct, then the only worry is about resizing surfaces which are not visible anyway. Why not change the compositor to wait with resizing them until they are actually visible? That would save the protocol from changes.

[dos1]

@dcz-purism a) they still are kinda visible, since the surface in question contains window switcher with thumbnails b) the unfolding is animated and will be swipeable by the user in the future, which means that the exclusive zone (and surface height) has to be updated already at the animation/swipe start, at which point the underneath surfaces are still visible c) in our specific case the surface has opaque black background indeed, but nothing prevents it from being translucent

[dcz-purism]

Thanks. The problem sounds like a huge corner case. Consiering only the static case, having a full-screen surface with a non-full-screen exclusive zone doesn't make much sense. If we could find more examples of those, I would say that adding a way to select exclusive side makes sense.

Otherwise, it seems like we're trying to have a different model for sizing (exclusive zone stays the same), and a different one for rendering (visible surface size changes). I would vote for making this explicit and using a different item for the exclusive area, and a different one for the rendered area, except layer-shell is not very good at dealing with placing things in space once it gets slightly complex. So instead I'm not a fan of the two surface solution.

[emersion]

It's important that the compositor has the enough information to completely compute the layer surface layout before mapping them. In other words, it's not possible for a client to decide what its exclusive zone is in reply to a configure event, because this might change the layout and trigger another configure event. This leads to imperfect frames and possibly infinite feedback loops.

So in the protocol, we can't just make the exclusive zone a rectangle.

---

What about this approach: open one layer surface for the bottom bar, set its exclusive zone as usual. When the user begins a swipe up, open a fullscreen layer surface without an exclusive zone, initially completely transparent. Keep the bottom bar layer surface with its exclusive zone mapped, maybe make it transparent. Draw the animation on the fullscreen layer surface.

Would that work?

[dos1]

So in the protocol, we can't just make the exclusive zone a rectangle.

There's no need to make it a rectangle or set it in reaction to configure events. All that would be needed is to be able to express something like "apply the exclusive zone to the BOTTOM edge" when setting anchors to both BOTTOM & TOP, as that's the only missing part here. With this information provided by the surface, exclusive zone anchoring restrictions could be greatly reduced.

Something like layer_surface::set_exclusive_zone_anchor that only allows a single edge out of those set by layer_surface::set_anchor should provide everything that's needed for what I described in this issue. Does that solution sound reasonable?

Would that work?

If I understand it correctly, not without ensuring the order in which exclusive zones are applied by the compositor. IIRC sway uses an opposite order than phoc, and the protocol leaves it unspecified.

[emersion]

wlr-protocols has migrated to gitlab.freedesktop.org. This issue has been moved to:

https://gitlab.freedesktop.org/wlroots/wlr-protocols/-/issues/102