Proper handling of EWMH desktop windows.

[jarcode-foss]

Currently the handling for _NET_WM_WINDOW_TYPE_DESKTOP is technically compliant but unfortunately assigns a frame (parent) to the X11 client. This causes problems with most applications that use this window type, notably:

• Desktop file managers
• Desktop audio visualizers and other widgets (ie. conky with own_window).
• Especially any desktop window that uses XShape since they require workarounds to handle the parent window.

Most window managers do not assign frames or parent windows here. Also, the specification offers some commentary behind _NET_WM_WINDOW_TYPE_DESKTOP:

This spec suggests implementing the file manager desktop by mapping a desktop-sized window (no shape) to all desktops, with _NET_WM_WINDOW_TYPE_DESKTOP. This makes the desktop focusable and greatly simplifies implementation of the file manager. It is also faster than managing lots of small shaped windows. 

At the moment application developers can work around this by setting the newly supported motif hints or ignoring the window manager entirely with override_direct and re-parenting the window from within the application.

Ideally it should be assumed that there are no frames for a desktop window.

[psychon]

What kind of problems?

Especially any desktop window that uses XShape since they require workarounds to handle the parent window.

Huh? What kind of XShape-related workarounds are required?

On IRC you mentioned Glava. A quick search only finds one shape-related thing in there: https://github.com/wacossusca34/glava/blob/d640ac5d3c31111ebd456bae0f1b4d782481ffa8/glx_wcb.c#L278-L298

So... Do you mean "the input shape does not work correctly" here? If so:

• You should check the SHAPE version that the server supports. The input shape only exists in version 1.1 or later. With version 1.0, you will likely get a protocol error (but your error handler silently ignores all errors; I guess I do not want to know why).
• I did not yet find a nice example program to test shape input support with and so AwesomeWM does not do anything special here. Bounding shape and parts of clip shape (or was it the other way around?) can be tested with xeyes.

Untested patch to add support for client input shape (one important thing is: This needs to be tested with a client with a non-zero border width; I always get the border handling wrong with shapes):

diff --git a/event.c b/event.c
index 989c0110c..5879c5ea1 100644
--- a/event.c
+++ b/event.c
@@ -948,6 +948,8 @@ event_handle_shape_notify(xcb_shape_notify_event_t *ev)
             luaA_object_emit_signal(L, -1, "property::shape_client_bounding", 0);
         if (ev->shape_kind == XCB_SHAPE_SK_CLIP)
             luaA_object_emit_signal(L, -1, "property::shape_client_clip", 0);
+        if (ev->shape_kind == XCB_SHAPE_SK_INPUT)
+            luaA_object_emit_signal(L, -1, "property::shape_client_input", 0);
         lua_pop(L, 1);
     }
 }
diff --git a/lib/awful/client/shape.lua b/lib/awful/client/shape.lua
index 6dcb563e4..91ec721e5 100644
--- a/lib/awful/client/shape.lua
+++ b/lib/awful/client/shape.lua
@@ -22,7 +22,7 @@ shape.update = {}
 -- @client c The client whose shape should be retrieved
 -- @tparam string shape_name Either "bounding" or "clip"
 function shape.get_transformed(c, shape_name)
-    local border = shape_name == "bounding" and c.border_width or 0
+    local border = (shape_name == "bounding" or shape_name == "input") and c.border_width or 0
     local shape_img = surface.load_silently(c["client_shape_" .. shape_name], false)
     local _shape = c._shape
     if not (shape_img or _shape) then return end
@@ -95,6 +95,7 @@ end
 function shape.update.all(c)
     shape.update.bounding(c)
     shape.update.clip(c)
+    shape.update.input(c)
 end

 --- Update a client's bounding shape from the shape the client set itself.
@@ -121,8 +122,21 @@ function shape.update.clip(c)
     end
 end

+--- Update a client's input shape from the shape the client set itself.
+-- @function awful.client.shape.update.input
+-- @client c The client to act on
+function shape.update.input(c)
+    local res = shape.get_transformed(c, "input")
+    c.shape_input = res and res._native
+    -- Free memory
+    if res then
+        res:finish()
+    end
+end
+
 capi.client.connect_signal("property::shape_client_bounding", shape.update.bounding)
 capi.client.connect_signal("property::shape_client_clip", shape.update.clip)
+capi.client.connect_signal("property::shape_client_input", shape.update.input)
 capi.client.connect_signal("property::size", shape.update.all)
 capi.client.connect_signal("property::border_width", shape.update.all)

diff --git a/objects/client.c b/objects/client.c
index f38e1626b..8f3b8387e 100644
--- a/objects/client.c
+++ b/objects/client.c
@@ -795,6 +795,17 @@
  * @param surface
  */

+/**
+ * The client's input shape as set by the program as a (native) cairo surface.
+ *
+ * **Signal:**
+ *
+ *  * *property::shape\_client\_input*
+ *
+ * @property client_shape_input
+ * @param surface
+ */
+
 /**
  * The FreeDesktop StartId.
  *
@@ -3601,6 +3612,22 @@ luaA_client_set_shape_clip(lua_State *L, client_t *c)
     return 0;
 }

+/** Get the client's child window input shape.
+ * \param L The Lua VM state.
+ * \param client The client object.
+ * \return The number of elements pushed on stack.
+ */
+static int
+luaA_client_get_client_shape_input(lua_State *L, client_t *c)
+{
+    cairo_surface_t *surf = xwindow_get_shape(c->window, XCB_SHAPE_SK_INPUT);
+    if (!surf)
+        return 0;
+    /* lua has to make sure to free the ref or we have a leak */
+    lua_pushlightuserdata(L, surf);
+    return 1;
+}
+
 /** Get the client's frame window input shape.
  * \param L The Lua VM state.
  * \param client The client object.
@@ -3951,6 +3978,10 @@ client_class_setup(lua_State *L)
                             NULL,
                             (lua_class_propfunc_t) luaA_client_get_client_shape_clip,
                             NULL);
+    luaA_class_add_property(&client_class, "client_shape_input",
+                            NULL,
+                            (lua_class_propfunc_t) luaA_client_get_client_shape_input,
+                            NULL);
     luaA_class_add_property(&client_class, "first_tag",
                             NULL,
                             (lua_class_propfunc_t) luaA_client_get_first_tag,

Most window managers do not assign frames or parent windows here.

My usual go-to example is Fluxbox. All that is done here is the following and so I'd say "Fluxbox does create a frame/parent window" (function FluxboxWindow::setWindowType in src/Window.cc):

    case WindowState::TYPE_DESKTOP:
        /*
         * _NET_WM_WINDOW_TYPE_DESKTOP indicates a "false desktop" window
         * We let it be the size it wants, but it gets no decoration,
         * is hidden in the toolbar and window cycling list, plus
         * windows don't tab with it and is right on the bottom.
         */
        setFocusHidden(true);
        setIconHidden(true);
        setFocusNew(false);
        setMouseFocus(false);
        moveToLayer(::ResourceLayer::DESKTOP);
        setDecorationMask(WindowState::DECOR_NONE);
        setTabable(false);
        setMovable(false);
        setResizable(false);
        setStuck(true);
        break;

application developers can work around this by setting the newly supported motif hints

Did you test this? Does it actually work? Yay. :-) (But the default config ignores these hints and only the docs mention these hints, I think; e.g. https://awesomewm.org/doc/api/documentation/05-awesomerc.md.html (search for no_titlebar))

[jarcode-foss]

Huh? What kind of XShape-related workarounds are required?

The 'XShape' related functionality in GLava contains the workaround via iterating through parent windows. This is not nessecary on Fluxbox. Very few applications do this.

My usual go-to example is Fluxbox. All that is done here is the following and so I'd say "Fluxbox does create a frame/parent window"

I am assuming moveToLayer is re-parenting the window window accordingly here in Fluxbox, but I'm not sure what's actually going on behind the scenes. It could be setting the shape of the parent frame accordingly as well (actually, IIRC Mutter also does this).

Most window managers seem to respond with shape changes solely to the client window when it is placed on the desktop layer, and applications generally expect this.

Supporting input shapes for all client types is probably the best approach; your patch at a glance seems to address this entire class of issues.

[xinhaoyuan]

I tried to support input shape in my fork (https://github.com/xinhaoyuan/awesome), which ends up mostly identical to the patch here, but there is a catch:

xwindow_get_shape will return a filled surface that matches the server-side geometry of the client. One of the problematic cases is that, when set c.maximized = true, shape.update.input(c) will be called before the actual server-side geometry of the client is changed. Thus the maximized client will have the same area of input as before maximizing. For clients actually use input shape, this would not be a problem since it will also update its own shapes, and trigger shape.update.input with the updated geometry of the client, but this would be a problem for clients that do not use input shape.

In my fork I used a workaround that the client_shape_input is only used after shape_client_input signal is captured for a client. I'm not sure if it is the best way to solve this, but it works in my usage.