X11 DisplayServer does not set correct `_NET_WM_WINDOW_TYPE` for dialogs, causing issues with tiling window managers (DisplayServer regression)

[Amatrelan]

Godot version

v4.0.dev.custom_build [69a194f05]

System information

Linux, x11, nvidia

Issue description

Many new windows what is created has _NET_WM_WINDOW_TYPE(ATOM) = _NET_WM_WINDOW_TYPE_NORMAL what creates jarring user experience with Tiling Window Managers.

I would recommend change _NET_WM_WINDOW_TYPE(ATOM) set to _NET_WM_WINDOW_TYPE_DIALOG for all dialog windows.

Places where this should be changed to _NET_WM_WINDOW_TYPE_DIALOG as I found out in fast testing is:

• Add child
• Instantiate Child Scene
• Open Scene
• Save (When selecting place where to save)
• In right click menu basically all selections creates new dialogs but has _NET_WM_WINDOW_TYPE = NORMAL.
• Quick Load
• When creating new scene and selecting Other Node as base type.

On many applications only main window type is set to _NET_WM_WINDOW_TYPE_NORMAL and all others are set to other ones (mostly to _NET_WM_WINDOW_TYPE_DIALOG). And this is how most of tiling window managers are coded to handle _NET_WM_WINDOW_TYPE_DIALOG as floating window automatically.

Here is one possible fix for this.

index 74ec8b6..97f603e 100644
--- a/platform/linuxbsd/display_server_x11.cpp
+++ b/platform/linuxbsd/display_server_x11.cpp
@@ -4179,10 +4179,17 @@ DisplayServerX11::WindowID DisplayServerX11::_create_window(WindowMode p_mode, V
                        if (wt_atom != None && type_atom != None) {
                                XChangeProperty(x11_display, wd.x11_window, wt_atom, XA_ATOM, 32, PropModeReplace, (unsigned char *)&type_atom, 1);
                        }
-               } else {
+               } else if (MAIN_WINDOW_ID == id){
                        Atom type_atom = XInternAtom(x11_display, "_NET_WM_WINDOW_TYPE_NORMAL", False);
                        Atom wt_atom = XInternAtom(x11_display, "_NET_WM_WINDOW_TYPE", False);

+                       if (wt_atom != None && type_atom != None) {
+                               XChangeProperty(x11_display, wd.x11_window, wt_atom, XA_ATOM, 32, PropModeReplace, (unsigned char *)&type_atom, 1);
+                       }
+               } else {
+                       Atom type_atom = XInternAtom(x11_display, "_NET_WM_WINDOW_TYPE_DIALOG", False);
+                       Atom wt_atom = XInternAtom(x11_display, "_NET_WM_WINDOW_TYPE", False);
+
                        if (wt_atom != None && type_atom != None) {
                                XChangeProperty(x11_display, wd.x11_window, wt_atom, XA_ATOM, 32, PropModeReplace, (unsigned char *)&type_atom, 1);
                        }

Steps to reproduce

• Install tiling window manager (For example: QTile)
• Build godot (master)
• Open any dialog window

Minimal reproduction project

No response

[rsubtil]

See also #37930 and #38727

I remember experimenting with the _NET_WM_WINDOW_TYPE_DIALOG atom a long time ago without great results, but that was before that PR was merged, so it might work without issues now.

[Amatrelan]

At least in testing I did yesterday, with that patch it seemed to work correctly in QTile. And as far I recall _NET_WM_WINDOW_TYPE_DIALOG is also default floating window in i3wm, if not but it can be easily enabled as floating window type in i3wm if not default.