Window decorations broken with GLFW and libdecor

LDAP commented 6 months ago

Hyprland Version

System/Version info

```sh Hyprland, built from branch main at commit 1d2acbe19355c8640d54a4b6cba225c6f4370c85 (config: add absolute monitor workspace selectors (5848)). Date: Fri May 3 19:38:00 2024 Tag: v0.39.1-138-g1d2acbe1, commits: 4598 flags: (if any) System Information: System name: Linux Node name: 10303B-arch Release: 6.8.9-arch1-1 Version: #1 SMP PREEMPT_DYNAMIC Thu, 02 May 2024 17:49:46 +0000 GPU information: 0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] [1002:73bf] (rev c1) (prog-if 00 [VGA controller]) os-release: NAME="Arch Linux" PRETTY_NAME="Arch Linux" ID=arch BUILD_ID=rolling ANSI_COLOR="38;2;23;147;209" HOME_URL="https://archlinux.org/" DOCUMENTATION_URL="https://wiki.archlinux.org/" SUPPORT_URL="https://bbs.archlinux.org/" BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues" PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/" LOGO=archlinux-logo plugins: ```

Bug or Regression?

Bug

Description

This happens if libdecor is used for window decorations in a GLFW window:

Window decorations are drawn above of the window
There is a gap between the decorations and the start of the content
The top pixels are repeated by the width of the decorations
Mouse clicks are registered at the wrong position

Does not happen when an X11 session is enforced with XDG_SESSION_TYPE=x11. Other compositors do also work (Mutter).

How to reproduce

Can't share my test application, but this also reproduces the issue: hanatos/vkdt (compile with make, then run ./bin/vkdt)

Details:

Create a GLFW 3.4 window and render something on it (using Vulkan)
Make sure GLFW_WAYLAND_LIBDECOR is hinted as GLFW_WAYLAND_PREFER_LIBDECOR (should be default)
Make sure the Wayland backend is used

Crash reports, logs, images, videos

issue

w0utert commented 5 months ago

I have the same issue with my application which is just glfw + vulkan and does not explicitly use or configure libdecor in any way. Not just the window decorations are broken but also mouse coordinates do not line up with the application window anymore. This appears to be a recent regression because until at most a few weeks ago GLFW + hyprland was working without problem.

SagaPDev commented 5 months ago

I'm experiencing the same in a application using glfw and opengl https://github.com/larfingshnew/3d-controller-overlay, a title bar will be drawn in top of the windows and will overlap with every other element on screen as long as the windows is focused, there is also a gap top and bottom, and there will be and small vertical offset between were the mouse pointer is and what is being selected. This does not happens in sway were the title bar is correctly omitted, and in gnome and kde it will just draw a normal titlebar 2024-05-26_12-13_2

vaxerski commented 5 months ago

can someone bisect this

w0utert commented 5 months ago

I could try to bisect later this week

phonetic112 commented 5 months ago

bad commit is ed69502ff6e79a6dad213333b0bc3a15e2247942

w0utert commented 5 months ago

@phonetic112 you just beat me to it ;-)

Looking at the debug log it appears like clients with GLFW windows do not end up in the XDGDecoration protocol handler.

For e.g. kitty I see this:

[LOG] Searching for matching rules for kitty (title: )
[LOG] Window rule suppressevent maximize ->  class:.* matched [Window 5adf5c7790d0: title: ""]
[LOG] [hookSystem] New hook event registered: windowUpdateRules
[LOG] Window 5adf5c7790d0 set title to kitty
[LOG] Searching for matching rules for kitty (title: kitty)
[LOG] Window rule suppressevent maximize ->  class:.* matched [Window 5adf5c7790d0: title: "kitty"]
[LOG] [XDGDecoration] setMode: MODE_SERVER_SIDE. Sending MODE_SERVER_SIDE as reply.

So Hyprland is sending MODE_SERVER_SIDE to the client which if I understand the protocol correctly should be the right thing.

For my GLFW window it never gets there, there are not other XDGDecoration messages in the log file at all.

I could try to debug further but since I have zero knowledge of the hyprland code base probably someone else can diagnose the problem much faster than I can figure it out.

vaxerski commented 5 months ago

I'm quite busy with the wayland core rewrite mr, try debugging the aptly named protocols/xdgdecoration.cpp file I guess

w0utert commented 5 months ago

I've debugged this and I'm starting to think this may actually a GLFW problem and not a Hyprland problem.

When starting a 'well-behaved client' like e.g. kitty, chromium or an SDL2 client using SDL_VIDEODRIVER=wayland, the Hyprland XDGDecoration protocol code goes through a zxdg_decoration_manager_v1::get_toplevel_decoration request first, then it receives a zxdg_toplevel_decoration_v1::set_mode request, ignoring whatever the client prefers and instead replying with a zxdg_toplevel_decoration_v1::configure event with zxdg_toplevel_decoration_v1::mode::server_side to tell the client it should not render any decorations.

For GLFW clients, not so much. Hyprland gets a zxdg_decoration_manager_v1::get_toplevel_decoration request but it never gets a zxdg_toplevel_decoration_v1::set_mode request after that, which means GLFW will default to client-side decorations (and fails to render those properly as well, apparently).

Looking at the stack trace at the point where the zxdg_toplevel_decoration_v1::set_mode is received for 'good clients' (this ends up in _CZxdgToplevelDecorationV1SetMode in protocols/xdg-decoration-unstable-v1.cpp), there is no Hyprland code between that point and libwayland, so I don't believe the problem is in Hyprland.

If this makes sense I can file a ticket with GLFW or try to debug it myself, but I could of course be missing something about the Hyprland Wayland protocol handling that could affect the set_mode request getting lost for GLFW clients.

vaxerski commented 5 months ago

I'd still send the hint. It wouldn't hurt I feel like.

Opposite34 commented 5 months ago

I don't know if this helps with further debugging or if it's nonsense (since I have no idea how any of these works), but if you open said GLFW with the borked decorators and drag it around without snapping it into tiled (mod key + mouse dragging around), then only the decorator will leave some artifacts and not properly clear until some time later (or until the window is tiled) while the actual content of the window will clear properly. Said behavior of leaving artifact doesn't happen in, say, firefox even though it has its own window decorator.

Here's an example where I run this GLFW example in odin lang (I'm trying to learn the language, which led me into this discussion):

https://github.com/hyprwm/Hyprland/assets/54463205/a3f5b4ae-16e7-43a5-adec-70398cefd41d

w0utert commented 5 months ago

I'm now knee-deep into debugging how GLFW and libdecor interact with the compositor, and I think maybe this is a Hyprland problem after all.

Observations:

GLFW defaults to using libdecor for decorations these days
libdecor uses zxdg_decoration_manager_v1::get_toplevel_decoration to ask the compositor about window decorations (if the compositor supports it)
if the compositor replies with zxdg_decoration_manager_v1::mode::server_side it will disable drawing decorations
Running my GFLW app on KWin, the debugger ends up in the libdecor protocol listener for zxdg_decoration_manager_v1::get_toplevel_decoration, receiving zxdg_decoration_manager_v1::mode::server_side (the value 2) => application window is drawn correctly using KWin server-side decorations
Running the exact same app on Hyprland, the listener callback is never hit => libdecor will decide to draw decorations using the gtk plugin (which is what you see in the screenshot above)

Most likely once you get to 5, there is some mismatch between what Hyprland thinks the size of the client area is vs what libdecor thinks, which leads to the display artefacts and mouse coordinate mismatch. Or something along those lines.

As for why 5 happens, I'm not entirely sure. I see XDGDecorations.cpp is written to respond to xdg_decoration_toplevel_v1::set_mode by emitting a xdg_decoration_toplevel_v1::configure event, and this works as expected for clients that do the set_mode request (cf. kitty, chromium, etc). But it seems set_mode is mostly intended for requesting a change in decorations, and clients are not actually required to call it, they can also get the decoration mode when doing get_toplevel_decoration. Specifically, libdecor does not use set_mode anywhere, ie: it has no support for changing decorations. GLFW does use set_mode if you disable libdecor by the way, by removing the GLFW_WAYLAND_PREFER_LIBDECOR window hint. If you do that, the problem goes away but then your Gnome users will have no window decorations because Gnome refuses to support xdg-decorations...

I cannot figure out how to fix this though, because I don't understand the mechanisms by which the Hyprland protocol handlers are supposed to communicate with the client. It's probably trivial to fix this if all that is needed is to make sure Hyprland sends 'zxdg_decoration_manager_v1::mode::server_side when a client is requesting zxdg_decoration_manager_v1::get_toplevel_decoration.

w0utert commented 5 months ago

Ok I was able to fix this with a one-liner and created PR #6308

The fix is to send the xdg_decoration_toplevel_v1::configure event when the CXDGDecoration resource is created in response to a xdg_decoration_toplevel_v1::get_toplevel_decoration request.

I just threw the event emit into the CXDGDecoration constructor which is probably not the right place to do it, but at least it demonstrates how this can be fixed.

hyprwm / Hyprland