Screencast capture from Wayland session freezes when you load a fullscreen game (beta version)

[cold-distance]

Hi, I've been trying to do a screencast capture from GNOME on Wayland with the beta version of OBS (26.1.0). Everything worked correctly for me except the fullscreen games, but there is a strange behavior.

I tried three games: Doom 2016 (through Proton with the Steam client), Yamagi Quake 2 (Flatpak version from Flathub) and Xonotic (RPM version from Fedora's repositories). With Doom 2016 and Xonotic the screencast capture turns freeze after the game is loaded, but the screen capture is still working and it can be recovered with the alt+tab combination.

With Yamagi Quake 2 the behavior is different, but the are two probabilities about that: First, the game is in Flatpak version, so OBS Flatpak might have a problem managing "native" XWayland applications. By other hand, the mouse cursor doesn't hide after starting the application, so the mouse cursor could be preventing the bug.

I share three videos to show what I'm explaining because my English is very bad and I don't understand exactly what is failing here. My system is Fedora 33 Workstation with 3.38.4 and all packages updated from the official repositories, RPMFusion and Flathub. The hardware is a Radeon RX 580 8GB GPU and a Ryzen 7 1700 CPU. I don't know if the issue is related to this.

Doom 2016: https://drive.google.com/file/d/11NqHefq06o9CNwdvm2oA2fFvJxkEfLFa/view?usp=sharing Xonotic: https://drive.google.com/file/d/1YXTPJzFcMatAGbToIrHtGf5ifUBddNF-/view?usp=sharing Yamagi Quake 2: https://drive.google.com/file/d/1P6C2PnhZXIEi3xoaNQEIhYsGiki4iXsa/view?usp=sharing

[cold-distance]

I see that the bug affects the Steam's Big Picture interface too.

[fastoslinux]

I had no problem here. I don't have anything from RPMFusion

Fedora Silverblue 33 GNOME Wayland 3.38.4 RX 580 8GB / i7 4790k / 16GB ddr3

Steam flatpak: Doom 2016 (vulkan) proton 5.13-6 community buld - ok Steam Big Picture - ok

Xonotic flatpak - ok

maybe something with the OBS recording config? tried to downgrade OBS?

[cold-distance]

Are you talking about the version from the Stable repo instead of the Beta? I tried VAAPI (with your config) and x264 with Flatpak versions of Steam (I just tried Big Picture) and Xonotic and I had the same result, so the record keeps temporarily frozen with fullscreen applications until I close the app or the fullscreen mode. It seems that the mouse cursor is preventing the bug in Yamagi Quake 2.

With OBS took from the stable Flathub repo the Wayland screencapture and the previsualization are directly full frozen just after opening the application, so I can't record anything (and yes, I tried to record and I only get frozen screen videos).

What version of XWayland are you using, @FastOSlinux? I don't know if XWayland is the source of the problem, but if you can record fullscreen applications and me not, it could be the key component.

I just read on Phoronix about EGL support in XWayland 21.1 (I'm just asking).

[fastoslinux]

Are you talking about the version from the Stable repo instead of the Beta? I tried VAAPI (with your config) and x264 with Flatpak versions of Steam (I just tried Big Picture) and Xonotic and I had the same result, so the record keeps temporarily frozen with fullscreen applications until I close the app or the fullscreen mode. It seems that the mouse cursor is preventing the bug in Yamagi Quake 2.

With OBS took from the stable Flathub repo the Wayland screencapture and the previsualization are directly full frozen just after opening the application, so I can't record anything (and yes, I tried to record and I only get frozen screen videos).

What version of XWayland are you using, @FastOSlinux? I don't know if XWayland is the source of the problem, but if you can record fullscreen applications and me not, it could be the key component.

I just read on Phoronix about EGL support in XWayland 21.1 (I'm just asking).

I'm using flathub-beta OBS and the same xwayland

as a test, try to enable this:

gsettings set org.gnome.mutter experimental-features "['dma-buf-screen-sharing']"

and restart

[cold-distance]

YEEAAHHHHHHHH !!!!!!!!!!!!!!!!!!

I run the command you wrote:

gsettings set org.gnome.mutter experimental-features "['dma-buf-screen-sharing']"

I restarted and now I can record fullscreen applications.

Thank you, thank you very much.