[BUG] Application freezes after updating flatpak (AMD)

[fiftydinar]

Describe the bug When trying to stop the recording, application completely freezes, no matter which container or recording quality is selected.
I don't get this bug on AMD Vega 7 or RX 5500 GPU.

To Reproduce Start the recording & try to stop it.

Expected behavior Recording should be stopped & saved successfully.

System info

• Wayland
• Gnome 45.5
• Fedora Silverblue 39 (Universal Blue version, so proprietary stuff like mesa-freeworld is installed)
• AMD RX 6600
• Flatpak v3.6.3

Additional context I compared the logs that I got from RX 6600 & RX 5500 GPUs by running flatpak run --verbose & I didn't see anything differing in the logs, except that it did not go past gsr info: gsr_kms_client_init: waiting for server to connect part of the log.

amdgpu: amdgpu_cs_ctx_create2 failed. (-13)
pkexec version 123
/var/home/gidro
amdgpu: amdgpu_cs_ctx_create2 failed. (-13)
[hevc_vaapi @ 0x560ad42ac900] ignoring invalid SAR: 0/0
[hevc_vaapi @ 0x560ad42ac900] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
Info: using hevc encoder because a codec was not specified
gsr info: gsr_kms_client_init: waiting for server to connect

OBS GPU-accelerated recording works.

[dec05eba]

Can you run flatpak run --command=bash com.dec05eba.gpu_screen_recorder and then echo $HOME? it seems like it's a distro specific issue because of extra sandboxing.

[fiftydinar]

Can you run flatpak run --command=bash com.dec05eba.gpu_screen_recorder and then echo $HOME? it seems like it's a distro specific issue because of extra sandboxing.

I forgot to mention that RX 5500 & Vega 7 where recording is successful use the same distro.

But I will quickly come back with the output of your command.

Output is /home/gidro, which is a symlink to /var/home/gidro in Fedora Silverblue. It looks good.

[dec05eba]

Hmm, can you run flatpak run --command=bash com.dec05eba.gpu_screen_recorder and inside that shell run echo "world" > hello, then inside another terminal shell run sudo cat $HOME/hello. What does it output?

From the terminal output it should be unrelated to the gpu you are using.

[fiftydinar]

Hmm, can you run flatpak run --command=bash com.dec05eba.gpu_screen_recorder and inside that shell run echo "world" > hello, then inside another terminal shell run sudo cat $HOME/hello. What does it output?

Output is world

Looks also fine.

[dec05eba]

How long time did you wait at the gsr info: gsr_kms_client_init: waiting for server to connect part?

[fiftydinar]

How long time did you wait at the gsr info: gsr_kms_client_init: waiting for server to connect part?

Around 15 seconds is the longest, before I confirmed Gnome's force close dialog.

[dec05eba]

Can you run ls -lah $HOME/.local/share/gpu-screen-recorder/kms-server-proxy and tell me what it outputs?

[fiftydinar]

Can you run ls -lah $HOME/.local/share/gpu-screen-recorder/kms-server-proxy and tell me what it outputs?

-rwxr-xr-x. 1 root root 677K 17. mar. у 19:53 /home/gidro/.local/share/gpu-screen-recorder/kms-server-proxy

[dec05eba]

Hmm, it's a bit weird. Can you run running sudo getcap /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server ? and tell me the output? and then reinstall gpu screen recorder and sudo rm $HOME/.local/share/gpu-screen-recorder/kms-server-proxy before trying again?

[fiftydinar]

Hmm, it's a bit weird. Can you run running sudo getcap /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server ? and tell me the output?

Output is nothing.

and then reinstall gpu screen recorder and sudo rm $HOME/.local/share/gpu-screen-recorder/kms-server-proxy before trying again?

Now it works!

The output after this with getcap is: /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server cap_sys_admin=ep

I wonder if this happened to me when GPU Screen Recorder migrated from GUI setting to not ask polkit again about the password, to proper one-time automatic password prompt?

[dec05eba]

Its something related to that yes. Can you try recording again? the first time you record it runs a different part of the code. If it works, can you run sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server and try recording again? if that breaks it then it will be broken after an update :/

[fiftydinar]

Can you try recording again? the first time you record it runs a different part of the code. If it works, can you run sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server

Recording is successful, output is nothing.

and try recording again? if that breaks it then it will be broken after an update :/

Ah, yes. Freeze happens again.

[dec05eba]

Ok it seems to indeed be a distro specific permissions thing then I believe?. Finally, can you run this sudo getcap $HOME/.local/share/gpu-screen-recorder/kms-server-proxy ?

[fiftydinar]

Ok it seems to indeed be a distro specific permissions thing then I believe?. Finally, can you run this sudo getcap $HOME/.local/share/gpu-screen-recorder/kms-server-proxy ?

Output: /home/gidro/.local/share/gpu-screen-recorder/kms-server-proxy cap_sys_admin,cap_setfcap=ep

[dec05eba]

Ok thats weird, everything looks the same as on arch where it works. Can you try running sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server, then $HOME/.local/share/gpu-screen-recorder/kms-server-proxy /tmp/abc /dev/dri/blabla "$HOME", if it doesn't output anything run echo $?. Then run sudo getcap /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server.

[fiftydinar]

Ok thats weird, everything looks the same as on arch where it works. Can you try running sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server

sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server
[sudo] password for gidro: 
Failed to set capabilities on file `/var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server' (No data available)

[dec05eba]

i think that error is fine, can you also run the other commands after that?

[fiftydinar]

i think that error is fine, can you also run the other commands after that?

Output was nothing on 1st command, so I had to close the process with Ctrl + C

gidro@gidro-fedora:~$ $HOME/.local/share/gpu-screen-recorder/kms-server-proxy /tmp/abc /dev/dri/blabla "$HOME"
^C
gidro@gidro-fedora:~$ echo $?
130
gidro@gidro-fedora:~$ sudo getcap /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server

[dec05eba]

ok so it froze on the first command? that would explain the issue

[fiftydinar]

ok so it froze on the first command? that would explain the issue

Yes

Feel free to rename the issue title, to reflect the issue more accurately

[dec05eba]

Ok I think I know where the issue is but I dont know why it would behave like that on a different distro. I know that people have recorded on fedora silverblue before without this issue (unless they got the issue later but haven't reported anything). I think I will need to test it on fedora silverblue 39. Thanks.

But can you try one final thing, run readlink /proc/self/exe

[fiftydinar]

But can you try one final thing, run readlink /proc/self/exe

/usr/bin/readlink

[dec05eba]

Oh, universal blue is a separate thing from official fedora packages. They have several different distributions on their website https://universal-blue.org/, in this the one you are using: https://bazzite.gg/ ?

[fiftydinar]

Oh, universal blue is a separate thing from official fedora packages. They have several different distributions on their website universal-blue.org, in this the one you are using: bazzite.gg ?

To be more precise, I use my own custom image: https://github.com/fiftydinar/gidro-os

This is what Universal Blue builds as base images on top of Fedora (my image, Bazzite & Bluefin are based on it): https://github.com/ublue-os/main

You can see packages.json for what is additionaly included in main images.

[dec05eba]

Oh I see the issue now. It's most likely the symlink from /home/... to /var/home/.. I'll see if I can fix that

[dec05eba]

I got some error when trying to install bazzite on my system. I couldn't bother trying to figure out why (it didn't provide any proper error message) so im going with that my prognosis of the issue is correct instead.

Can you try installing the flatpak update before its merged to see if that really was the issue? run flatpak uninstall com.dec05eba.gpu_screen_recorder then flatpak install --system https://dl.flathub.org/build-repo/95531/com.dec05eba.gpu_screen_recorder.flatpakref and then run sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server and then start gpu screen recorder and record (a password prompt should show up again). After that try recording to see that it works again. Then run sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server and try recording again.

[fiftydinar]

run flatpak uninstall com.dec05eba.gpu_screen_recorder then flatpak install --system https://dl.flathub.org/build-repo/95531/com.dec05eba.gpu_screen_recorder.flatpakref

Done.

then run sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server and then start gpu screen recorder and record (a password prompt should show up again).

Recording works.

After that try recording to see that it works again.

Recording works.

Then run sudo setcap -r /var/lib/flatpak/app/com.dec05eba.gpu_screen_recorder/current/active/files/bin/gsr-kms-server and try recording again.

Recording works.

[dec05eba]

Thanks for testing, I'll likely merge the fix later today