LizardByte / Sunshine

Self-hosted game stream host for Moonlight.
http://app.lizardbyte.dev/Sunshine/
GNU General Public License v3.0
19.94k stars 959 forks source link

[Linux] RTSP handshake fails when run as user #2778

Open ptrsr opened 4 months ago

ptrsr commented 4 months ago

Is there an existing issue for this?

Is your issue described in the documentation?

Is your issue present in the latest beta/pre-release?

Yes

Describe the Bug

Everything works as expected when running sunshine as root. However, when running sunshine as a user, moonlight is not able to connect due to "Failed to start RTSP handshake (error 110)". Sunshine then dumps its core after detecting a hang. There is no additional info in the logs, except for the ping timeout. Note that I have enabled permissions for KMS capture.

Expected Behavior

I expect a stream to be established successfully.

Additional Context

I am running Plasma 6 using Wayland.

The pairing doesn't work as smooth as when running as root either. The pairing occasionally requires the PIN to be put in more than once. It's almost like there is a networking issue, but there is no reason for one.

Host Operating System

Linux

Operating System Version

Arch Linux (kernel 6.9.3-arch1-1)

Architecture

64 bit

Sunshine commit or version

v0.23.1

Package

Arch linux package (sunshine.pkg.tar.zst)

GPU Type

AMD

GPU Model

Radeon RX 6700

GPU Driver/Mesa Version

mesa 1:24.1.2-1

Capture Method

KMX (Linux)

Config

origin_web_ui_allowed = wan
ping_timeout = 1000

Apps

No response

Relevant log output

[2024:06:30:18:32:35]: Info: Screencasting with KMS
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:35]: Info: Encoder [nvenc] failed
[2024:06:30:18:32:35]: Info: Trying encoder [vaapi]
[2024:06:30:18:32:35]: Info: Screencasting with KMS
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:35]: Info: Found connector ID [112]
[2024:06:30:18:32:35]: Info: Found cursor plane [87]
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [h264_vaapi @ 0x5efa14772400] No usable encoding entrypoint found for profile VAProfileH264High (7).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [h264_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Warning: [h264_vaapi @ 0x5efa14a5ef80] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [hevc_vaapi @ 0x5efa14a5ef80] No usable encoding entrypoint found for profile VAProfileHEVCMain (17).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Warning: [hevc_vaapi @ 0x5efa14772400] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa14772400] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa14a5ef80] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa14772400] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa152dec40] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:35]: Info: Screencasting with KMS
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:35]: Info: Found connector ID [112]
[2024:06:30:18:32:35]: Info: Found cursor plane [87]
[2024:06:30:18:32:35]: Info: HDR color coding [Rec. 2020 + SMPTE 2084 PQ]
[2024:06:30:18:32:35]: Info: Color depth: 10-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [hevc_vaapi @ 0x5efa152dec40] No usable encoding entrypoint found for profile VAProfileHEVCMain10 (18).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Warning: [hevc_vaapi @ 0x5efa14772400] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:35]: Info: 
[2024:06:30:18:32:35]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:06:30:18:32:35]: Info: 
[2024:06:30:18:32:35]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:06:30:18:32:35]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
[2024:06:30:18:32:35]: Error: Failed to create client: Daemon not running
[2024:06:30:18:32:35]: Info: Configuration UI available at [https://localhost:47990]
[2024:06:30:18:32:41]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:06:30:18:32:41]: Info: Trying encoder [nvenc]
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: Encoder [nvenc] failed
[2024:06:30:18:32:41]: Info: Trying encoder [vaapi]
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:41]: Info: Found connector ID [112]
[2024:06:30:18:32:41]: Info: Found cursor plane [87]
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [h264_vaapi @ 0x77a72c1b5f40] No usable encoding entrypoint found for profile VAProfileH264High (7).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [h264_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Warning: [h264_vaapi @ 0x77a72c3be880] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [hevc_vaapi @ 0x77a72c3be880] No usable encoding entrypoint found for profile VAProfileHEVCMain (17).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Warning: [hevc_vaapi @ 0x77a72c1b5f40] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72c3be880] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72ccff100] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72ccff100] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72c3be880] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:41]: Info: Found connector ID [112]
[2024:06:30:18:32:41]: Info: Found cursor plane [87]
[2024:06:30:18:32:41]: Info: HDR color coding [Rec. 2020 + SMPTE 2084 PQ]
[2024:06:30:18:32:41]: Info: Color depth: 10-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [hevc_vaapi @ 0x77a72ccff100] No usable encoding entrypoint found for profile VAProfileHEVCMain10 (18).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Warning: [hevc_vaapi @ 0x77a72c085740] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:41]: Info: 
[2024:06:30:18:32:41]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:06:30:18:32:41]: Info: 
[2024:06:30:18:32:41]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:06:30:18:32:41]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
[2024:06:30:18:32:41]: Info: Executing [Desktop]
[2024:06:30:18:32:42]: Error: Initial Ping Timeout
[2024:06:30:18:32:42]: Info: 192.168.1.19: Ping Timeout
[2024:06:30:18:33:16]: Fatal: Hang detected! Session failed to terminate in 10 seconds.
Trace/breakpoint trap (core dumped)
ReenigneArcher commented 4 months ago

Test the pre-release version...

ptrsr commented 4 months ago

Done, same issue persists.

gschintgen commented 4 months ago

Why did you reduce ping_timeout by a factor of 10?

Sunshine should terminate gracefully though.

ptrsr commented 4 months ago

I've tried setting it to 1000ms and 10000ms, it does not change anything. I feel like there is something weird going on regarding the UDP/TCP sockets in user mode. The PIN input has a lot of timeouts as well.

Is there a way to get sunshine to connect with pulseaudio in root mode? I have tried running pulse audio as root but I still get access denied errors when the stream has started.

ptrsr commented 4 months ago

As a workaround, I start Sunshine as root (sudo sunshine). This allows screen capture to work. However, audio capture failed as users can not connect to the PulseAudio instance of other users by default.

To fix this, create a .pulse folder in the home folder of the user;

mkdir ~/.pulse

And copy the default PulseAudio config;

cp /etc/pulse/default.pa ~/.pulse/

And add the following line to the ~/.pulse/default.pa file to allow TCP local connections;

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1

Restart pulseaudio;

pulseaudio -k
pulseaudio -D

and run sunshine using;

sudo PULSE_SERVER=tcp:127.0.0.1:4713 sunshine

Now both video and audio should work. TCP does have some overhead though, so instead I would recommend using unix sockets. I have yet to try this myself, though.

gschintgen commented 4 months ago

For the pairing: Do you have to enter the PIN multiple times for the same connection attempt? If not, could it simply be the result of connecting once to regular user-sunshine and then connecting to root-sunshine? Both have their separate configs, so it would be expected behaviour to do the initial pairing twice (once for each sunshine instance).

With the binary packages there should be no need to do any manual setup steps (like setcap). There should also be no reason at all to run Sunshine as root. (I know, "should". Apparently it does not work as intended on your system.)

As for your workaround, you could probably just reuse the command documented for the flatpak (which does need root for KMS capture):

sudo -i PULSE_SERVER=unix:$(pactl info | awk '/Server String/{print$3}') sunshine
Theslees commented 3 months ago

try switching to x11 @ptrsr if you're on wayland

svenvvv commented 3 months ago

I encountered a very similar problem under X11 running with AMD GPU on Arch Linux. Pairing took a lot of time as the requests kept timing out and I wasn't able to start a stream at all (RTSP handshare error 110). I did not try running as root. I did a bit of debugging and for me the cause of the problem was a deadlock in the tray icon library used in Sunshine.

Try building the sunshine/sunshine-git package from AUR and edit the PKGBUILD to disable tray icon during building. This fixed it for me. E.g. add these to CMake configure arguments in the PKGBUILD build() command:

    -D SUNSHINE_ENABLE_TRAY=0 \
    -D SUNSHINE_REQUIRE_TRAY=0 \

I don't really care for the tray icon, so I'm not going to debug it any further and just disable it, but in case someone wants to debug it then this bug was most likely already present to some degree in sunshine 0.21. Pairing and connecting used to work, but I remember having Sunshine crash fairly often with the "hang detected" message after I quit Moonlight.

MichaelThomas0721 commented 3 months ago

Thank you @svenvvv , your solution worked for me as well. System: hyprland (wayland) on arch with nvidia gpu

Djagur commented 1 month ago

My thanks as well @svenvvv, your approach helped me to fix an issue that I had been struggling with for weeks. I'm on Wayland Fedora 40, nvidia

schindlershadow commented 2 weeks ago

I can confirm that the bug was introduced in v0.21.0. I downgraded to v0.20.0 and its working

ReenigneArcher commented 2 weeks ago

@schindlershadow While I understand you are facing an issue, that's really not a good idea. https://github.com/LizardByte/Sunshine/security/advisories

schindlershadow commented 2 weeks ago

Thanks for letting me know. I'll close off the open ports on my fw and leave it LAN only for now until this is fixed