[Linux] RTSP handshake fails when run as user

Is there an existing issue for this?

[X] I have searched the existing issues

Is your issue described in the documentation?

[X] I have read the documentation

Is your issue present in the latest beta/pre-release?

Yes

Describe the Bug

Everything works as expected when running sunshine as root. However, when running sunshine as a user, moonlight is not able to connect due to "Failed to start RTSP handshake (error 110)". Sunshine then dumps its core after detecting a hang. There is no additional info in the logs, except for the ping timeout. Note that I have enabled permissions for KMS capture.

Expected Behavior

I expect a stream to be established successfully.

Additional Context

I am running Plasma 6 using Wayland.

The pairing doesn't work as smooth as when running as root either. The pairing occasionally requires the PIN to be put in more than once. It's almost like there is a networking issue, but there is no reason for one.

Host Operating System

Linux

Operating System Version

Arch Linux (kernel 6.9.3-arch1-1)

Architecture

64 bit

Sunshine commit or version

v0.23.1

Package

Arch linux package (sunshine.pkg.tar.zst)

GPU Type

AMD

GPU Model

Radeon RX 6700

GPU Driver/Mesa Version

mesa 1:24.1.2-1

Capture Method

KMX (Linux)

Config

origin_web_ui_allowed = wan
ping_timeout = 1000

Apps

No response

Relevant log output

[2024:06:30:18:32:35]: Info: Screencasting with KMS
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:35]: Info: Encoder [nvenc] failed
[2024:06:30:18:32:35]: Info: Trying encoder [vaapi]
[2024:06:30:18:32:35]: Info: Screencasting with KMS
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:35]: Info: Found connector ID [112]
[2024:06:30:18:32:35]: Info: Found cursor plane [87]
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [h264_vaapi @ 0x5efa14772400] No usable encoding entrypoint found for profile VAProfileH264High (7).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [h264_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Warning: [h264_vaapi @ 0x5efa14a5ef80] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [hevc_vaapi @ 0x5efa14a5ef80] No usable encoding entrypoint found for profile VAProfileHEVCMain (17).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Warning: [hevc_vaapi @ 0x5efa14772400] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa14772400] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa14a5ef80] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:35]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:35]: Info: Color depth: 8-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa14772400] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Error: [av1_vaapi @ 0x5efa152dec40] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:35]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:35]: Info: Screencasting with KMS
[2024:06:30:18:32:35]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:35]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:35]: Info: Found connector ID [112]
[2024:06:30:18:32:35]: Info: Found cursor plane [87]
[2024:06:30:18:32:35]: Info: HDR color coding [Rec. 2020 + SMPTE 2084 PQ]
[2024:06:30:18:32:35]: Info: Color depth: 10-bit
[2024:06:30:18:32:35]: Info: Color range: [JPEG]
[2024:06:30:18:32:35]: Error: [hevc_vaapi @ 0x5efa152dec40] No usable encoding entrypoint found for profile VAProfileHEVCMain10 (18).
[2024:06:30:18:32:35]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:35]: Warning: [hevc_vaapi @ 0x5efa14772400] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:35]: Info: 
[2024:06:30:18:32:35]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:06:30:18:32:35]: Info: 
[2024:06:30:18:32:35]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:06:30:18:32:35]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
[2024:06:30:18:32:35]: Error: Failed to create client: Daemon not running
[2024:06:30:18:32:35]: Info: Configuration UI available at [https://localhost:47990]
[2024:06:30:18:32:41]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:06:30:18:32:41]: Info: Trying encoder [nvenc]
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Error: Couldn't find monitor [0]
[2024:06:30:18:32:41]: Info: Encoder [nvenc] failed
[2024:06:30:18:32:41]: Info: Trying encoder [vaapi]
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:41]: Info: Found connector ID [112]
[2024:06:30:18:32:41]: Info: Found cursor plane [87]
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [h264_vaapi @ 0x77a72c1b5f40] No usable encoding entrypoint found for profile VAProfileH264High (7).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [h264_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Warning: [h264_vaapi @ 0x77a72c3be880] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [hevc_vaapi @ 0x77a72c3be880] No usable encoding entrypoint found for profile VAProfileHEVCMain (17).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Warning: [hevc_vaapi @ 0x77a72c1b5f40] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72c3be880] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72ccff100] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:41]: Info: SDR color coding [Rec. 601]
[2024:06:30:18:32:41]: Info: Color depth: 8-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72ccff100] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Error: [av1_vaapi @ 0x77a72c3be880] No usable encoding entrypoint found for profile VAProfileAV1Profile0 (32).
[2024:06:30:18:32:41]: Error: Could not open codec [av1_vaapi]: Function not implemented
[2024:06:30:18:32:41]: Info: Screencasting with KMS
[2024:06:30:18:32:41]: Info: /dev/dri/card1 -> amdgpu
[2024:06:30:18:32:41]: Info: Found monitor for DRM screencasting
[2024:06:30:18:32:41]: Info: Found connector ID [112]
[2024:06:30:18:32:41]: Info: Found cursor plane [87]
[2024:06:30:18:32:41]: Info: HDR color coding [Rec. 2020 + SMPTE 2084 PQ]
[2024:06:30:18:32:41]: Info: Color depth: 10-bit
[2024:06:30:18:32:41]: Info: Color range: [JPEG]
[2024:06:30:18:32:41]: Error: [hevc_vaapi @ 0x77a72ccff100] No usable encoding entrypoint found for profile VAProfileHEVCMain10 (18).
[2024:06:30:18:32:41]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Function not implemented
[2024:06:30:18:32:41]: Warning: [hevc_vaapi @ 0x77a72c085740] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[2024:06:30:18:32:41]: Info: 
[2024:06:30:18:32:41]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:06:30:18:32:41]: Info: 
[2024:06:30:18:32:41]: Info: Found H.264 encoder: h264_vaapi [vaapi]
[2024:06:30:18:32:41]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
[2024:06:30:18:32:41]: Info: Executing [Desktop]
[2024:06:30:18:32:42]: Error: Initial Ping Timeout
[2024:06:30:18:32:42]: Info: 192.168.1.19: Ping Timeout
[2024:06:30:18:33:16]: Fatal: Hang detected! Session failed to terminate in 10 seconds.
Trace/breakpoint trap (core dumped)

LizardByte / Sunshine