Sunshine (git master) fails to detect display and encoder on nvidia + wayland

[rubin55]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your issue described in the documentation?

• [X] I have read the documentation

Is your issue present in the latest beta/pre-release?

This issue is present in the latest pre-release

Describe the Bug

My hardware: NVIDIA 4090, Viewsonic VX1655-4K-OLED, Dell UP3218k My software: Linux 6.9.9, Gnome + Wayland, NVIDIA drivers 555.28.02

Sunshine compiled from master git branch. Verified working on Xorg on the same hardware + software.

When I go to the UI, to Configuration, I see the following detected display information (I noticed that the logs do not show detected display information which I found curious):

Info: Detecting displays
              Info: Detected display: DVI-D-0 (id: 0) connected: false
              Info: Detected display: HDMI-0 (id: 1) connected: true
              Info: Detected display: DP-0 (id: 2) connected: true
              Info: Detected display: DP-1 (id: 3) connected: false
              Info: Detected display: DVI-D-1 (id: 4) connected: false

DP-0 is (probably) the Dell UP3218k, HDMI-0 is the Viewsonic 4k. It doesn't seem to make a difference if I set these display id's in the config file, I keep getting the same message:

Unable to find display or encoder during startup.

Expected Behavior

I expect Sunshine to work using wayland on my setup, just as it does using Xorg.

Additional Context

I do have Sunshine working on a laptop with Wayland, but it's not a 4K monitor and it doesn't have multiple displays. Maybe that matters.

Host Operating System

Linux

Operating System Version

6.9.9

Architecture

64 bit

Sunshine commit or version

f3a164e95ecd7778ddb71823f703a76769e1c150

Package

other (self built)

GPU Type

Nvidia

GPU Model

4090

GPU Driver/Mesa Version

555.28.02

Capture Method

NvFBC (Linux)

Config

resolutions = [
    1280x720,
    1920x1080,
    1920x1200,
    2256x1504,
    2880x1920,
    3840x2160,
    7680x4320
]
fps = [30,60]
origin_web_ui_allowed = pc
output_name = 1
channels = 2
capture = nvfbc
encoder = nvenc
nv_preset = default
nvenc_preset = 7

Apps

No response

Relevant log output

[nv_preset] -- [default]
[encoder] -- [nvenc]
[capture] -- [nvfbc]
[channels] -- [2]
[output_name] -- [1]
[origin_web_ui_allowed] -- [pc]
[nvenc_preset] -- [7]
[fps] -- [[30,60]]
[resolutions] -- [[
    1280x720,
    1920x1080,
    1920x1200,
    2256x1504,
    2880x1920,
    3840x2160,
    7680x4320
]]
Warning: Unrecognized configurable option [nv_preset]
[2024:07:12:20:43:14]: Info: Sunshine version: 0.0.0
[2024:07:12:20:43:14]: Info: System tray created
[2024:07:12:20:43:14]: Error: Failed to create session:
[2024:07:12:20:43:14]: Error: Unable to initialize capture method
[2024:07:12:20:43:14]: Error: Platform failed to initialize
[2024:07:12:20:43:14]: Info: Trying encoder [nvenc]

(sunshine:40658): Gtk-CRITICAL **: 20:43:14.399: gtk_widget_get_scale_factor: assertion 'GTK_IS_WIDGET (widget)' failed
[2024:07:12:20:43:14]: Info: Encoder [nvenc] failed
[2024:07:12:20:43:14]: Error: Couldn't find any working encoder matching [nvenc]
[2024:07:12:20:43:14]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:07:12:20:43:14]: Info: Trying encoder [vaapi]
[2024:07:12:20:43:15]: Info: Encoder [vaapi] failed
[2024:07:12:20:43:15]: Info: Trying encoder [software]
[2024:07:12:20:43:15]: Info: Encoder [software] failed
[2024:07:12:20:43:15]: Fatal: Unable to find display or encoder during startup.
[2024:07:12:20:43:15]: Fatal: Please ensure your manually chosen GPU and monitor are connected and powered on.
[2024:07:12:20:43:15]: Error: Video failed to find working encoder
[2024:07:12:20:43:15]: Error: Failed to create client: Daemon not running
[2024:07:12:20:43:15]: Info: Configuration UI available at [https://localhost:47990]

[rubin55]

I just re-checked with latest commit on master (f3a164e95ecd7778ddb71823f703a76769e1c150), just to be sure, but same thing happens. Let me know if I can do something to maybe get more relevant log output.

[rubin55]

I noted that #1631 is similar error-wise, but @cgutman indicates that it's likely another issue and a new issue should be filed.

[ReenigneArcher]

Please share cmake and build logs

[rubin55]

I collected the stdout/err in build-output.log, got .ninja.log from the Cmake build_dir. build-output.log ninja.log

Is this helpful?

[ReenigneArcher]

Yes, although I don't see anything obvious that would cause the issue.

[gschintgen]

When I go to the UI, to Configuration, I see the following detected display information (I noticed that the logs do not show detected display information which I found curious):

Info: Detecting displays
              Info: Detected display: DVI-D-0 (id: 0) connected: false
              Info: Detected display: HDMI-0 (id: 1) connected: true
              Info: Detected display: DP-0 (id: 2) connected: true
              Info: Detected display: DP-1 (id: 3) connected: false
              Info: Detected display: DVI-D-1 (id: 4) connected: false

This is an explanatory example output only. My system shows the exact same text in the WebUI. The wording in the UI is quite misleading. (I also mistook it for actually detected displays.)

[rubin55]

@ReenigneArcher is there a way I can get more logging? I tried min_log_level = debug but it didn't give me significantly more information about why Sunshine seems to be unable to initialize the encoders and/or displays. When I run in X11 mode, sunshine works as expected, fwiw.

[rubin55]

Update: I think it is due to NvFBC being X11 only. Once I removed nvfbc from my config, sunshine correctly picks up the display. Source: https://github.com/LizardByte/Sunshine/discussions/2060