QuickSync and HEVC in UHD 630 not working

[RebelliousX]

Is there an existing issue for this?

• [x] I have searched the existing issues

Is your issue described in the documentation?

• [X] I have read the documentation

Is your issue present in the nightly release?

• [X] This issue is present in the nightly release

Describe the Bug

hevc_qsv not working in Sunshine, but works with other recording applications for example.

Expected Behavior

I expect HEVC to work, I am able to use video capture programs such as Action! and realtime encoding with HEVC codec. I am forced to use H264 which is very unfriendly with bandwidth.

Additional Context

I can't even use NVENC of my 2nd graphics card GTX 1050 TI, I tried to use Nvidia Optimus to change the main graphics to be the GTX 1050TI and according to dxgi-info the first and only display is for the GTX card, but still I can't use NVENC.

Host Operating System

Windows

Operating System Version

Win11 Pro

Architecture

64 bit

Sunshine commit or version

v21 and nightly too.

Package

Windows - installer

GPU Type

Intel

GPU Model

UHD 630 and GTX 1050 TI

GPU Driver/Mesa Version

Intel UHD 630 Driver 31.0.101.2127 and Geforce 551.61

Capture Method (Linux Only)

No response

Config

hevc_mode = 0
resolutions = [
    1280x720,
    1920x1080,
    2560x1080,
    3440x1440,
    1920x1200,
    3840x2160,
    3840x1600
]
upnp = enabled
fec_percentage = 10
sunshine_name = AcerNitro5
min_threads = 3
av1_mode = 1
qsv_preset = fast

Apps

.

Relevant log output

Device Description : Intel(R) UHD Graphics 630
Device Vendor ID   : 0x00008086
Device Device ID   : 0x00003E9B
Device Video Mem   : 128 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 8112 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:25:20:08:46]: Info: Active GPU has HAGS disabled
[2024:02:25:20:08:46]: Info: Using realtime GPU priority
[2024:02:25:20:08:46]: Info: Desktop resolution [1920x1080]
[2024:02:25:20:08:46]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:20:08:46]: Info: Display refresh rate [60.0204Hz]
[2024:02:25:20:08:46]: Info: Requested frame rate [60fps]
[2024:02:25:20:08:46]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 6
Red Primary        : [0.580078,0.349609]
Green Primary      : [0.339844,0.55957]
Blue Primary       : [0.155273,0.125]
White Point        : [0.313477,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:25:20:08:46]: Info: SDR color coding [Rec. 601]
[2024:02:25:20:08:46]: Info: Color depth: 8-bit
[2024:02:25:20:08:46]: Info: Color range: [JPEG]
[2024:02:25:20:08:46]: Error: [hevc_qsv @ 00000177de7ed500] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
[2024:02:25:20:08:46]: Error: Could not open codec [hevc_qsv]: Function not implemented

[cgutman]

[2024:02:25:20:08:46]: Error: [hevc_qsv @ 00000177de7ed500] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
[2024:02:25:20:08:46]: Error: Could not open codec [hevc_qsv]: Function not implemented

Low power encoding is only supported using H.264 on your GPU. This is the case for some older Intel GPUs.

Realtime encoding erformance when not using low power encoding mode is quite poor, so Sunshine is (correctly) opting to use H.264 instead of trying to use HEVC with partial acceleration.

[RebelliousX]

@cgutman thanks for your input. That is weird, I just tried to re-encode an mp4 file (h264) with quicksync using ffmpeg and I get between 90-99FPS (1.9x original speed which is about 50fps), it is a 1080p video too.

ffmpeg -i 1.mp4 -c:v hevc_qsv -c:a copy output.mp4

I wouldn't consider that that weak. I tried Parsec and it allowed HEVC codec and it was smooth ( I didn't get audio, I don't like Parsec though ).

[cgutman]

A few things to keep in mind though:

• Transcoding was probably the only thing your GPU was doing. When you're using Sunshine, your GPU will also be copying frames for capture, running color conversion shaders, and handling output of the game to the display
• You will note significantly higher GPU usage in Task Manager for HEVC without low_power=1 over H.264 with low_power=1. Obviously we want to minimize GPU usage to avoid reducing game performance.
• 1080p at 60 FPS is pretty much the absolute minimum bar for encoding performance. A modern encoder (and probably your encoder with H.264 in low power mode) is able to handle 2-4x the amount of pixels per second to support 1440p60 and 4K60 modes without putting noticable strain on the shader cores (which are needed to render the images we are encoding)

[RebelliousX]

I still want to try it, is there a way to override that setting somewhere, probably command line? As I said, I tried Parsec with HEVC and it was smooth. I tried realtime capture software like Action and records the whole screen fine without sweating 1080p 60fps using HEVC. While gaming, the game is using GTX 1050 Ti anyway.

Edit: Also, I tried nightly after PR #2004, which is supposed to have lenient fallback and try to re-initialize the encoder low_power=0, and still fails. But I didn't see an option in the Web UI for that though.

[cgutman]

There is no way to override the low power mode requirement for HEVC in config today. In the past, we required low power for H.264 and would fall back to software encoding if it wasn't supported. Now we allow non-low power H.264 as a fallback, but we always prefer low power H.264 over non-low power HEVC. It would be possible (and easy) to add an option to allow non-low power HEVC as a fallback.

Another thing to realize is smoothness isn't all that matters. Encoding latency is another critical metric for remote gaming. With your FFmpeg command executing at 90-100 FPS, that's an average of 10-11ms ms of encoding latency on every frame. A high performance encoder can average between 2-4x lower latency than those numbers you got.

I can't speak for Parsec and how they are configuring then encoder (or even which encoder they are using) or other screen recording software. Different software makes different tradeoffs that work differently for different configurations. Your setup happens to be one where non-low power HEVC can be beneficial (bandwidth constrained, no GPU load, relative low resolution and frame rate).

It makes sense to provide the option for HEVC, but the default will probably remain as it is today preferring H.264 for that scenario.

[RebelliousX]

The log of the nightly outright says HEVC not supported which is not true. I still wish if there is an option to allow it even if it is thought to have bad performance. Or a way to allow the encoder of the secondary dGPU (in my case, NVENC) even when it is not assigned for the main display, dxgi-info after I changed the prefered GPU to High Performance in Nvidia Control Panel, shows the Geforce as the one assigned with the display but still fails to initialize the Nvidia encoder .

Device Description : Intel(R) UHD Graphics 630
Device Vendor ID   : 0x00008086
Device Device ID   : 0x00003E9B
Device Video Mem   : 128 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 8112 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:26:03:24:06]: Info: Active GPU has HAGS disabled
[2024:02:26:03:24:06]: Info: Using realtime GPU priority
[2024:02:26:03:24:06]: Info: Desktop resolution [1920x1080]
[2024:02:26:03:24:06]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:26:03:24:06]: Info: Display refresh rate [60.0204Hz]
[2024:02:26:03:24:06]: Info: Requested frame rate [60fps]
[2024:02:26:03:24:06]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 6
Red Primary        : [0.580078,0.349609]
Green Primary      : [0.339844,0.55957]
Blue Primary       : [0.155273,0.125]
White Point        : [0.313477,0.329102]
Min Luminance      : 0.5 nits
Max Luminance      : 270 nits
Max Full Luminance : 270 nits
[2024:02:26:03:24:06]: Warning: Encoder [quicksync] does not support HEVC on this system
[2024:02:26:03:24:06]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:02:26:03:24:06]: Info: 
[2024:02:26:03:24:06]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:02:26:03:24:06]: Info: 
[2024:02:26:03:24:06]: Info: Found H.264 encoder: h264_qsv [quicksync]
[2024:02:26:03:24:06]: Info: Executing [Desktop]
[2024:02:26:03:24:07]: Info: nvprefs: No need to modify application profile settings
[2024:02:26:03:24:07]: Info: nvprefs: No need to modify global profile settings
[2024:02:26:03:24:07]: Info: WLAN interface 0 is now in low latency mode
[2024:02:26:03:24:07]: Info: CLIENT CONNECTED
[2024:02:26:03:24:08]: Info: 
Device Description : Intel(R) UHD Graphics 630
Device Vendor ID   : 0x00008086
Device Device ID   : 0x00003E9B
Device Video Mem   : 128 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 8112 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2024:02:26:03:24:08]: Info: Active GPU has HAGS disabled
[2024:02:26:03:24:08]: Info: Using realtime GPU priority
[2024:02:26:03:24:08]: Info: Desktop resolution [1920x1080]
[2024:02:26:03:24:08]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:26:03:24:08]: Info: Display refresh rate [60.0204Hz]
[2024:02:26:03:24:08]: Info: Requested frame rate [60fps]

Edit: And I don't understand why on earth the settings for the encoders are baked in the CPP file and not in a JSON file? Every time there is a change in the settings a new project build is needed. This is bad in my opinion. https://github.com/cgutman/LB_Sunshine/blob/25745ae3ea9d9dee4aa8d8aa0e892792636c41a1/src/video.cpp

Also, if HEVC would be so slow and that is why it is not enabled, I don't think the Software encoder (h264) would be faster that the hardware encoder QuickSync (HEVC).

[cgutman]

The log of the nightly outright says HEVC not supported which is not true.

We don't control what the log message says. It is coming directly from FFmpeg.

I still wish if there is an option to allow it even if it is thought to have bad performance.

I agree. I'm implementing the option in #2181.

Here's a test build with the option present: https://github.com/LizardByte/Sunshine/actions/runs/8059745652/artifacts/1277594466

FWIW, on my i5 6400T test system, using non-lowpower HEVC results in 3D engine GPU usage peaks of 90% and 40 ms average of encoding latency (compared with 10ms on H.264 and 20% GPU usage) when streaming 2560x1440 60 FPS at 40 Mbps. So our fears driving the conservative H.264 default do indeed hold true, at least on this hardware.

Or a way to allow the encoder of the secondary dGPU (in my case, NVENC) even when it is not assigned for the main display, dxgi-info after I changed the prefered GPU to High Performance in Nvidia Control Panel, shows the Geforce as the one assigned with the display but still fails to initialize the Nvidia encoder .

This is a limitation of the DXGI Desktop Duplication API per Microsoft's documentation. There is work progressing to implement a capture mechanism that can perform cross-adapter capture which should land shortly after v0.22.0.

The Nvidia control panel is changing the render GPU used by applications. It doesn't change which output the GPU is physically connected to (which can't be done without a hardware mux).

Edit: And I don't understand why on earth the settings for the encoders are baked in the CPP file and not in a JSON file? Every time there is a change in the settings a new project build is needed. This is bad in my opinion. https://github.com/cgutman/LB_Sunshine/blob/25745ae3ea9d9dee4aa8d8aa0e892792636c41a1/src/video.cpp

The encoder options that are intended to be user configurable are exposed in the UI and config.json (performance/quality presets, rate control, etc), . Things like async_depth, forced_idr, max_dec_frame_buffering, and most of the other hardcoded options there are critical encoder parameters that are not user-adjustible because they would cause significant performance or correctness issues in Sunshine. For example, some encoders need special options to produce full IDR frames instead of regular I-frames or to omit lengthy SEI NAL units. Without these options, Sunshine would break the GameStream protocol guarantees with Moonlight regarding IDR frames for packet loss recovery.

I agree that it would be useful to accept arbitrary options (with major warnings, since some can break encoding entirely). We generally try to keep some guardrails to prevent users from breaking things too badly from the UI, but it might be worth doing as a config.json option.

Also, if HEVC would be so slow and that is why it is not enabled, I don't think the Software encoder (h264) would be faster that the hardware encoder QuickSync (HEVC).

Correct, it isn't. The fact that we entirely failed QuickSync init and fell back to libx264 in that case was wrong and is already fixed.

[RebelliousX]

@cgutman Much appreciated, thanks a lot, I will test right away. Initial test on my phone it is working good, I will test the performance on my Shield TV. 👍

[RebelliousX]

Update, working well.. Better than software encoding. 1080p with 25Mb bandwidth, recording while playing (on Shield TV), game is installed on HDD not SSD thus I believe it gets spike framerates while loading some parts because of that.

The CPU is i5-8300H - iGPU UHD 630.

I believe I can do 720p 60fps with lower bandwidth like 8Mb or lower, hopefully that will get rid of the random stutter I used to get, I will test more later. But so far so good.

https://github.com/LizardByte/Sunshine/assets/1888186/92c74263-7d13-4594-a65a-5907c659a926

(I had to re-encode the file again, it was recorded as H264 ~55MB, now with H265 around 5MB 😊file limit on Github for videos is 10MB).. Testing preview, Github Firefox doesn't like HEVC (no MIME supported).

Here is a link to YouTube

Oh, if you hear some clicking noise in the recording because I am using Shield TV Android 9, that was fixed in Android 11 firmware but I refuse to upgrade 😊That noise doesn't exist while playing, and it has nothing to do with this new option you made.

[RebelliousX]

@cgutman Another one, but this time, I set QuickSync profile to fastest, 1080p at 8Mbps. At least for me the quality seems to be better than 1080p 20Mbps encoding in H264, I could be wrong. But I saw less distorted colors in distances or when looking closely at flames for example.

Latency is really negligible too. I played for like an hour and the average "Latency" message bubble notification when done streaming was saying 4ms encode and 1ms decode HEVC. H265 (HEVC) is definitely slower than AVC (H264), but when I set the profile to fastest encoding, it turns out not that much. It becomes severe difference when the encoding profile is set to more demanding profile than the fastest.

Benefit of this, better quality at lower bandwidth, less traffic in congested network (when all family members or devices are using the internet heavily for example), seldomly see the "Lower your video bandwidth" message appearing randomly due to previous reason. Only disadvantage is, it is slower, but when set to fastest encoding, not by much.