devusr1x
commented
8 months ago Related: #2056
Closed devusr1x closed 8 months ago
devusr1x
commented
8 months ago Related: #2056
cgutman
commented
8 months ago Your GPU driver is quite old. Please try updating to the latest driver (551.23) and see if that resolves the issue.
devusr1x
commented
8 months ago I just tried with 551.52 (released yesterday) and it indeed seems to work properly. I was able to get a HEVC/HDR feed with Moonlight and the GPU encoder was being used.
But the issue is - the reason why I was using 516.93 in the first place is because I use this machine for remote video editing (Premiere Pro). Ever since the changes introduced in 522.30 (see changelog at https://www.nvidia.com/download/driverResults.aspx/193884/en-us/, they mention Adobe products specifically), video rendering with NVENC will instantly nuke and BSOD the entire host system (as in, the actual physical machine hosting the VM).
I'll just stick with 0.20 for now, until I upgrade to the 4000 series. I assume both Sunshine and Premiere Pro are working properly with these newer cards and latest drivers.
You could close the issue if you wish, but from a user experience perspective, this was pretty awful, and it is odd that I can use older drivers + NVENC with both Sunshine 0.20 and Parsec.
devusr1x
commented
8 months ago I ran a few more tests since I had everything already open, and it seems that my guess was right - 522.30 is the exact version where the issue starts to appear, and it somehow coincides with the Premiere Pro BSOD issues. Here is what I've learned, in case this helps:
ReenigneArcher
commented
8 months ago it is odd that I can use older drivers + NVENC with both Sunshine 0.20 and Parsec
Standalone NVENC encoder was added in v0.21.0... so if you're using v0.20.0 it's not using the same encoders.
devusr1x
commented
8 months ago it is odd that I can use older drivers + NVENC with both Sunshine 0.20 and Parsec
Standalone NVENC encoder was added in v0.21.0... so if you're using v0.20.0 it's not using the same encoders.
Yes, I understand, but that's exactly the issue (hence the issue title).
I also got a bit confused by the wording since it is a bunch of software versions and combinations (plus the issue might or might not be related to GPU passthrough), so let me just write the matrix here (this is specifically under Hyper-V GPU passthrough):
Just to be even more clear, I want the combination of Premiere Pro: NVENC works and Sunshine: NVENC works, which is only possible with 517.40 and earlier and Sunshine 0.20 and earlier right know (or any recent Parsec version). I can't upgrade the drivers since it would crash my system, so right now (and possibly for the future, in case this isn't addressed) I'm restricted to Sunshine 0.20 and earlier.
Again, for my own daily life I will just stick with 0.20 since this GPU can't benefit from 0.21 AV1 support, so personally that's a non-issue for me right now.
However, even though this looks like an edge case, you might still to restore support for NVENC + older drivers with the new encoder. Here's why - I found this today while investigating the issue:
https://github.com/jamesstringerparsec/Easy-GPU-PV/issues/352
This user was able to BSOD his host system with Win11/546.33/GTX 3060 when running a simple program from the NVIDIA SDK, so unfortunately the BSOD issue might be present in newer cards as well. I suspect Premiere Pro is also BSOD'ing the system when calling the same functions on newer drivers.
This indicates that this is an issue with drivers 522.30 and later and NVIDIA GPU passthrough in general, so users might want to stick with older drivers in such cases.
Considering this, restoring NVENC support for older drivers on versions 0.21+ seems to actually be relevant and useful. It would be pretty disappointing to upgrade to the 3000-4000 series to find out that NVENC might BSOD my system on some programs with newer drivers, and that I can't use AV1 with Sunshine on older drivers.
I wonder if the issue is present on the 4000 series as well (maybe someone could tell us). If it is not, then the issue is solvable by money and buying the latest hardware, which makes it less of an actual issue. If it is still present, then the situation is pretty bad, since this would mean that most of the recent NVIDIA cards have a problem with NVENC+GPU passthrough on Windows (at least) while using newer drivers, possibly due to NVIDIA's own fault, and that Sunshine NVENC AV1 is not usable with older drivers.
For what it's worth, for the Premiere situation I know for a fact that this is related to virtualization. Until last year I was running it on the host system (with newer drivers released in 2023, i.e. later than 522.30) instead of the VM, and never had any BSODs or instability. This issue is very specific to NVIDIA+NVENC+GPU passthrough on Windows.
cgutman
commented
8 months ago The reason for the "breakage" is upgrading of the Nvidia Video Codec SDK from v11.1 to v12.0. It's not really "broken" per se, but the driver version requirement has increased.
The Video Codec SDK explicitly does not maintain backwards compatibility with older driver versions. This is Nvidia's decision, not something we can control and other vendors like AMD and Intel don't do this. You can see the minimum driver versions for v11.1 branch vs 12.0 listed in the readme files there.
Upgrading from Video Codec SDK v11.1 to v12.0 was a requirement to support one of the main new features of Sunshine v0.21 - AV1 support. The reason Parsec doesn't suffer this issue is because they don't support AV1, so they don't need to use the newer Video Codec SDK version.
You could close the issue if you wish, but from a user experience perspective, this was pretty awful, and it is odd that I can use older drivers + NVENC with both Sunshine 0.20 and Parsec.
I agree that silent breakage across a Sunshine update is a really bad user experience. However, based on the fact that this is only the second report of a breakage from a build we shipped 3 months ago which was downloaded over 250K times, I'd say this isn't a major issue in practice.
It may be possible to hack some header files to make Sunshine act like Video Codec SDK v11.1 when initializing for H.264 and HEVC, then only use v12.0 versions when initializing AV1. The Video Codec SDK doesn't officially support this and doesn't make it easy, but it may be possible. At the very least, I can add a banner message in the UI to inform users that they need to update their drivers.
devusr1x
commented
8 months ago Thanks for clarifying, I was unaware of the SDK driver requirements and the introduction of AV1 only in 12.0. Now things are making a bit more sense. I was also assuming that Parsec was using AV1 when they talk about 4:4:4 in their paid plan.
I can't tell you whether playing around with headers is a good idea or not (maybe add an optional fallback mode?), but with the information that you just provided, it seems like there is no solution to getting AV1 support while still using a driver version that doesn't crash the system running Hyper-V, so the only actual functional benefit in doing that would be support for the non-AV1 features introduced beyond 0.21. In other words, I was assuming there was a way to get AV1 on older/stable drivers, which is the most relevant new feature for me, but it seems like it's not possible at all.
Not exactly related to Sunshine, but if anyone finds out why newer NVIDIA drivers are crashing the host system with some NVENC applications under GPU passthrough, and if there's a workaround, please let me know. Other users are encountering similar issues in different applications as per my last post.
cgutman
commented
8 months ago @devusr1x Please try this test build from #2143 which reintroduces compatibility with older drivers: https://github.com/LizardByte/Sunshine/actions/runs/7911611286/artifacts/1247026125
Not exactly related to Sunshine, but if anyone finds out why newer NVIDIA drivers are crashing the host system with some NVENC applications under GPU passthrough, and if there's a workaround, please let me know. Other users are encountering similar issues in different applications as per my last post.
In the past, I've had success filing a formal support ticket with them at https://nvidia.custhelp.com/app/account/overview/. If you have simple steps for them to reproduce the issue, they should be able to look into it.
devusr1x
commented
8 months ago I just tested it with the drivers I had installed (517.40) and it seems to be working fine. All the signals are good (encoder has activity on the host GPU monitor, Moonlight reports HEVC Main 10, low CPU activity, no mouse lag). Thank you very much.
Let met know if testing other driver versions would be useful (I'm guessing it's not).
In the past, I've had success filing a formal support ticket with them at https://nvidia.custhelp.com/app/account/overview/. If you have simple steps for them to reproduce the issue, they should be able to look into it.
I might try to file a bug report later, but I'm not very hopeful since partitioning consumer GPUs is probably not officially supported and they provide some very expensive virtualization solutions for businesses.
rikka0w0
commented
8 months ago Same issue here. Before enabling Hyper-v (hypervisor) inside the Hyper-v guest VM, everything works (sunshine 0.21.0). Here is the log:
[2024:02:25:23:44:07]: Info: nvprefs: Opened undo file from previous improper termination
[2024:02:25:23:44:07]: Info: nvprefs: OGL_CPL_PREFER_DXPRESENT has been changed from our value in base profile, not restoring
[2024:02:25:23:44:07]: Error: nvprefs: NvAPI error: NVAPI_ACCESS_DENIED
[2024:02:25:23:44:07]: Error: nvprefs: NvAPI_DRS_SaveSettings() failed
[2024:02:25:23:44:07]: Error: nvprefs: Failed to restore global profile settings from undo file, deleting the file anyway
[2024:02:25:23:44:07]: Info: nvprefs: Changed PREFERRED_PSTATE to PREFERRED_PSTATE_PREFER_MAX for sunshine.exe
[2024:02:25:23:44:07]: Error: nvprefs: NvAPI error: NVAPI_ACCESS_DENIED
[2024:02:25:23:44:07]: Error: nvprefs: NvAPI_DRS_SaveSettings() failed
[2024:02:25:23:44:07]: Error: nvprefs: Couldn't save application profile settings
[2024:02:25:23:44:07]: Info: nvprefs: Changed OGL_CPL_PREFER_DXPRESENT to OGL_CPL_PREFER_DXPRESENT_PREFER_ENABLED for base profile
[2024:02:25:23:44:07]: Error: nvprefs: NvAPI error: NVAPI_ACCESS_DENIED
[2024:02:25:23:44:07]: Error: nvprefs: NvAPI_DRS_SaveSettings() failed
[2024:02:25:23:44:07]: Error: nvprefs: Couldn't save global profile settings
[2024:02:25:23:44:07]: Info: Sunshine version: 0.21.0
[2024:02:25:23:44:07]: Info: Compiling shaders...
[2024:02:25:23:44:07]: Info: Compiled shaders
[2024:02:25:23:44:07]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:02:25:23:44:07]: Info: Trying encoder [nvenc]
[2024:02:25:23:44:08]: Info: ddprobe.exe [1] [\\.\DISPLAY2] returned: 0x80070005
[2024:02:25:23:44:08]: Info: Set GPU preference: 1
[2024:02:25:23:44:08]: Info:
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002684
Device Video Mem : 24156 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 65389 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 2944x1080
[2024:02:25:23:44:08]: Info: Active GPU has HAGS enabled
[2024:02:25:23:44:08]: Info: Using realtime GPU priority
[2024:02:25:23:44:08]: Info: Desktop resolution [1920x1080]
[2024:02:25:23:44:08]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:23:44:08]: Info: Display refresh rate [59.95Hz]
[2024:02:25:23:44:08]: Info: Requested frame rate [60fps]
[2024:02:25:23:44:08]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 10
Red Primary : [0.694336,0.292969]
Green Primary : [0.274414,0.65918]
Blue Primary : [0.148438,0.0566406]
White Point : [0.313477,0.329102]
Min Luminance : 0.0006 nits
Max Luminance : 1015.27 nits
Max Full Luminance : 603.698 nits
[2024:02:25:23:44:08]: Info:
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002684
Device Video Mem : 24156 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 65389 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 2944x1080
[2024:02:25:23:44:08]: Info: Active GPU has HAGS enabled
[2024:02:25:23:44:08]: Info: Using realtime GPU priority
[2024:02:25:23:44:08]: Info: Desktop resolution [1920x1080]
[2024:02:25:23:44:08]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:23:44:08]: Info: Display refresh rate [59.95Hz]
[2024:02:25:23:44:08]: Info: Requested frame rate [60fps]
[2024:02:25:23:44:08]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 10
Red Primary : [0.694336,0.292969]
Green Primary : [0.274414,0.65918]
Blue Primary : [0.148438,0.0566406]
White Point : [0.313477,0.329102]
Min Luminance : 0.0006 nits
Max Luminance : 1015.27 nits
Max Full Luminance : 603.698 nits
[2024:02:25:23:44:08]: Info: SDR color coding [Rec. 601]
[2024:02:25:23:44:08]: Info: Color depth: 8-bit
[2024:02:25:23:44:08]: Info: Color range: [JPEG]
[2024:02:25:23:44:08]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:02:25:23:44:09]: Info:
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002684
Device Video Mem : 24156 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 65389 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 2944x1080
[2024:02:25:23:44:09]: Info: Active GPU has HAGS enabled
[2024:02:25:23:44:09]: Info: Using realtime GPU priority
[2024:02:25:23:44:09]: Info: Desktop resolution [1920x1080]
[2024:02:25:23:44:09]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:23:44:09]: Info: Display refresh rate [59.95Hz]
[2024:02:25:23:44:09]: Info: Requested frame rate [60fps]
[2024:02:25:23:44:09]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 10
Red Primary : [0.694336,0.292969]
Green Primary : [0.274414,0.65918]
Blue Primary : [0.148438,0.0566406]
White Point : [0.313477,0.329102]
Min Luminance : 0.0006 nits
Max Luminance : 1015.27 nits
Max Full Luminance : 603.698 nits
[2024:02:25:23:44:09]: Info: SDR color coding [Rec. 601]
[2024:02:25:23:44:09]: Info: Color depth: 8-bit
[2024:02:25:23:44:09]: Info: Color range: [JPEG]
[2024:02:25:23:44:09]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:02:25:23:44:09]: Info:
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002684
Device Video Mem : 24156 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 65389 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 2944x1080
[2024:02:25:23:44:09]: Info: Active GPU has HAGS enabled
[2024:02:25:23:44:09]: Info: Using realtime GPU priority
[2024:02:25:23:44:09]: Info: Desktop resolution [1920x1080]
[2024:02:25:23:44:09]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:23:44:09]: Info: Display refresh rate [59.95Hz]
[2024:02:25:23:44:09]: Info: Requested frame rate [60fps]
[2024:02:25:23:44:09]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 10
Red Primary : [0.694336,0.292969]
Green Primary : [0.274414,0.65918]
Blue Primary : [0.148438,0.0566406]
White Point : [0.313477,0.329102]
Min Luminance : 0.0006 nits
Max Luminance : 1015.27 nits
Max Full Luminance : 603.698 nits
[2024:02:25:23:44:09]: Info: SDR color coding [Rec. 601]
[2024:02:25:23:44:09]: Info: Color depth: 8-bit
[2024:02:25:23:44:09]: Info: Color range: [JPEG]
[2024:02:25:23:44:09]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:02:25:23:44:10]: Info:
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002684
Device Video Mem : 24156 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 65389 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 2944x1080
[2024:02:25:23:44:10]: Info: Active GPU has HAGS enabled
[2024:02:25:23:44:10]: Info: Using realtime GPU priority
[2024:02:25:23:44:10]: Info: Desktop resolution [1920x1080]
[2024:02:25:23:44:10]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:23:44:10]: Info: Display refresh rate [59.95Hz]
[2024:02:25:23:44:10]: Info: Requested frame rate [60fps]
[2024:02:25:23:44:10]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 10
Red Primary : [0.694336,0.292969]
Green Primary : [0.274414,0.65918]
Blue Primary : [0.148438,0.0566406]
White Point : [0.313477,0.329102]
Min Luminance : 0.0006 nits
Max Luminance : 1015.27 nits
Max Full Luminance : 603.698 nits
[2024:02:25:23:44:10]: Info: SDR color coding [Rec. 709]
[2024:02:25:23:44:10]: Info: Color depth: 10-bit
[2024:02:25:23:44:10]: Info: Color range: [JPEG]
[2024:02:25:23:44:10]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:02:25:23:44:11]: Info:
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002684
Device Video Mem : 24156 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 65389 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 2944x1080
[2024:02:25:23:44:11]: Info: Active GPU has HAGS enabled
[2024:02:25:23:44:11]: Info: Using realtime GPU priority
[2024:02:25:23:44:11]: Info: Desktop resolution [1920x1080]
[2024:02:25:23:44:11]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:02:25:23:44:11]: Info: Display refresh rate [59.95Hz]
[2024:02:25:23:44:11]: Info: Requested frame rate [60fps]
[2024:02:25:23:44:11]: Info:
Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color : 10
Red Primary : [0.694336,0.292969]
Green Primary : [0.274414,0.65918]
Blue Primary : [0.148438,0.0566406]
White Point : [0.313477,0.329102]
Min Luminance : 0.0006 nits
Max Luminance : 1015.27 nits
Max Full Luminance : 603.698 nits
[2024:02:25:23:44:11]: Info: SDR color coding [Rec. 709]
[2024:02:25:23:44:11]: Info: Color depth: 10-bit
[2024:02:25:23:44:11]: Info: Color range: [JPEG]
[2024:02:25:23:44:11]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:02:25:23:44:11]: Info:
[2024:02:25:23:44:11]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:02:25:23:44:11]: Info:
[2024:02:25:23:44:11]: Info: Found H.264 encoder: h264_nvenc [nvenc]
[2024:02:25:23:44:11]: Info: Found HEVC encoder: hevc_nvenc [nvenc]
[2024:02:25:23:44:11]: Info: Found AV1 encoder: av1_nvenc [nvenc]
[2024:02:25:23:44:11]: Info: Configuration UI available at [https://localhost:47990]
[2024:02:25:23:44:12]: Info: Web UI: [::1] -- not authorized
[2024:02:25:23:44:12]: Info: Registered Sunshine mDNS service
ConteXCrown
commented
6 months ago Tried 551.52, didnt work for me. Didnt want to use 517.40, an driver from 2022. Studio Driver 536.99 works for me and i can play CS2.
sschamp
commented
6 months ago @rikka0w0 did you find a solution? Driver 517.40 does not support RTX 4090, lowest I could find is Driver 522.25 But that already uses NVIDIA Video Codec SDK v12.0 And thus crashes the Host
Is there an existing issue for this?
Is your issue described in the documentation?
Is your issue present in the nightly release?
Describe the Bug
NVENC is not being properly detected on a Hyper-V Windows 11 machine with GPU passthrough:
Expected Behavior
NVENC is properly detected and used.
Additional Context
No response
Host Operating System
Windows
Operating System Version
11 23H2
Architecture
64 bit
Sunshine commit or version
0.20.0 and #69a3edd
Package
Windows - installer
GPU Type
Nvidia
GPU Model
GTX 1060 6GB
GPU Driver/Mesa Version
516.93 (Studio)
Capture Method (Linux Only)
No response
Config
Apps
No response
Relevant log output