Linux NvFBC: high VRAM usage and performance drop

[electronic-van]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your issue described in the documentation?

• [X] I have read the documentation

Is your issue present in the nightly release?

• [X] This issue is present in the nightly release

Describe the Bug

When using NvFBC under Linux, nvidia-smi shows 650MB of VRAM usage at 1080p, and 2GB at 4K. Game performance also drops more than it should, with MangoHud showing more frametime variance and GPU usage around 85-90%.

Expected Behavior

Lower VRAM usage at higher resolutions and less of a performance hit

Additional Context

This is specific to Sunshine and not the Nvidia driver, as gpu-screen-recorder doesn't drop the performance much and uses around 300MB of VRAM at 4K.

I tried both nvidia-utils-nvlax in the AUR and nvidia-patch to enable NvFBC with the same results.

Host Operating System

Linux

Operating System Version

Arch Linux

Architecture

64 bit

Sunshine commit or version

0.18.1

Package

Linux - AUR

GPU Type

Nvidia

GPU Model

GeForce GTX 1660 Ti

GPU Driver/Mesa Version

525.85.05

Capture Method (Linux Only)

NvFBC

Relevant log output

[encoder] -- [nvenc]
[amd_rc] -- [auto]
[audio_sink] -- [sink-sunshine-stereo]
[fps] -- [[10,30,60,90,120]]
[qsv_preset] -- [medium]
[origin_pin_allowed] -- [pc]
[resolutions] -- [[
    352x240,
    480x360,
    858x480,
    1280x720,
    1920x1080,
    2560x1080,
    3440x1440,
    1920x1200,
    3860x2160,
    3840x1600,
    1366x768,
    1280x800
]]
[nv_coder] -- [auto]
[vt_coder] -- [auto]
[min_log_level] -- [2]
[origin_web_ui_allowed] -- [pc]
[amd_quality] -- [default]
[vt_software] -- [auto]
[nv_tune] -- [ull]
[vt_realtime] -- [enabled]
[key_rightalt_to_key_win] -- [disabled]
[qsv_coder] -- [auto]
[gamepad] -- [x360]
[nv_preset] -- [p4]
[amd_coder] -- [auto]
[upnp] -- [disabled]
[dwmflush] -- [enabled]
[channels] -- [1]
[nv_rc] -- [cbr]
[hevc_mode] -- [0]
[2023:02:07:21:37:05]: Info: Found [1] outputs
[2023:02:07:21:37:05]: Info: Virtual Desktop: 1920x1080
[2023:02:07:21:37:05]: Info: XrandR: available
[2023:02:07:21:37:05]: Info: -- Output --
[2023:02:07:21:37:05]: Info:   Resolution: 1920x1080
[2023:02:07:21:37:05]: Info:   Offset: 0x0
[2023:02:07:21:37:05]: Error: Couldn't destroy session handle: Unable to cleanup NvFBC
[2023:02:07:21:37:05]: Error: Environment variable WAYLAND_DISPLAY has not been defined
[2023:02:07:21:37:05]: Info: Detecting connected monitors
[2023:02:07:21:37:05]: Info: Trying encoder [nvenc]
[2023:02:07:21:37:05]: Info: Screencasting with NvFBC
[2023:02:07:21:37:05]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:05]: Info: Color range: [JPEG]
[2023:02:07:21:37:05]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:05]: Info: Screencasting with NvFBC
[2023:02:07:21:37:05]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:05]: Info: Color range: [JPEG]
[2023:02:07:21:37:05]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:05]: Info: Screencasting with NvFBC
[2023:02:07:21:37:05]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:05]: Info: Color range: [JPEG]
[2023:02:07:21:37:05]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:05]: Info: Screencasting with NvFBC
[2023:02:07:21:37:06]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:06]: Info: Color range: [JPEG]
[2023:02:07:21:37:06]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:06]: Info: Screencasting with NvFBC
[2023:02:07:21:37:06]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:06]: Info: Color range: [JPEG]
[2023:02:07:21:37:06]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:06]: Info: Screencasting with NvFBC
[2023:02:07:21:37:06]: Info: SDR color coding [Rec. 709]
[2023:02:07:21:37:06]: Info: Color range: [JPEG]
[2023:02:07:21:37:06]: Error: Could not open codec [h264_nvenc]: Function not implemented
[2023:02:07:21:37:06]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:06]: Info: Screencasting with NvFBC
[2023:02:07:21:37:06]: Info: SDR color coding [Rec. 709]
[2023:02:07:21:37:06]: Info: Color range: [JPEG]
[2023:02:07:21:37:06]: Error: cuda::cuda_t doesn't support any format other than AV_PIX_FMT_NV12
[2023:02:07:21:37:06]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:06]: Info: Screencasting with NvFBC
[2023:02:07:21:37:06]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:06]: Info: Color range: [JPEG]
[2023:02:07:21:37:06]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:06]: Info: Screencasting with NvFBC
[2023:02:07:21:37:06]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:06]: Info: Color range: [JPEG]
[2023:02:07:21:37:06]: Warning: nvenc: hevc: replacing nalu prefix data
[2023:02:07:21:37:06]: Error: Couldn't destroy session handle: The context is not bound
[2023:02:07:21:37:06]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2023:02:07:21:37:06]: Info: 
[2023:02:07:21:37:06]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2023:02:07:21:37:06]: Info: 
[2023:02:07:21:37:06]: Info: Found encoder nvenc: [h264_nvenc, hevc_nvenc]
[2023:02:07:21:37:06]: Info: Adding avahi service Sunshine
[2023:02:07:21:37:06]: Info: Configuration UI available at [https://localhost:47990]
[2023:02:07:21:37:07]: Info: Avahi service Sunshine successfully established.
[2023:02:07:21:37:10]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:10]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:13]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:16]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:19]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:22]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:25]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:28]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:31]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:31]: Info: Executing [Desktop]
[2023:02:07:21:37:31]: Info: CLIENT CONNECTED
[2023:02:07:21:37:31]: Info: Found [1] outputs
[2023:02:07:21:37:31]: Info: Virtual Desktop: 1920x1080
[2023:02:07:21:37:31]: Info: XrandR: available
[2023:02:07:21:37:31]: Info: -- Output --
[2023:02:07:21:37:31]: Info:   Resolution: 1920x1080
[2023:02:07:21:37:31]: Info:   Offset: 0x0
[2023:02:07:21:37:31]: Error: Couldn't destroy session handle: Unable to cleanup NvFBC
[2023:02:07:21:37:31]: Info: Screencasting with NvFBC
[2023:02:07:21:37:31]: Info: SDR color coding [Rec. 601]
[2023:02:07:21:37:31]: Info: Color range: [MPEG]
[2023:02:07:21:37:31]: Info: Setting default sink to: [sink-sunshine-stereo]
[2023:02:07:21:37:31]: Info: Found default monitor by name: sink-sunshine-stereo.monitor
[2023:02:07:21:37:46]: Info: CLIENT DISCONNECTED
[2023:02:07:21:37:46]: Info: Setting default sink to: [sink-sunshine-stereo]
[2023:02:07:21:37:46]: Error: Couldn't destroy session handle: Unable to cleanup NvFBC
[2023:02:07:21:37:46]: Error: Couldn't release NvFBC context from current thread: Unable to cleanup NvFBC
[2023:02:07:21:37:46]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:46]: Info: /CN=NVIDIA GameStream Client -- verified
[2023:02:07:21:37:49]: Info: /CN=NVIDIA GameStream Client -- verified
^C[2023:02:07:21:37:51]: Info: Interrupt handler called

[dec05eba]

Try hevc instead of h264, that seems to drop RAM and VRAM usage by half (at least in other applications that use nvenc)

[electronic-van]

I originally tested with hevc, but switching to h264 does use about double the vram (130mb vs 350mb). 4k Xorg capture only gets 24fps on my machine so I have to use nvfbc. With h264, vram increases by around the same amount as Xorg capture but on top of the 2gb nvfbc adds.

[cgutman]

Likely fixed by #1098. Please retest the latest nightly.

[electronic-van]

Can confirm the VRAM part is fixed with #1098. VRAM usage is down to 500MB but the GPU still maxes out at 87% usage.

[cgutman]

Can you post the actual usage stats from nvidia-smi pmon -i 0? I want to see the per-engine utilization numbers, because a global GPU utilization number is not necessarily representative of the utilization of all engines.

For example, some tools will report the 3D engine only, while others will report the utilization of the engine with the highest utilization. An 87% utilization of NVENC may be expected based on encoding settings (and wouldn't impact game performance), but 87% usage of the 3D engine by Sunshine would be very unexpected

gpu-screen-recorder mentions that memory clocks are reduced by the Nvidia driver when executing CUDA workloads. Do you think that could be happening on your system?

[electronic-van]

I won't be able to test until the end of the week, but that does sound like a possible cause.

[electronic-van]

I did some testing with both Sunshine and gpu-screen-recorder and Sunshine seems to limit the actual 3D usage. The memory clock speeds are the same as not streaming or recording. This happens on all games I tested, different games don't change anything. These results are with Minecraft shaders.

I also tried Sunshine on Windows, which performs similarly to gpu-screen-recorder on Linux.

Not streaming or recording


# gpu         pid  type    sm    mem    enc    dec    command
# Idx           #   C/G     %      %      %      %    name
    0       1332   C+G     -     -     -     -   sunshine       
    0       1519     G     -     -     -     -   plasmashell    
    0       1572     G     -     -     -     -   firefox        
    0       1803     G     -     -     -     -   plasma-browser-
    0       8370     G     -     -     -     -   python         
    0      16463     G     -     -     -     -   konsole        
    0      20998     G     -     -     -     -   krunner        
    0      21035     G     -     -     -     -   baloorunner    
    0      21268     G    99    33     -     -   java           

Streaming with Sunshine

(low encoder usage due to 1080p encoding) 

  # gpu         pid  type    sm    mem    enc    dec    command
  # Idx           #   C/G     %      %      %      %    name   
    0       1332   C+G     -     -    18     -   sunshine       
    0       1519     G     -     -     -     -   plasmashell    
    0       1572     G     -     -     -     -   firefox        
    0       1803     G     -     -     -     -   plasma-browser-
    0       8370     G     -     -     -     -   python         
    0      16463     G     -     -     -     -   konsole        
    0      20998     G     -     -     -     -   krunner        
    0      21035     G     -     -     -     -   baloorunner    
    0      21268     G    87    31     -     -   java

Recording with gpu-screen-recorder

Does drop the framerate slightly, but only a couple frames 

# gpu         pid  type    sm    mem    enc    dec    command
# Idx           #   C/G     %      %      %      %    name
    0      21268     G    99    29     -     -   java           
    0      22530     G     -     -     -     -   dolphin        
    0      23127     G     -     -     -     -   konsole        
    0      25016   C+G     -     -    63     -   gpu-screen-reco

[CarlKath]

I suppose that asking this here is wrong.

How to install the nightly version? I am using Ubuntu 22.04.

[ns6089]

I suppose that asking this here is wrong.

How to install the nightly version? I am using Ubuntu 22.04.

https://github.com/LizardByte/Sunshine/releases/tag/nightly-dev

[CarlKath]

I suppose that asking this here is wrong. How to install the nightly version? I am using Ubuntu 22.04.

https://github.com/LizardByte/Sunshine/releases/tag/nightly-dev

Thank you very much :)

[LizardByte-bot]

This issue is stale because it has been open for 90 days with no activity. Comment or remove the stale label, otherwise this will be closed in 10 days.

[LizardByte-bot]

This issue was closed because it has been stalled for 10 days with no activity.