search

microsoft / MixedReality-HolographicRemoting-Samples

Code samples for using Microsoft's Holographic Remoting library.
Other
141 stars 47 forks source link

NVENC concurrent sessions limitation #100

Closed galdalali closed 7 months ago

galdalali commented 8 months ago

Hi there,

I'm not sure if this is a proper question, but I'm trying to understand the NVENC limitation. I implemented the Nvidia-patch described here to remove Nvidia's restriction on the number of simultaneous NVENC video encoding sessions as described here.

I verified the patch works by running 6 h264_nvenc sessions simultaneously with ffmpeg as described here. However, I am still limited to two holographic remoting sessions for some reason, and I am getting the following error message: "Cannot establish a connection to holographic remoting player on the target with IP address.. "

Any idea why?

Remote side (Windows PC) OS version: Win10 Pro 22H2 Unity version: 2022.3.6f1 I'm using OpenXR. Graphics card(s) / graphics driver version: NVIDIA GeForce 4070

Player side (HoloLens 2) OS version: 20348.1535.arm64fre.fe_release_svc_sydney_rel_prod.230106-1454 Store player version 2.9.3

lappelsmeier commented 8 months ago

Hi @galdalali ,

I can't comment on the nVidia workarounds that you mentioned - have you also tried to run 6 h265 sessions at the same time? Holographic remoting by default uses h265, you can change the codec in the settings but the default is h265.

galdalali commented 8 months ago

I did. With this patch, 6 H265 FFMPEG pipelines work, and of course I made sure it uses NVENC. Without this patch, it can only handle 5 sessions at a time.

For some reason, even when patched, Holographic remoting refuses to open more than two sessions, regardless of the CODEC, tried H264 and H265.

lappelsmeier commented 8 months ago

Unfortunately we won't be able to repro this issue given the nature of the nVidia workaround. What you can try on your end: can you have six parallel H265 encode sessions running using MFT (not nvEnc directly) - we use Windows Media Foundation to talk to the encoder.

galdalali commented 7 months ago

@lappelsmeier, NVENC sessions have been increased to 8 in Nvidia's latest driver for. On a consumer grade graphics card, I was able to run 4 sessions of Holographic remoting with this driver. Close this issue if you wish. Thanks.

https://streamguides.gg/2024/01/nvenc-update-all-nvidia-geforce-cards-quietly-updated-to-8-encoding-sessions/