Max Audio Buffering When Using NVIDIA Noise Removal Filter

Deadmano commented 2 years ago

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

28.0.3

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/Hcs3K_LHq4u5WMZA

OBS Studio Crash Log URL

No response

Expected Behavior

To be able to use NVIDIA Noise Removal without a massive audio delay, without the audio device disconnecting and reconnecting, and without my monitors flickering as the audio device is reset by OBS.

Current Behavior

Whenever I attempt to use NVIDIA Noise Removal, not streaming or recording, just talking into my mic, eventually my monitor will start to flicker briefly and that is when the log starts writing about "max audio buffering". This repeats over and over. But the moment I choose another noise suppression filter (such as RNNoise) the issue goes away immediately.

Steps to Reproduce

Get NVIDIA Audio Effects SDK v1.2.0.13 from here.
Use an RTX card with NVIDIA drivers v517.48.
Set your Microphone as an input in VoiceMeeter or in OBS.
Apply the NVIDIA Noise Removal filter to your microphone.
Speak into your microphone, and after a short while the issue manifests. ...

Anything else we should know?

I recall having this issue in the past, where the device would just "die" and sometimes crash the stream using NVIDIA Noise Removal. It has been a while since, and I had stuck to using RNNoise but seeing how I got a new RTX 3080 and switched over (clean install) to Windows 11 22H2, I was hoping things would have improved.

I have a simple setup when it comes to the microphone; a Samson G-Track Pro that plugs in via USB. I did test both USB 2.0 and 3.0 ports, and different ports in general; no change. The issue is specific to the NVIDIA plugin as I have never had an issue with my mic using any of the other filters or in combination with virtual mixers such as VoiceMeeter.

Deadmano commented 2 years ago

When the "buffering" happens my display drivers crash, as per the Event Viewer in Windows:



If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event: 

\Device\Video3
Error occurred on GPUID: a00

The message resource is present but the message was not found in the message table

Deadmano commented 2 years ago

Just as an update to this, I did a full removal of the NVIDIA drivers using DDU, from safe mode, and then did a clean install of the latest NVIDIA drivers again (v517.48). Upon switching to the NVIDIA Noise Removal filter I immediately get the brief black screen flicker and check the event viewer to see the same error.

pkviet commented 2 years ago

If you also use the nvidia background removal filter , update also the video fx sdk. Mismatched sdks can cause issues (like newer audio sdk installed along an old video sdk). I'll investigate but i can't repro. If i need more details, i'll ping you

RytoEX commented 2 years ago

If you also use the nvidia background removal filter , update also the video fx sdk.

The log indicates that they do not have the Video FX SDK installed.

06:30:29.104: [NVIDIA RTX VIDEO FX]: FX disabled, redistributable not found.

Deadmano commented 2 years ago

If you also use the nvidia background removal filter , update also the video fx sdk. Mismatched sdks can cause issues (like newer audio sdk installed along an old video sdk). I'll investigate but i can't repro. If i need more details, i'll ping you

As @RytoEX mentioned I don't have the video SDK installed (I have no need for it yet). I'm also checking out various settings on my end, I'll revert if I come up with anything, but please do keep me in the loop!

pkviet commented 2 years ago

Tried to repro using a global audio device as in the log but nope. Definitely can't repro. I suspect the device. What is the current microphone used ? can you test with another one (ex: webcam mike for instance)

Deadmano commented 2 years ago

@pkviet just tried with my webcam, see log: https://obsproject.com/logs/qSQ2APpi_D5b__QT

Same issue. I even tried downgrading drivers to 516.40 which were known to be stable.

The moment I add the NVIDIA Noise Removal filter to any audio output it locks up OBS and crashes the display driver.

The description for Event ID 0 from source nvlddmkm cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event: 

\Device\Video5
Error occurred on GPUID: a00

The message resource is present but the message was not found in the message table

Display driver nvlddmkm stopped responding and has successfully recovered.

Could you perhaps try using VoiceMeeter to see if that extra bit of latency triggers it for you?

Set your VoiceMeeter A1 to your primary audio device that you listen to audio with. In Windows set VoiceMeeter Input as the default Playback device, and VoiceMeeter Output as the default Recording device.

Add an audio source in strip 1 with the input of your microphone, and tick B1.

Then in OBS: Settings -> Audio -> Mic/Auxilary Audio -> VoiceMeeter Output (VB-Audio VoiceMeeter VAIO).

Add a new filter to the microphone source -> Noise Suppression -> NVIDIA Noise Removal.

pkviet commented 2 years ago

Could you try to disable Voicemeeter ? I can't test it atm, it's not installed. I've had negative experience with it with the asio plugin i co-authored. I wouldn't be surprised it's messing up stuff with wasapi

Deadmano commented 2 years ago

Alright so I removed VoiceMeeter, and just testing with my Samson G-Track Pro... Same issue, log here: https://obsproject.com/logs/mn7o4afCrYGzoD74

I did notice something, every time the issue happens, and the driver crashes, my GPU clock speed/memory gets maxed when normally it is sitting comfortably idle AKA clock goes from 210MHz to 1800MHz.

There is also a spike that can be noticed in the task manager. Related somehow?

pkviet commented 2 years ago

I'm not sure if there's a relation with the flickering. I don't have any for sure. We were reported a flickering issue with recent nvidia drivers, which is under investigation. Btw, there'll be a new set of drivers on the 12 for the rtx 4xxx series. Let's wait and try these new drivers.

Deadmano commented 2 years ago

Alright, some more feedback for you, @pkviet! Did a fresh new scene, closed all running applications, just added my headphones (to hear) and my microphone, no VoiceMeeter added: https://obsproject.com/logs/7hDRNpjEn1DP6TYL

It does the same issue, fresh install of OBS, no plugins, no extra data, nothing.

One thing I noticed when monitoring via OBS is I can hear it begin immediately to deteriorate. It's like those old school calls we had back in the day, you know the robotic kind that took forever to complete a word? Plenty of stuttering/distortion etc. before it finally catches up (crashing the driver and restarting) only to repeat...

I'd just really like to make use of my 3080... Is there a more stable version of RTX Noise Removal I can use? Or do you think the 4x drivers will be a better match? And will those work on the 3.x?

pkviet commented 2 years ago

Ok i want to be sure it's not wasapi related. Could you install obs-asio ? If you have realtek, it has an asio driver. If you use another soundcard, without asio, you can use voicemeeter which exposes an asio virtual device.

Deadmano commented 2 years ago

Sorry, a bit confused there. I have a Sound Blaster AE-5 Plus soundcard. I haven't really played around with ASIO drivers so not sure what I'm looking for or how to go about adding it. But if you can give me a set of instructions to try specific to OBS and this NVIDIA issue, I am more than happy to test and report the results.

FYI the soundcard wouldn't matter I just realised as my microphone is USB, even my webcam is USB... I'd have to see if I can even get a 3.5mm standard mic to plug into my soundcard if you feel that is worth testing.

itsDavidFarrell commented 2 years ago

Hi there, thought I'd chime in. I also have the exact same problem, using a 4090, and the same issue occurred for me with a 3090. Using the Nvidia Broadcast app is fine, but audio filter through OBS = same problem as described.

I'm using a GoXLR for my main audio, but as per Deadmano's experience, it doesn't matter which audio source I apply it to.

pkviet commented 2 years ago

Both of you, could you test with these binaries ? https://github.com/obsproject/obs-studio/actions/runs/3386201532

Download the windows x64 zip Unzip. The exe is in the bin subfolder. It's obs modified to use updated audio sdk.

This fixed an issue for someone else. Maybe it'll help for you too. Test and report. I can't repro on my side so i'm shooting in the dark. I have an rtx 4090 and 3070 which both work perfectly with the fx.

itsDavidFarrell commented 2 years ago

It'll be a few days till I can test it, I'll give it a go and report back at that point as soon as I can. Cheers!

pkviet commented 2 years ago

Also for the test, please :

test in a clean obs setup, so launch it with --portable arg (create a shortcut of the exe and add the arg by opening the Properties of the shortcut)
update the redists of both audio and video effects sdk to last version,
optionally, reinstall the gpu driver (uninstall cleanly with ddu, then reinstall)
post a log If you want to compare with what's expected, listen at the demos I produced here: https://github.com/obsproject/obs-studio/pull/7466

itsDavidFarrell commented 2 years ago

In the meantime, I just remembered something else, for whatever it might be worth - the audio buffering going ham would occur when OBS was on the primary monitor, but there wouldn't be the complete display driver failure and screen blink; that would occur only when OBS was on the secondary, vertical monitor.

itsDavidFarrell commented 2 years ago

Okay, so I decided I'd do some steps individually

tried making sure both of the newest SDKs were installed, and tried on current OBS install (which is also a portable 28.1 beta and which also got a new update today which I had installed) - same problem
Keeping profile and scene collection, copied that portable installation, then overwrote it with the binaries you linked (and ofc didn't install the update upon prompt) - success, problem gone. Though it does appear to introduce some latency, we're talking maybe in the region of 30-100ms (not measured, just gut feeling), and that seems in line with what the Broadcast App version induces, so seems like expected behaviour.

Seems like whatever's in those binaries you linked that's different from the regular 28.1 beta hides the secret of it.

Would it be useful for me to install the prompted update over that install to see if the issue returns? Or, if there's anything else that could provide useful information (dxdiag or other), do let me know.

pkviet commented 2 years ago

@itsDavidFarrell Thanks for the feedback. Can you post the log ? If the zip build fixes the issue for you, that's all good. I'll mark it as such. Waiting for confirmation from the original poster.

pkviet commented 2 years ago

@itsDavidFarrell no need to update the zipped build. I'm quite sure the issue will come back.

Deadmano commented 2 years ago

Thanks for sharing your findings @itsDavidFarrell it's much appreciated, and great to know I'm not alone in this! @pkviet I could test, but given things are in beta I'm going to be honest that I do not feel comfortable using it for actual streaming/recording when it quite literally crashes OBS and my stream. It is unsafe to use in its current state. Would it be possible to catch that error and switch to another noise suppression filter with a warning? That's the only way I could see myself using it again.

As soon as I get a chance I will however test to see if it resolves the flickering/crashing and revert back here.

pkviet commented 2 years ago

@Deadmano Yeah, don't use it for prod. But just test to see if it solves your issues. If it does, it'll accelerate the merging of the fix. It is scheduled to be in next beta in two weeks at the latest anyway. But you'll have to wait December for the release of 29.

NikolaSocrates commented 1 year ago

Any new happenings with this issue? I just installed a Zotac 4090 Trinity OC and seem to have this issue also. My monitors blink every 10 seconds or so after adding the NVidia noise removal filter to my microphone. Side note: if I turn off the preview, the blinking stops.

pkviet commented 1 year ago

Update: Could you all try to disable HAGS ? We have confirmation from several people that HAGS might be causing the issues. Go to Settings > Display > graphics and disable hags there : https://obsproject.com/wiki/How-to-disable-Windows-10-Hardware-GPU-Scheduler

NikolaSocrates commented 1 year ago

That seems to do the trick. Much appreciated!

RytoEX commented 1 year ago

Based on the information in this thread, this sounds like an issue with some combination of:

NVIDIA drivers version 510+
Hardware-accelerated GPU scheduling (HAGS)

This is not an issue in OBS itself that we can fix.

obsproject / obs-studio