PipeWire RNNoise Filter appears to cause skipping/stutter in audio output

[duncanyoyo1]

Since the move to PipeWire Filters ( and the renaming to EasyEffects ) I have been having stutter / skipping whenever I use any filters in EE.

Just having the limiter on is enough to cause it.

The source of the audio doesn't matter, it happens from anything playing audio. And it only happens with EasyEffects open. When I close it the issue completely goes away.

Easy Effects Log File

pipewire[2311]: (pe_soe_spectrum-274) client too slow! rate:2048/192000 pos:732744192 status:triggered
pipewire[2311]: (pe_sie_spectrum-458) client too slow! rate:2048/192000 pos:635938816 status:triggered
pipewire[2311]: (pe_soe_limiter-202) client too slow! rate:2048/192000 pos:728615424 status:awake
pipewire[2311]: (pe_soe_compressor-122) client too slow! rate:2048/192000 pos:722106880 status:triggered
pipewire[2311]: (pe_soe_output_level-377) client too slow! rate:2048/192000 pos:698382848 status:triggered
pipewire[2311]: (pe_sie_output_level-428) client too slow! rate:2048/192000 pos:637106176 status:triggered
pipewire[2311]: loop 0x55d7b19b7a50: failed to read timer fd 18: Resource temporarily unavailable

I can see a latency spike in pw-top when the skips/stutter happens. Unfortunately it happens so fast it's very difficult for me to snap a screenshot or copy the text.

A couple things to note about my setup, I have to output at 192000 otherwise I get noise/static on my left channel. I have a similar issue in Windows, so I do not believe it to be related to the operating system/PipeWire/EasyEffects. And this isn't my first motherboard with this chipset. My previous board had the same thing ( and many other people seem to have the issue as well ). The Audio chipset I have is the ALC1220 which seems well supported in many configurations, but for some reason on Ryzen or Threadripper systems it can be less than ideal.

I have changed a few lines in my /etc/pipewire/pipewire.conf

 default.clock.rate        = 192000
 default.clock.quantum     = 512
 default.clock.min-quantum = 32
 default.clock.max-quantum = 4096

Setting the default.clock.rate lower causes the noise issue separate from PipeWire/EasyEffects, it was a problem with Pulse as well.

Please let me know if there is any more information that may be useful in figuring out what the issue is here. I'd love to be able to have stable audio again.

[wwmm]

I also have an ALC1220. Whenever it goes from active to suspended it makes loud noises. This chipset support could be better...

Just having the limiter on is enough to cause it

And without any plugins selected? In this case the only active filters should be the spectrum and the global level meter. It may help to know if the problem also happens with only these two filters active or if more have to be enabled to trigger it..

I think I will have to do some tests with the sampling rate set to 192000. This may be related because the filter cpu usage will probably be higher in this case.

[wwmm]

So far I did not see anything unusual in your logs. The only thing that caught my attention is that your card profile is set to Pro. Is that right? I did not do many tests with my card in the Pro mode.

[wwmm]

/etc/pipewire/pipewire.conf

Only now I noticed that I do not have this file. Is this in a new location now?

[duncanyoyo1]

It's just copied from the /usr/share/pipewire/pipewire.conf location since I edited it.

And yes for whatever reason in Pro mode it seems to be less bad. When I use the normal output it's even worse/almost unusable.

If you'd like I can switch it and take the logs again.

With no filters on, it doesn't seem to happen. Only when another program interrupts the audio ( e.g. a notification or something else, which I believe is a known issue )

[wwmm]

And yes for whatever reason in Pro mode it seems to be less bad. When I use the normal output it's even worse/almost unusable.

In that case there is no need for me to test in the Pro mode. I will change my sampling rate to 192000 and do tests on the standard hardware profile.

With no filters on, it doesn't seem to happen

So one of the other filters has to be linked for the skipping/stutters to happen. Hummm... When you have time enable one fo the plugins that cause the problem and run pw-dot. Attach the generated file here on github. I will use xdot to view the connections between filters. Maybe something is going wrong with this.

[duncanyoyo1]

pw.dot

Had to change the extension to txt to get GitHub to let me attach it.

[wwmm]

And to my surprise I have no sound when I force PipeWire to use 192000 Hz. Even when EasyEffects is not running I have no sound. That is weird.

[duncanyoyo1]

Check pavucontrol or Helvum, sometimes I had it where the output would get confused for whatever reason and be set to the wrong place.

[wwmm]

I have already tried to unmute the device. Still silent =/

[wwmm]

My monitor HDMI card is working. So far no problems using EasyEffects on it with 192000 Hz. But my ALC1220 optical port does not work with 192000. Now I wonder if it even supports this sampling rate. I just assumed it would. But maybe it doesn't.

[wwmm]

That will be a problem. What I really use is the optical port in my onboard card. I was hoping to use it at 192000 for a while to test this. The quality of my monitor sound is horrible. I use it only for quick tests...

[duncanyoyo1]

Hmm I've never used the optical port on mine, just the rear panel 3.5mm.

The specs for the ALC1220 seems to suggest it supports 24bit, 192kHz from the optical port.

[duncanyoyo1]

I think I have figured out the issue though, and it's a weird one.

I didn't realize I had the input filters on still for my microphone. Disabling those seems to have stopped the stutters. and I don't see the client too slow errors in the journal anymore.

And that's with the output filters on, Bass Enhancer, Compressor and Limiter.

Why the input filters would effect output, I have not the slightest clue.

After some quick testing, it appears to be the RNNoise filter specifically that is causing the issue.

[wwmm]

I didn't realize I had the input filters on still for my microphone. Disabling those seems to have stopped the stutters. and I don't see the client too slow errors in the journal anymore.

I've noticed in your pw.dot file that you had plugins enabled in the microphone chain and nothing but Pavucontrol monitors recording from our source. Pavucontrol is probably the one that was keeping the mic pipeline active. Unfortunately it has this side effect.

That being said I agree that it should not have been a problem.

Now I see that the headphone works when I plug it in my onboard card. It is only the optical port that so far doesn't. Maybe my Edifier R2000DB is the one that does not support 192000 Hz

[wwmm]

After some quick testing, it appears to be the RNNoise filter specifically that is causing the issue.

Ok. I will do some tests with it at 192000. That plugin is special because its algorithm only works with 48 kHz. I had to add an internal resampling to it. So the rate has to go to 48 kHz and then back to 192000. This may be causing problems.

[wwmm]

Why the input filters would effect output, I have not the slightest clue.

They may be affecting the global server latency. And as changes in latency in this situation mean changes in buffer size you may notice something in the audio depending on the buffer size difference that was caused.

So far I see no problems with the rnnoise plugin. Depending on the content we feed to it the sound may seem a little "chopped" but that is because this plugin is intended to remove background noises from a content that has mostly peopl talking. So my test now with music will lead to weird results.

[wwmm]

They may be affecting the global server latency

Not the filters themselfs but the fact that the pipeline became active. Pavucontrol would be the one requesting a latency value in this case.

[duncanyoyo1]

Yeah, in my case I'm not feeding it anything, or at least shouldn't be. The mic isn't being used by anything ( apart from Pavucontrols monitor ), and it doesn't show any "recorders" in EasyEffects.

It just seems strange a filter on an input, not even being played back, would cause glitches on the output. May be an issue with Pavucontrol. Let me try killing it and seeing what happens.

So with Pavucontrol closed, it's perfectly fine with the RNNoise plugin. No issues that I can tell.

It seems I can have one or the other, Pavucontrol or RNNoise. In that case I'd easily pick RNNoise and just forget about pavucontrol. My DE gives me basically the same thing in my taskbar.

I need to see if the glitches come back when I actually use the mic. Yes, they do. So if I am using the mic, and listening to audio it's really bad. Even with Pavucontrol closed.

[wwmm]

and it doesn't show any "recorders" in EasyEffects.

That is because since Pulseaudio I have been blocklisting it. But as PipeWire filter chains behave a little different than GStreamer streams Pavucontrol monitors are still activating our pipeline. So for now the best thing to do is not letting plugins in the mic pipeline when you are not using it. Or closing Pavucontrol.

[duncanyoyo1]

Have you only tested the RNNoise plugin with the default model? I have a few other models in mine ( tuned for different things ), but they all have the same result with glitchy audio including the default model.

It seems like any time the RNNoise filter is active, the issue happens.

What was apparently happening was Pavucontrol was keeping it open the whole time because of it's monitor. So it was always happening. Then when I closed it, it closed the stream so it stopped.

But using the mic in another program ( e.g. Discord ) makes it happen again, even with Pavucontrol closed. It also uses a lot of CPU when it's active ( taking 50% of 2 threads/cores ) but this is probably pretty normal so not sure how important it is.

[wwmm]

Have you only tested the RNNoise plugin with the default model?

Yes. Send me your custom models so I can test them too.

It also uses a lot of CPU when it's active ( taking 50% of 2 threads/cores )

That may be related to the resampling that this plugin requires to be done. Like I said before rnnoise is intended to be used at 48 kHz. So in your case the data is converted from 192000 to 48000 and then back to 192000. That certainly is eating cpu.

[duncanyoyo1]

They are the ones from https://github.com/GregorR/rnnoise-models

bd, cb, lq, mp, sh

I wonder if would be possible to set the input sample rate separately from the output sample rate. My mic is a USB mic so it's not using the rear panel connectors.

Only the Input and Output are 192000 in pw-top, the rest are 48000.

If I could set my mic to be 48000 that might reduce some of the CPU usage. Doubt that is related to the issue though.

[wwmm]

I wonder if would be possible to set the input sample rate separately from the output sample rate.

That depends on whether PipeWire allows that. Our filters work with whatever it sends us. I would expect different rates between input and output devices to be a problem in our simple and naive echo canceller filter. The orders should be fine.

Are you able to somehow record the audio issues you are having? Sometimes this is the better way to understand exactly what is happening on the user side.

[wwmm]

They are the ones from https://github.com/GregorR/rnnoise-models

I tested these ones. Honestly I never noticed any difference between them. So usually I just use the standard. I imagine it depends greatly on what we are filtering. Maybe what I have used so far is not the kind of content that show the difference between these models.

[wwmm]

Only the Input and Output are 192000 in pw-top, the rest are 48000.

I do not think this is possible. If your mic and your soundcard are at 192000 everything will be too.

[duncanyoyo1]

Any applications playing audio are either showing as 44100 or 48000, only the input and output devices are showing as 192000 in pw-top. I am guessing PipeWire is resampling them before outputting them.

Currently trying to setup a loopback device to record the audio. But Audacity doesn't want to play nice with PipeWire for whatever reason. Might have to use OBS to record the audio then strip the audio track out.

[duncanyoyo1]

You can hear the noise from the start till about 26s in when I removed the RNNoise filter.

https://user-images.githubusercontent.com/24286951/125353679-da940080-e330-11eb-9865-f893d91e577f.mp4

[wwmm]

That kind of noise is one of the hardest to figure out the source. In these years developing this project I have seen this happening in Pulseaudio and PipeWire for very different reasons. The last one of them was related to speech-dispatcher. I have even removed it from my system. It was a source of random noises both in Pulseaudio and PipeWire.

Any applications playing audio are either showing as 44100 or 48000, only the input and output devices are showing as 192000 in pw-top. I am guessing PipeWire is resampling them before outputting them.

Yes. They have to be resampled.

[wwmm]

I am doing some tests with an usb microphone in Discord's mic test while I whatch something on Twitch. So far I did not listen to this noise. But my tests are being done at 48 kHz because of the previous problems trying to use this rate in my optical port.

[wwmm]

I was able to reproduce the issue. It seems to only happen when using 192000 Hz. And the noise remover plugin is not the only one that can trigger it. All the plugins that require very specific audio array sizes can too. My guess is that the higher buffer sizes used by PipeWire when running at 192000 Hz are breaking the code I wrote to adapt the buffer size for these plugins that can not work with any size. That sucks =/

[wwmm]

After doing more tests I am starting to suspect that the problem is related to resampling. After increasing the quality I started to listen more noise. It may seem strange at first but higher quality requires much more processing time. And inside a realtime thread this increase may be problematic. That may be what is going on. And it would explain why sometimes things seem to go well for while and suddenly there are noises.

[duncanyoyo1]

That would make sense.

I wonder what PipeWire is doing because their resampling doesn't seem to cause any issues.

Could be that having to do multiple resamples just takes too long to hit the window it needs.

I wonder if increasing the latency would have any effect. I wouldn't mind a bit more latency as long as it was reasonable.

[wwmm]

I wonder what PipeWire is doing because their resampling doesn't seem to cause any issues.

It uses a custom resampling algorithm. I am using the library libsamplerate. I think it is a good library. I may just be using it in the wrong way. After reducing the quality and tweak a little the size of its input array the sound is almost fine now. Most of the noise is gone. At least so far XD

Could be that having to do multiple resamples just takes too long to hit the window it needs.

I think that is part of the issue. And would explain client too slow messages in PipeWire's logs.

I wonder if increasing the latency would have any effect. I wouldn't mind a bit more latency as long as it was reasonable.

If the problem is resampling I think it would not help. And even if it did I don't think I can force a higher latency. We had a latency setting in Pulseeffects because GStreamer creates an audio stream like mpv or firefox do. Now we just link filters. There isn't a stream. Each filter may have latency but we just report it to PipeWire. It does not force the whole graph to be in a higher latency. Or lower.

[duncanyoyo1]

Yeah I don't think there is a way to adjust the latency for one specific source in PipeWire, just the whole PW server.

And since there aren't really any xruns it doesn't seem like it's running out of quantums to play either.

I think you could possibly buffer the streams inside EE but that would likely require a lot of work.

[wwmm]

I have sent some updates to the master branch. If you are using Arch Linux rebuild the aur package easyeffects-git. Let's see if the rnnoise plugin is better in your computer. It is a lot better here on mine.

Unfortunately this isn't the only plugin that is not playing nicely with 192000 Hz. The maximizer and the pitch plugin are also misbehaving. And these ones do not use the resampler I am using in rnnoise. I still have no idea why they are having problems with this sampling rate.

[duncanyoyo1]

So far it is much better. Still getting some client too slow, but the glitches are greatly reduced.

[wwmm]

I decided to test discord in Firefox instead of its desktop app and the difference is huge. I cannot listen to any noise in the pitch plugin when using discord in Firefox. WTF?!? It does not make sense. Testing the microphone through the command gst-launch-1.0 pipewiresrc ! queue ! pulsesink instead of the Discord app also gives a result that also seem to have no noise. I think we stumbled upon a very weird corner case. Discord's desktop app may be triggering something unusual in PipeWire.

[IDeathByte]

I see this issue if RNNoise filter applied for input and output in discord. If use filter only for one - no issues. Also all fine in 48k and less cracks on 96k

[toofar]

I believe I ran into this today too. I was pretty consistently hearing crackling from my output pipeline when my input pipeline was open with rnnoise enabled. I believe I've collected more evidence that it is xrun of some sort. I think the "BUSY" and "B/Q" columns of the pw-top tool are pretty telling (what a handy tool!):

The values for rnnoise are significantly higher than everything else and I could see the value in the ERR column go up occasionally while rnnoise was enabled (and it didn't when it wasn't). So in my case it seems likely that rnnoise was just being to slow for everything to be processed in time. I didn't try adjusting the headroom like it says to here. Instead I switched to this optimised (and untested!) branch of rnnoise: https://github.com/xiph/rnnoise/pull/191

After that the "WAIT" value for rnnoise is more like ~1ms when both pipelines are in use, compared to the ~5ms it was before with the same clients running. So thats nice. I can't comment on the PRs quality yet though, I'll see tomorrow if people think I sound different.

A couple of other notes:

• easyeffects wasn't every taking up a whole CPU core that I could see and my system wasn't under heavy load (like 15%), I also tried lowering its nice value but that didn't seem to help
• it's un-intuitive to me that a misbehaving input pipeline would cause issues in the output one (I guess they are handled in the same loop/thread?) - on the other hand it was probably causing issues on the sound other people where hearing too so I'm glad it drew my attention too it, there are a bunch of other stats in the UI, I wonder if an "errors in the pipeline/element" indicator would be valuable, like the same data as in pw-top

[wwmm]

it's un-intuitive to me that a misbehaving input pipeline would cause issues in the output one (I guess they are handled in the same loop/thread?)

In my experience when a second pipeline becomes active PipeWire changes the buffer size (latency) most of the times. But I think that this is the only thing that can happen.

wonder if an "errors in the pipeline/element" indicator would be valuable, like the same data as in pw-top

I never tried to see how pw-top is getting the information it shows in the WAIT and BUSY columns. So I do not know if it is doable. In any case as it plugin has its own value we would need ot be creative when finding space in the window to show this information. Specially for the multiband plugins that are already space hungry.

[ak2766]

Wow! I've been suffering from stuttering/skipping/chopping audio for a long long time with PulseAudio and finally migrated to Pipewire a few months back thinking it would eradicate the issue. It did not quite disappear as expected but it became less of an issue. It has been ok(ish) when wired but really terrible when using a bluetooth headset.

However, after making changes as documented by @duncanyoyo1 (the OP), the stuttering/skiping/chopping has completely gone! Seriously, I keep waiting to hear a stutter, a skip , or some chopping - but it has been smooth sailing for the last few hours AND on bluetooth too... I really hope that is the last I hear of it.