[Feature Request] Add input processing

[Grafcube]

Is it possible to add some kind of background noise reduction and de-esser similar to easyeffects?

[james34602]

Yes, that's the feature can be added.

[james34602]

The one in easyeffects uses RNNoise(Open source), it's noise suppression is great, but too great that sometimes harm the actual signal, I would probably not going the same path as they did.

[Grafcube]

I personally never had an issue with it, but I'll admit I don't have much experience in this area. The reason I stopped using easyeffects was only because of the output filters. It would stutter heavily when Pipewire changed latency and forcing a fixed latency has its own issues, like audio getting cut sometimes. And I don't think it's a good idea to set a fixed latency anyway. But the input filters worked correctly. This is a known issue.

I did notice JamesDSP also cut the audio sometimes (dynamic latency) but it wasn't nearly as bad as easyeffects.

[james34602]

I'm not sure which part in the whole system introduce latency. The only part I can think of is sample rate conversion, but the latency of this one won't exceed 4 ms, I guess. Convolver won't introduce latency unless the impulse response is designed to have some

[timschneeb]

The problem isn't caused directly by JamesDSP core lib itself but by how PipeWire manages buffer sizes.

For example, suppose an audio pipeline is processing audio segments with buffer lengths of 512 each. Suddenly, a new application appears that wants to stream audio in buffers of length 128 (for lower latency). In that case, Pipewire requests all other elements (JamesDspPlugin + Headset/Speaker sink) in the pipeline to change their expected buffer lengths from 512 to 128 as well.

The stuttering that happens on your machine during this process is probably caused by an internal PipeWire bug since it also happens for EasyEffects.

Even when forcing apps to run at low latencies (to test the dynamic latency feature) using this environmental variable: PIPEWIRE_LATENCY=128/48000 (and while monitoring the pipeline using pw-top), I was unable to reproduce the stuttering issue. For context, I'm on Pipewire version 0.3.40.

---

Looking at the changelog for libpipewire Version 0.3.39:

* Fix a bug where the quantum could exceed the maximum because it was scaled with the sample rate.

I'm purely guessing here, but that bug fix might be related to your issue (quantum = sample count per buffer). Maybe you could try to update to the latest PipeWire version.

More details about Pipewire's buffering technique: https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/FAQ#pipewire-buffering-explained

[Grafcube]

I am using libpipewire 0.3.40. I set my pipewire.conf default and min quantum value to 1024 and max to 8192. This seems to have the least stuttering. I also switched back to easyeffects because the audio was very distorted on JamesDSP. I'm not sure why but it's probably because I didn't configure it correctly.

There are multiple issues discussing this on easyeffects as well. For now the suggested workaround is to force a fixed latency.

[timschneeb]

Hmm, okay. I'll look through the issues on EasyEffects to get a better idea of what is going on.

[wwmm]

Fix a bug where the quantum could exceed the maximum because it was scaled with the sample rate.

I was the one that reported this bug to the PipeWire developers. It had something else as side effect. In many places PipeWire has the maximum quantum size(audio buffer size) of 8192 hardcoded in its sources. But when the dynamic sampling rate was activated it could go beyond that value in some cases. What was making the filters I implemented in EasyEffects through the pw_filter object crash. It was worse than distorted sound. The filter had a segmentation fault trying to access non allocated data in the audio buffers and the whole application crashed with it. Fortunately they have fixed this bug in PipeWire's dynamic sampling rate switching.

[wwmm]

For now the suggested workaround is to force a fixed latency.

Things will be better in our next release because now I know LV2 plugins are somehow able to deal with on the fly audio buffer size changes on their own. So there is no need to reinitialize them like I was doing before. But for the plugins I have written myself the problem will still be there. The dynamic latency and sampling rate switching that PipeWire does is good for the user but dealing with it in the application code without introducing an additional fixed latency can be complicated...

[JanChec]

I'd love to be able to add dynamics compression for a more broadcast-like quality :slightly_smiling_face:

[atomflunder]

Hi, just wanted to check in if there are any updates on this 😊

Thanks for your work