werman / noise-suppression-for-voice

Noise suppression plugin based on Xiph's RNNoise
GNU General Public License v3.0
4.73k stars 228 forks source link

How to add support for pipewire? #62

Closed hr567 closed 2 years ago

hr567 commented 3 years ago

Fedora 34 has a change proposal route all audio to pipewire by default. Is it possible to make this noise-suppression-for-voice plugin work with pulseaudio drop-in replacement? (pactl command doesn't work correctly and the pulseaudio config file does not work with pipewire)

kode54 commented 3 years ago

Currently, you can enable JACK2 support in PipeWire, and either system-wide enable the JACK2 wrapper libraries, or run with pw-jack, a JACK2 graph client such as Carla. From there, you will see all of your PipeWire clients as objects interconnected, and you may add either the mono or stereo LADSPA (or LV2?) plugin into the graph and reroute your microphone through the filter before it hits whatever is usually recording from it.

This is not ideal, however. For one thing, you cannot set the filter as the default source, so new recordings will still connect directly to the microphone. For another, the plugin will only live as long as you continuously run the JACK2 client which is hosting it. For three, Carla is a very buggy host at this point, so maybe it would be better to use a different host. I can't think of any off-hand, other than Reaper, and I haven't figured out the workflow for that host yet.

kode54 commented 3 years ago

Carla has lots of bugs, by the way. I should mention these on Carla's tracker, assuming it has one. For one thing, it doesn't like the nature of PipeWire adding and removing objects automatically, and will exhibit artifacts such as the interconnection bezier paths remaining on the graph indefinitely. And it will sometimes decide that it doesn't like you dragging an object around, and will suddenly decide that object should reside permanently at a specific spot on the graph view.

llitz commented 3 years ago

This is a little bit old, but you can use jack-rack for ladspa plugins

pipewire has also implemented virtual sinks and they can be set as default. For example

pactl load-module module-null-sink sink_name=virt_mic object.linger=1 media.class=Audio/Source/Virtual channel_map=FL,FR
pactl set-default-source virt_misc

I also have qjackctl automatically linking my jack-rack to the virt_mic.

For jack-jack, you can save your plugins and have it load the plugins + configurations automatically from the command line qjackctl can also do the same.

hr567 commented 3 years ago

I use the RNNoise function provided by Pulseeffects now. It works fine but I am still finding a lightweight solution.

sloonz commented 3 years ago

With virt_mic setup like that, you can just do :

gst-launch-1.0 pipewiresrc path=40 ! capsfilter caps=audio/x-raw,format=F32LE,channels=2,rate=48000 ! audioresample ! peadapter blocksize=480 ! pernnoise ! peadapter blocksize=1024 ! audioresample ! capsfilter caps=audio/x-raw,format=F32LE,channels=2,rate=48000 ! pipewiresink path=137

When path=40 is your real mic source and path=137 is the node id of virt_mic (see pw-cli dump. Be sure to take the node id, not one of the 4 ports)

THS-on commented 3 years ago

LADSPA filter chain support has just landed in Pipewire (but is not in the latest released version 0.3.27): https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Filter-Chain

I'm currently using carla started as a systemd user service and a virtual sink and it seems to work fine. I'll try the filter chain option soon as it is in a released version.

eskoONE commented 3 years ago

i dont understand much of what you guys are doing here. what is the consensus of using this with pipewire?

MartinX3 commented 3 years ago

i dont understand much of what you guys are doing here. what is the consensus of using this with pipewire?

Just wait until this issue ticket gets closed.

werman commented 3 years ago

There are several seemingly ways described above, personally, at the moment I'm using PulseEffects since I also need an equalizer, and sometimes want to denoise conference calls.

THS-on commented 3 years ago

I've now tested filter chains in PipeWire for a week now and it works for my use case (persistent filtering on one mic). But currently the media session in PipeWire does not allow creating Links to filters automatically based on custom rules. I have a script that does that for me using WirePlumber, but it has it's own issues: https://gist.github.com/THS-on/a8fe22891c1008454a948e86d25315f5

PipeWire has a ladspa-source module for PulseAudio and if you replace the pacmd commands with pactl the old setup should also work on PipeWire without using a null sink, but I haven't tried that yet.

thrimbor commented 3 years ago

PipeWire has a ladspa-source module for PulseAudio and if you replace the pacmd commands with pactl the old setup should also work on PipeWire without using a null sink, but I haven't tried that yet.

I tried that today after encountering issues with pulseeffects, and it works, with one small catch: I had to add a node.description property to the module loads, or TeamSpeak 3 would crash when launching (this is described in https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/462).

werman commented 3 years ago

The example at https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/Filter-Chain works perfectly and with no overhead when no one uses the source.

werman commented 3 years ago

Ok, I added a how-to to the readme since filter-chain works for me.

TLATER commented 3 years ago

Sadly this still fails for me on pipewire 0.3.30:

Aug 08 18:46:21 yui pipewire[7127]: can't load graph: Operation not supported
Aug 08 18:46:21 yui pipewire[7127]: config 0x5608832fdc80: could not load mandatory module "libpipewire-module-filter-chain": Operation not supported
Aug 08 18:46:21 yui pipewire[7127]: failed to create context: Operation not supported

This seems to only occur when setting the VAD control. When not setting that I get more un-enlightening errors:

Aug 08 19:00:25 yui pipewire[2642]: node 0x7f465c017010: set_param Spa:Enum:ParamId:PortConfig (11) 0x558354979608: Input/output error
Aug 08 19:00:25 yui pipewire[2642]: impl-core 0x5583548e7c80: error -5 for resource 2: node_set_param(Spa:Enum:ParamId:PortConfig) failed: Input/output error
Aug 08 19:00:25 yui pipewire[2642]: client-node 0x5583549efd20: error seq:39 -5 (node_set_param(Spa:Enum:ParamId:PortConfig) failed: Input/output error)
Aug 08 19:00:25 yui pipewire[2642]: (pipewire-37) idle -> error (error changing node state: Input/output error)
Aug 08 19:00:25 yui pipewire[2642]: link 0x558354ee4ba0: one of the nodes is in error out:running in:error
Aug 08 19:00:25 yui pipewire[2642]: link 0x558354ee53b0: one of the nodes is in error out:running in:error
Aug 08 19:00:25 yui pipewire[2642]: node 0x7f4657622010: set_param Spa:Enum:ParamId:PortConfig (11) 0x558354979828: Input/output error
Aug 08 19:00:25 yui pipewire[2642]: impl-core 0x5583548e7c80: error -5 for resource 3: node_set_param(Spa:Enum:ParamId:PortConfig) failed: Input/output error
Aug 08 19:00:25 yui pipewire[2642]: client-node 0x5583549f5330: error seq:57 -5 (node_set_param(Spa:Enum:ParamId:PortConfig) failed: Input/output error)
Aug 08 19:00:27 yui pipewire[2642]: 94 events suppressed
Aug 08 19:00:29 yui pipewire[2642]: 94 events suppressed
Aug 08 19:00:31 yui pipewire[2642]: 93 events suppressed

Edit: Pasted wrong part of logs

Neither of these errors really help figuring out what's going wrong though.

@werman for reference, which pipewire version did you test that with? There also seems to be a typo in the upstream example: "VAD Threshold (%)" 50.0; I don't think that's valid syntax.

werman commented 3 years ago

@TLATER

I have the same pipewire version:

pipewire
Compiled with libpipewire 0.3.30
Linked with libpipewire 0.3.30

Packages installed:

extra/gst-plugin-pipewire 1:0.3.32-1 [installed: 1:0.3.30-1]
extra/pipewire 1:0.3.32-1 [installed: 1:0.3.30-1]
extra/pipewire-alsa 1:0.3.32-1 [installed: 1:0.3.30-1]
extra/pipewire-jack 1:0.3.32-1 [installed: 1:0.3.30-1]
extra/pipewire-media-session 1:0.3.32-1 [installed: 1:0.3.30-1]
extra/pipewire-pulse 1:0.3.32-1 [installed: 1:0.3.30-1]

That config works fine, I only set a path to the plugin and noise_suppressor_mono instead of stereo. The service runs fine. Try launching it manually, not through service and see if anything changes.

The errors you got are extremely user hostile...

werman commented 3 years ago

Otherwise if you double checked and everything is fine - open an issue in pipewire's bugtracker.

TLATER commented 3 years ago

Hrmm, I still don't know what's going wrong. As it turns out, feeding the config through nixos' modules convers it to real json instead of pipewire's faux json (to do some validation I presume?).

After shortcutting that and just using the verbatim upstream config it works. For the minute, it seems that configuring pipewire with json doesn't work here? I might try and write an issue for pipewire on this.

Well, either way, all irrelevant to this ticket, the plugin seems to work fine for me with that fixed, without the issue of cascading lag I had on pulse, so I'm happy for now :)

TLATER commented 3 years ago

In case anyone else ever runs into this, I've solved the mystery: The ordering of the filters in the graph matters, so if you sort the json then things will break. Somewhat obvious in retrospect, I just didn't realize my configuration was being converted, let alone sorted.

Edit so I don't ping the whole list again with something unrelated, here is the real issue: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1526.

I.e., filter graph node properties for some reason need to be ordered correctly; this violates some assumptions that json formatters typically have (and is either a bug or desperately needs documentation).

werman commented 2 years ago

PipeWire has split-file configuration for a while now. I made a new release, and this issue is rather aimless - if you have any particular issue please open a new one.