Suggestion: Per-audio-stream filtering

[mikenrafter]

I think it'd be nice to be able to specify specific effects for specific sinks/sources.

An example of where this'd be useful: auto-gain on outputs, one output is really loud, while another is really quiet. You won't be able to hear the quiet one, because the other one's loud.
Normally, you wouldn't be able to hear the quiet one at all, but auto-gain's supposed to make it easier to hear quiet things, and to keep your ears safe when things get loud.

So, the ability to specify that each sink should be processed separately would be very nice.

[wwmm]

Isn't the preset autoloading for specific devices enough for this? i am not sure I understand what you want.

[mikenrafter]

Different programs output to different streams, naturally. However, sometimes when multiple sources are playing, one expresses traits that cause a filter, to change both streams, because they're unified as an output. Sometimes that effect is undesired for the second stream.

If there was the option to apply filters to specific streams, be it manually (not preferred, though potentially useful) or automatically (preferred, and the most effective for that autogain example I gave above), then this problem would go away.

[wwmm]

If there was the option to apply filters to specific streams

There isn't. People asked for this before. The problem is that this require a very different application design when compared to the one we evolved to. We would have to handle multiple pipelines as well as multiple plugins instances. And the user interface and presets system would have to deal with this somehow. Super hard to do.

What you can do is blocklisting some applications so that they do not have any effects at all. But I do not see different effects for different streams (audio apps) being possible in the far future.

As we can not use more than one output or input device at the same time I do not see how "per source or sink" presets would help. It is the same as loading them manually depending on the sink you are using or setting an autoloading profile that will do this. That is assuming I understand what you mean by source or sink. In this context these words are usually used for devices and no apps.

[mikenrafter]

I see. I understand that handling multiple pipelines, especially from a UI perspective, is challenging.
However, this issue still persists, and I'd like an automatic do this for each, not do this to all button. Rather than manually blocklisting players, when an issue arises. I'm not sure how this'd play out internally, but it'd definitely improve the quality of things like auto-gain.

Now, this could cause performance issues, so it'd definitely be an opt-in option, if it were included.

On that last point, having the ability to use multiple output and input devices would be useful, perhaps an instancing feature? If the instancing feature was in use, the naming scheme would need to change from easyeffects sink/source to easyeffects sink/source #1,2,3.... Just a thought...

[mikenrafter]

After reading through some other issues, it appears as though the last section of my last comment may be a challenge due to the tight integration to gsettings. The naming scheme would need to change there too.
From /com/github/wwmm/easyeffects/stream(in/out)puts to /com/github/wwmm/easyeffects/streams/(1/2/3/...)/(in/out)put.

This' sort-of diverging from my original issue, however, so perhaps I'll split this into another one.

[wwmm]

The naming scheme would need to change there too.

Probably. It is the same drama as supporting multiple instances of the same plugin in the same pipeline. But if the problem was just changing how we handle gsettings it wouldn't be so hard. Tons of other places will have to adapt.

In the end it is one of those situations where the limitation comes from the fact that competing with plugins hosts like Ardour or Carla was never the intention. We evolved around the idea of being a simple tool to apply global audio effects. Multiple pipelines and plugins instances require a very different design.

[mikenrafter]

... if the problem was just changing how we handle gsettings it wouldn't be so hard.

For the multi-instancing, I think it could be achieved just by changing a few names (the source/sink names, and gsettings), and adding a command-line option to specify which instance should be ran.

---

I understand that the idea is to keep the application simple. I, however, think that these things are fairly simple.
If I was to try implementing instancing and/or the for this effect, treat each audio stream separately button, where in the code would I begin?

[wwmm]

For the multi-instancing, I think it could be achieved just by changing a few names (the source/sink names, and gsettings), and adding a command-line option to specify which instance should be ran.

Most of the work will have to be done on the C++ code. Including the part where presets are handled and the ones related to the user interface.

I understand that the idea is to keep the application simple. I, however, think that these things are fairly simple. If I was to try implementing instancing and/or the for this effect, treat each audio stream separately button, where in the code would I begin?

Unfortunately it is not going to be as simple as you think. Many classes will need some kind of adjustment to achieve multiple instances. And it is important that the graphical interface is also improved to handle this accordingly. But from the top of my head the most import ones would be

https://github.com/wwmm/easyeffects/blob/master/src/effects_base.cpp https://github.com/wwmm/easyeffects/blob/master/src/effects_base_ui.cpp https://github.com/wwmm/easyeffects/blob/master/src/stream_output_effects.cpp https://github.com/wwmm/easyeffects/blob/master/src/stream_input_effects.cpp https://github.com/wwmm/easyeffects/blob/master/src/presets_manager.cpp

And you can add to that list all the classes that have _preset in their names because they have to be reworked in one way or another to handle multiple instances.

[wwmm]

treat each audio stream separately button, where in the code would I begin?

In order to actually apply effects per application the interaction with PipeWire will have to go through deep modifications. The class handling most of these operations is https://github.com/wwmm/easyeffects/blob/master/src/pipe_manager.cpp. In order to apply effects we have to redirect the application somewhere. In our case it is our virtual sink/source. If you want multiple pipelines each of them will need some kind of suitable input/ouput node that the applications can be linked to. As most of them use Pulseaudio API this means having virtual sinks/sources for each pipeline.

So the pipe_manager class will have to be improved to handle the links from apps to the pipelines in this case. And load these additional devices on demand. And the presets system will have to be extended in a way that all this new behavior is saved to the presets while keeping the current behavior for people that do not need multiple pipelines. Like I said there is a fair amount of work. It is not impossible. But there are tons of changes that will have to be made all around the code.

[wwmm]

In order to apply effects we have to redirect the application somewhere

And the corresponding pipeline has to send its processed audio somewhere. With multiple processing chains we now have a situation that did not exist before. They could be playing to different devices. So the logic being used to handle the connection between the pipeline and the devices will have to be improved in the process.

[Sethox]

I didn't want to open up another ticket for this question/request. It's basically based on this ticket (loosely). The idea is per-audio-stream from application to device (output/input). Having multiple devices can cause that one device to need more filters than others (example: one device is close to surrounding noise). Thus creating multiple customizable filters per EasyEffect's sink, in other words having different filters to multiple devices that in turn creates another EasyEffect sink for said device. Would it be possible?

[wwmm]

Would it be possible?

It is the same situation. All the difficulties described above still apply. It isn't impossible. But the whole application has to be redesigned for this.

[smichel17]

You can do a limited version of this this today by using EasyEffects together with Helvum. The limitation is that there can only be one instance of each effect (https://github.com/wwmm/easyeffects/issues/656), so it can only apply to one stream (although maybe you could send both streams to the same effect at the end, after which you can't separate them again).

First, set up EasyEffects with all the effects you want to use (e.g. first the effects for stream 1, then the effects for stream 2). Then, open up Helvum to break the pipeline in the middle and send the different sources to the different starting points.

I've used this with voice chat (Mumble) so that I can play "radio" in a virtual coworking channel. The music gets a low pass filter applied (24dB, frequency=775) so that it's easy to talk over without being distracting (sounds like it's in the other room), and my voice gets RNNoise applied (if I did that in Mumble, it would take out the music). It works reasonably well, but is a pain to set up every time, since Helvum doesn't have a good way to save your configuration.

So anyway, my opinion is that https://github.com/wwmm/easyeffects/issues/656 would be totally sufficient for this, no need to build a more complex UI. Let EasyEffects be easy, and we can recommend using Helvum along with it if you need more complex stream filtering. Maybe an advanced/manual mode where EasyEffects creates the effects but doesn't hook them up in the audio pipeline, so that you can more easily use it alongside a program like Helvum.

[wwmm]

Maybe an advanced/manual mode where EasyEffects creates the effects but doesn't hook them up in the audio pipeline, so that you can more easily use it alongside a program like Helvum.

That is easy to do. The hard part will be to actually implement multiple filter instances. I am not sure when I will have the necessary time to focus on that.