wwmm / easyeffects

Limiter, compressor, convolver, equalizer and auto volume and many other plugins for PipeWire applications
GNU General Public License v3.0
6.18k stars 264 forks source link

Hosting external LV2 Plugins #1433

Open dbrgn opened 2 years ago

dbrgn commented 2 years ago

I'm used to digital audio mixing consoles from Yamaha, and I miss the visual feedback when using the non-graphical equalizer in easyeffects. Here's what the Yamaha CL series EQ looks like:

screenshot-20220315-203805

In the Linux world, I'm a fan of some of the Calf virtual effect devices, for example the graphical multi-band equalizer that is quite similar:

image

Specifically the combination of variable-frequency bands and combination with the audio level visualization in the background are really useful, because it allows you to see how your settings are influencing the mix. It also allows you to see and immediately fix issues like audio feedback.

I searched through the issue tracker regarding graphical equalizers, and found #634, #515 and #663, and it seems that you're using some APIs of existing effect plugins, but that graphing them is difficult, correct?

Might it be an alternative to allow hosting external LV2 plugins directly, similar to the way calfjackhost does it? Would that be possible? (I'm not familiar with the APIs involved, so I don't know what effort this would involve. Calfjackhost _looks_simple, but maybe it isn't.)

wwmm commented 2 years ago

and it seems that you're using some APIs of existing effect plugins, but that graphing them is difficult, correct?

Yes. The plugins can be used like a library. The built-in graphical interface is optional.

Might it be an alternative to allow hosting external LV2 plugins directly, similar to the way calfjackhost does it? Would that be possible?

It is hard to say. I looked into this situation a long time ago and it is a tough thing to handle. The main problem is that you can't under normal circumstances load a plugin that uses one version of a toolkit inside a window that uses a different version. For example you can not load a plugin that uses gtk2 inside a program that uses gtk3. And if you want to embed a plugin that uses QT inside GTK or vice versa things probably get worse.

In most cases plugins hosts seem to workaround that by using this http://drobilla.net/software/suil.html. But there are some disadvantages

It is tough. I would really like to have the graphs. But the disadvantages are huge. If we were a general purpose like Ardour or Carla it would make sense. But we are not.

wwmm commented 2 years ago

But the disadvantages are huge.

And I did not even talked about the possible difficulties in synchronizing what happens in the plugin window with our preset system. So it is far from being something simple to do. Unfortunately.

dbrgn commented 2 years ago

Ah, I see. Since calfjackhost is calf-specific, they can sync their toolkit versions across all plugins, I guess.

Then the best way to use these graphical plugins with PipeWire is probably to go through Jack, along with some setup script, right?

Bleuzen commented 2 years ago

I do agree that having proper interfaces for the plugins would be very useful.

For the downsides:

I looked into this situation a long time ago and it is a tough thing to handle. The main problem is that you can't under normal circumstances load a plugin that uses one version of a toolkit inside a window that uses a different version. For example you can not load a plugin that uses gtk2 inside a program that uses gtk3. And if you want to embed a plugin that uses QT inside GTK or vice versa things probably get worse.

This is indeed a problem. I did also face it while packaging the Calf plugins as flatpak (had to disable their GUI so that they can run in GTK hosts).

However, under all the plugins packaged on flathub, afaik Calf is the only one where we had this issue.

Also there are two more I could think of (just not packaged and not very useful in EasyEffects anyway):

However these are not many. The very most plugins do not suffer from this problem because nowadays they use non-desktop UI toolkits which work perfectly fine alongside Qt and GTK hosts.

So yeah, while some plugins may not work, I wouldn't see this as a blocker. Most plugins will work just fine. This would already be a great benefit. (even without bothering with suil)

For example the LSP plugins already in use by EasyEffects do also use their independent UI toolkit, so EasyEffects could display their custom interfaces without a toolkit collision.

And for the rest... maybe EasyEffects could detect if a plugin is linked to GTK or QT and in that case disallow the "show external UI" button? Or alternatively some kind of allow/exlude-list approach could work.

And I did not even talked about the possible difficulties in synchronizing what happens in the plugin window with our preset system.

Possibly save the plugin states base64 encoded? Ardour seems to do this for VST3s. Not sure if LV2 also supports reading/setting full plugin states easily.

wwmm commented 2 years ago

The very most plugins do not suffer from this problem because nowadays they use non-desktop UI toolkits which work perfectly fine alongside Qt and GTK hosts.

I am not sure it is so straightforward. Based on the LV2 specifications https://lv2plug.in/ns/extensions/ui#X11UI when not using gtk or QT the plugin will create its window drawing directly to X11. A long time ago I saw a super old gtk code that showed a X11 window inside a container specifically designed for this. There was X11 only code everywhere... Besides the obvious problem of having something like this in Wayland times I think that the gtk container that did this isn't available even in gtk3. This may be one of the reasons why Ardour still uses the gtkmm version that wraps gtk2. I seriously doubt gtk4 has this kind of container. Its approach to handle drawings is very different.

For example the LSP plugins already in use by EasyEffects do also use their independent UI toolkit, so EasyEffects could display their custom interfaces without a toolkit collision.

It is not going to be this easy for the reasons I said above. If I am not mistaken this "independent UI toolkit" draws directly to X11 as expected from what the LV2 specs recommend. I doubt it is easy to put a X11 window inside a gtk4 container that is designed to work both on X11 and Wayland. I have never seen something like this in the gtk4 code examples or in its API.

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like... This kind of workflow may be fine for people that are used to do music production in the usual hosts. But I think it would be super weird in EasyEffects to have the plugins controls in other windows.

maybe EasyEffects could detect if a plugin is linked to GTK or QT and in that case disallow the "show external UI" button?

There is no need to disallow anything. It is just a matter of not accessing the window port. We already ignore it because my simple LV2 wrapper does not support the plugin window anyway.

Possibly save the plugin states base64 encoded?

I am not sure it would help. As I said above the LV2 standard has a port(a C pointer) that allows us to interact with the controls in the window. The problem is loading the window and embedding it in a gtk container so it is shown inside EasyEffects window like any other widget.

wwmm commented 2 years ago

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window.

What would still require code capable of displaying a X11 object in a standalone window that works both on X11 and Wayland. But I have the feeling this will require X11 code in our sources. It does not feel the right thing to do....

polarathene commented 1 year ago

In regards to the graphic EQ support, I got a bit lost with the discussion of toolkit concerns (and in related linked issues with how EasyEffects communicates with LSP and the difficulties there for supporting in EE UI).

Does any of that apply to making a separate standalone GUI (be that GTK/Qt or a browser tab web app) that can communicate with EasyEffects (or PipeWire if that makes more sense) to provide controls that update the EasyEffects EQ?

From what I've seen elsewhere, it was suggested to do the EQ in a different program that can export an APO preset?

Is it difficult to support an API (be that over a socket, dbus, or C FFI) to manipulate EasyEffects EQ from the external program? That way EasyEffects can run with the full chain of effects in place and have the EQ adjusted in real-time by an external program that provides different GUI / controls.


Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like... This kind of workflow may be fine for people that are used to do music production in the usual hosts. But I think it would be super weird in EasyEffects to have the plugins controls in other windows.

Perhaps that's what I'm asking for, a way that third-party app could hook into EasyEffects either to control the existing EQ effect or implement it's own EQ effect (apparently this is what an LV2 plugin is suitable for?).

Is the toolkit (and X11 vs Wayland) issue only regarding embedded UI into EasyEffects? Or a problem specific to LV2 support?

Is another option (if doing EQ externally of EasyEffects), to have an effect that has PipeWire routing to the external program and back to EasyEffects? I am still new to PipeWire, but that seemed viable from what I saw inspecting via Helvum.

The EQ visualization would be quite helpful for me, and if there are limitations that prevent EasyEffects supporting that, but don't apply if deferring to an external application, that'd be great to leverage.

wwmm commented 1 year ago

Is it difficult to support an API (be that over a socket, dbus, or C FFI) to manipulate EasyEffects EQ from the external program?

You can configure almost all EasyEffects parameters through any program that uses the GSettings API. Even through command line tools. Take a look at https://github.com/wwmm/easyeffects/discussions/916. The discussion was for PulseEffects but the same idea applies to EasyEffects.

What you are not going to be able to get at this moment are the level meter values. GSettings is not designed for this kind of task so this would require the development of an API.

Is the toolkit (and X11 vs Wayland) issue only regarding embedded UI into EasyEffects? Or a problem specific to LV2 support?

This is way beyond just EasyEffects. Audio hosts like Ardour or Carla also have to deal with this problem. Most of the DAWs running on Linux make use of the suil library to load the custom plugins interface. But besides the fact it probably does not work with gtk4 right now the translation handling will definitely be more complicated when using the plugin custom interface.

Is another option (if doing EQ externally of EasyEffects), to have an effect that has PipeWire routing to the external program and back to EasyEffects? I am still new to PipeWire, but that seemed viable from what I saw inspecting via Helvum.

You can use Helvum to insert a PipeWire filter in the middle of our pipeline. But as EasyEffects isn't designed assuming such level of customization you may have to remake these links quite often.

The EQ visualization would be quite helpful for me, and if there are limitations that prevent EasyEffects supporting that, but don't apply if deferring to an external application, that'd be great to leverage.

Unfortunately this is is impossible to do at this moment unless we write our own equalizer. More details at #515. But besides the fact I do not know how to write one matching the flexibility and quality of the one from Linux Studio Plugins is not going to be a trivial task.

I think that what some users do is loading the Linux Studio Plugins standalone interface to set the parameters and view the graphs. Once the curves are like they want they export an APO preset and import it in EasyEffects.

wwmm commented 1 year ago

Unfortunately this is is impossible to do at this moment unless we write our own equalizer. More details at https://github.com/wwmm/easyeffects/issues/515.

At the time we used GStreamer and it was totally impossible. Nowadays we have our own simple LV2 wrapper and in theory we now have the possibility of adding what was missing in the GStreamer wrapper... So maybe it isn't as impossible as it was in the past. I have to take some time to investigate if dealing with the custom plugin interface port would also be required to access the curves. In any case I doubt it is standard. It is probably necessary to read the plugin code to see what it is doing and pray for its developers do not change how the information is being coded/decoded.

polarathene commented 1 year ago

But as EasyEffects isn't designed assuming such level of customization you may have to remake these links quite often.

But an "effect" that could configure the routing link would resolve that right?

But besides the fact I do not know how to write one matching the flexibility and quality of the one from Linux Studio Plugins is not going to be a trivial task.

I thought that all the settings exposed in the current EQ effect would be the same, and the only difference being requested was regarding the visualization of those settings or alternative UI controls?

Hence if that was an issue for EasyEffects to handle in it's UI, a separate project could have a GUI that manipulates the same settings but can visualize the curve/graph.

It'd just need to update the same settings that EasyEffects handles via the EQ effect, which you've said can mostly be done via GSettings API, while the missing functionality would need some more development to support an API from EasyEffects? That seems alright :)


I have to take some time to investigate if dealing with the custom plugin interface port would also be required to access the curves. In any case I doubt it is standard. It is probably necessary to read the plugin code to see what it is doing and pray for its developers do not change how the information is being coded/decoded.

No worries, I don't expect you to sink time into this :+1:

I was just curious if EasyEffects had the flexibility for supporting it (either via direct contribution to EasyEffects, or a separate app integrating with EasyEffects). I might take a shot at it myself some time :)

It is a feature I often see on videos or guides that teach how to EQ vocals and for me was much easier to interpret than sliders / numbers evenly spaced apart (my ears aren't audiophile grade, but I can read the curves much better while I make adjustments):

vocal cheatsheet

p-chan5 commented 1 year ago

I don't know much about these libraries but I think that feature would have a lot of potential! It could unlock the limits of audio effects creativity ✨

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like...

Maybe you could create a new effect called Wrapper (bridge) that allows to manage external plugins (lv2/vst/vst3). There are some projects that are based on this feature (but for Windows plugins): https://github.com/robbert-vdh/yabridge https://github.com/osxmidi/LinVst

wwmm commented 1 year ago

Maybe you could create a new effect called Wrapper (bridge) that allows to manage external plugins (lv2/vst/vst3)

From a technical point of view I imagine it is possible. But the amount of work required to integrate this in EasyEffects is really huge. Even if we talk about only native Linux plugins. Not only there is the annoyance involved in showing the plugin window or creating a generic one for it but there is also the old problem of integrating everything with our presets infrastructure.

p-chan5 commented 1 year ago

It would be cool to see a feature of that level in EasyEffects! Don't throw this idea away, there must be some way to integrate this. I hope one day it will be possible. 😥️

awfulcooking commented 1 year ago

Something that might work is showing for each plugin a window that is totally independent from the EasyEffects window. But it is not an approach I like... This kind of workflow may be fine for people that are used to do music production in the usual hosts. But I think it would be super weird in EasyEffects to have the plugins controls in other windows.

Most people who'd like to use plugins are used to this workflow already, I think, and at least for me, this would rock / wouldn't be too weird at all.

Then, getting the GUIs to embed be a specific, narrow enhancement for whoever is so daring :-)

Storing the plugin's state as an opaque blob, without GTK controls (only managed by its own GUI), would unlock at least 80% of the utility for most too, I think :-)

wwmm commented 1 year ago

Most people who'd like to use plugins are used to this workflow already

I am not so sure about that. There are probably many that just want a fast and easy way to setup some global effects and moving on with their lives.

Storing the plugin's state as an opaque blob, without GTK controls (only managed by its own GUI)

This is almost on the same level as throwing away what we have right now and writing a new app. I know we don't really have to throw anything away. But in the end there would be two totally different paths that would have to coexist. The development and maintenance burden would increase considerably.

awfulcooking commented 1 year ago

Thanks @wwmm.

This is almost on the same level as throwing away what we have right now and writing a new app

From EasyEffects' perspective, couldn't it be just like any other effect module?

The UI for the effect when selected, would be:

State for the LV2 params stored in gsettings, but not presented.

There are probably many that just want a fast and easy way to setup some global effects and moving on with their lives.

For this, they have the wonderful existing effects already? :-)

wwmm commented 1 year ago

From EasyEffects' perspective, couldn't it be just like any other effect module?

I was talking about the part where "the settings were stored in a blob". We make heavy use of gsettings to manage our settings. Having an alternative way to handle them that is on the level of making another app.

selector to change the LV2 plugin used

I did not understand this. Changing the lv2 plugin being used?

State for the LV2 params stored in gsettings, but not presented.

This would mean destroying/recreating all the bindings between gtk widgets and the gsettings database based on whether the plugin native window is visible. Doable but a fair amount of code refactoring would be necessary.

For this, they have the wonderful existing effects already? :-)

If the current custom widgets remain available yes. I thought you meant a replacement of one approach by the other.

In any case I do not know yet if it is even viable to just show the plugin UI outside of Easyeffects window without major complications.

p-chan5 commented 1 year ago

I think a simpler approach would be easier to develop. What do you think about hosting only the ports leaving out the interfaces? Something like what happens with LADSPA plugins.

Only then, we would have to think of a container and a way to store the settings in gsettings

For example, if you want to load them, go to Add Effect > LADSPA (subcategory located to the right or above the main list) > Plus > Load .so file.

wwmm commented 1 year ago

I think a simpler approach would be easier to develop. What do you think about hosting only the ports leaving out the interfaces? Something like what happens with LADSPA plugins.

We already ignore the graphical interface. The only difference in your idea is not designing the widgets layout. But this approach is not going to work well with plugins that have an insane amount of parameters or parameters whose activation depend on others.

p-chan5 commented 1 year ago

But this approach is not going to work well with plugins that have an insane amount of parameters or parameters whose activation depend on others.

You're right, there are plugins that abuse this, maybe showing all its parameters (right click > show all) is the solution to this.

I am not so sure about that. There are probably many that just want a fast and easy way to setup some global effects and moving on with their lives.

Take it as a suggestion. I really don't see an easier way than that, ie some end users use LADSPA plugins in programs like Audacity or OBS.

awfulcooking commented 1 year ago

But this approach is not going to work well with plugins that have an insane amount of parameters or parameters whose activation depend on others.

I wondered if they could be managed only from the LV2 plugin's own UI, instead of replicated, for this reason.

I've never been satisfied with hosted knobs laid out by Carla or Bitwig.

Each plugin's own UI tends to be so characteristic & well-designed..

wwmm commented 1 year ago

Each plugin's own UI tends to be so characteristic & well-designed..

Not always but usually yes. But unfortunately there are all the technical difficulties I talked about before. Loading the plugin window is a pain because of the graphical toolkit dilemma. And you do not have a way to translate the window widgets labels anymore as this is now in the hands of the plugin window developer.

p-chan5 commented 1 year ago

I've never been satisfied with hosted knobs laid out by Carla or Bitwig.

This will also depend on the utility you want to give the plugin. The graphical interfaces are very useful, however EE's approach is different and the code should work based on that.

mickaelistria commented 1 year ago

There are probably many that just want a fast and easy way to setup some global effects and moving on with their lives.

FWIW, I'm very noob with sound effects in general. However, I recently grew found of EasyEffects to setup some music quizzes applying some extra effects upon tracks to add some complexity and fun. The great thing with EasyEffects is that it's trivial to transform the actual sound output, so I can apply an effect to any sound without pre-processing it. With EasyEffects, I can go to youtube or spotify and pick a music and the configured effects are applied, without need to prepare anything else than EasyEffects pre-sets, I can choose the tracks I want to play as the event goes on. However, there are some funny effects that most likely won't be available, in EasyEffects, but that are available as VST plugins. While I don't think EasyEffects can nor should try to directly integrate all possible effects, I think an interface allowing to delegate an effect to a VST would ideally serve my use-case. As I imagine it, I would expect that in the list of effect would be add some "Add external effect" entry, and when clicking it, I would be asked for a VST to use, and then in EasyEffects, this VST would appear in the list of applied effects. On the effect configuration page, EasyEffects could offer a simple button "Open effect interface" that would open the plugin UI, in a new window if needs be (where it's rendered doesn't really matter), let me configure it, and EasyEffects would apply the configured effect. However, I don't think I'll be able to contribute anything towards that anytime soon... So, I apologize in advance towards maintainers if this comment is less constructive than I hope it can be.

wwmm commented 1 year ago

So, I apologize in advance towards maintainers if this comment is less constructive than I hope it can be.

No problems :-)

I think an interface allowing to delegate an effect to a VST would ideally serve my use-case.

The first difficulty is being able to load a VST plugin and wrapping it in a PipeWire's pw_filter. For LV2 I have written my own simple wrapper. But there is nothing in EasyEffects for VST. So probably something from scratch would have to be done for them.

The second problem is that none of these custom VST/LV2 plugin settings are going to be saved to EasyEffects presets. Our presets infrastructure assumes that there are gsettings xml file schemas available for each plugin. But in the case of an user custom plugin there isn't a gsettings xml schema. And even if we tried to create this whole thing at runtime it would be pointless without knowing each plugin parameter type, name, range, etc.

In any case it only makes sense to think about what would happen to the preset files if we are able to load the plugin. For VST there is nothing and even for LV2 the support available in my wrapper is only partial. There are LV2 plugins it can not load because I did not understand how to implement all of the possible LV2 protocols. And on top of that there is also the native graphical interface situation.

wwmm commented 1 year ago

And even if we tried to create this whole thing at runtime it would be pointless without knowing each plugin parameter type, name, range, etc.

What would lead to another situation. Implementing a code that would query this whole information at runtime in a reliable way. For the plugins currently provided by EasyEffects I manually opened the plugin ttl file in a text editor and got the information. There is no runtime query implemented at this moment.

wwmm commented 1 year ago

Although not the direct topic of this discussion the latest changes in our master branch are related to this issue. We can finally see the native window of some LV2 plugins Screenshot from 2023-06-20 13-10-20

For now only the equalizer, the single band compressor and the limiter. More LSP plugins in the next days. It is nice that we can finally see the frequency response curves but have in mind that while usable there are still lots of rough edges to be fixed:

In any case I'd say this is a huge step forward.

wwmm commented 1 year ago

but have in mind that while usable there are still lots of rough edges to be fixed:

Another one is that the pipeline must be active to be able to view the window. This could be improved in the future. What is happening is that to show the window the plugin instance is required. But we only initialize it when we start to receive audio frames.

p-chan5 commented 1 year ago

It is a big step! You and your development team are amazing!

You can also ask falkTX (Carla's creator) for help. I mean your project is very good, so if you join together, together you could do even better.

awfulcooking commented 1 year ago

I have a patch which attempts to render widgets via Suil, but trouble is, its GTK4 support hasn't been merged upstream.

I can share my WIP if it's any use.

I wonder if @alex-tee has any tips. The authored the GTK4 support patch for Suil. And integrated it into Zrythm, I think. But I'm having trouble grokking how :-)

wwmm commented 1 year ago

I have a patch which attempts to render widgets via Suil, but trouble is, its GTK4 support hasn't been merged upstream.

That is why I have been avoiding trying to use it. Even if I make it work with gtk4 now once gtk5 becomes available I am not going to stay in gtk4 because of suil. So all the work to make it work with gtk would have to be done again...

I can share my WIP if it's any use.

Show it. Maybe it helps. Do you know if suil is useful only to embed the native ui in the chosen toolkit or is it also useful when we want the native ui to be in a standalone window isolated from the host tookit? After seeing how slow the native plugin window can be I do not think that replacing our custom widgets by it is the way to go. It is better to just show it in a separated window when the frequency response is needed and keeping the controls in our custom widgets.

Right now Calf plugins ui are crashing on my wrapper because they try to load gtk2 widgets in an environment set for gtk4. Even if I do not try to embed them in our window...

awfulcooking commented 1 year ago

I think Suil handles embedding in a parent window, as well as separate. It's designed to embed LV2 widgets from various toolkits, into various toolkits. It just doesn't have GTK4-as-a-host support upstream yet. I hear your preference to not be tied to Suil release cadence. Alex's patch did come early. If this feature went away until Suil supported GTK5, at least it would be supported for GTK4..

This patch is messy, no longer cleanly applies, but passing it over as a sketch. (If you drop the changes to {src/,}meson.build, and change LV2_UI__Gtk4UI to LV2_UI__Gtk3UI in lv2_wrapper.cpp, then it would git apply on 35f7ef57 from Jun 4)

I was trying to figure out how Zrythm includes its Suil Gtk4 support tonight, but didn't get there. This is the diff of my working tree as it stands.

Patch ```diff diff --git a/data/ui/bass_enhancer.ui b/data/ui/bass_enhancer.ui index 26bee9a5..afa0ddc2 100644 --- a/data/ui/bass_enhancer.ui +++ b/data/ui/bass_enhancer.ui @@ -14,6 +14,12 @@ + + + 12 + + + 12 diff --git a/include/bass_enhancer.hpp b/include/bass_enhancer.hpp index 28653042..f3b21956 100644 --- a/include/bass_enhancer.hpp +++ b/include/bass_enhancer.hpp @@ -47,6 +47,6 @@ class BassEnhancer : public PluginBase { double harmonics_port_value = 0.0; - private: + // private: std::unique_ptr lv2_wrapper; }; diff --git a/include/lv2_wrapper.hpp b/include/lv2_wrapper.hpp index 668b569e..30ca082a 100644 --- a/include/lv2_wrapper.hpp +++ b/include/lv2_wrapper.hpp @@ -33,6 +33,10 @@ #include "string_literal_wrapper.hpp" #include "util.hpp" +#include +#include +#include + namespace lv2 { using namespace std::string_literals; @@ -68,6 +72,12 @@ class Lv2Wrapper { auto create_instance(const uint& rate) -> bool; + void initialize_ui(); + + GtkWidget* get_ui_widget(); + void show_ui(); + void hide_ui(); + void set_n_samples(const uint& value); [[nodiscard]] auto get_n_samples() const -> uint; @@ -191,6 +201,11 @@ class Lv2Wrapper { LilvInstance* instance = nullptr; + SuilHost* suil_host; + SuilInstance* suil_instance; + GtkWidget* plugin_ui; + bool ui_initialized; + uint n_ports = 0U; uint n_audio_in = 0U; uint n_audio_out = 0U; diff --git a/meson.build b/meson.build index 45253a2f..37e347b5 100644 --- a/meson.build +++ b/meson.build @@ -44,6 +44,7 @@ bindir = join_paths(prefix, get_option('bindir')) datadir = join_paths(prefix, get_option('datadir')) localedir = join_paths(prefix, get_option('localedir')) include_dir = include_directories('include') +suil_include_dir = include_directories('/g/Programs/zrythm/src/plugins/lv2') config_h_dir = include_directories('.') status = [] diff --git a/src/bass_enhancer.cpp b/src/bass_enhancer.cpp index da7e9bdf..594f2a6a 100644 --- a/src/bass_enhancer.cpp +++ b/src/bass_enhancer.cpp @@ -31,7 +31,7 @@ BassEnhancer::BassEnhancer(const std::string& tag, util::debug(log_tag + "http://calf.sourceforge.net/plugins/BassEnhancer is not installed"); } - lv2_wrapper->bind_key_double_db<"amount", "amount">(settings); + this->lv2_wrapper->bind_key_double_db<"amount", "amount">(settings); lv2_wrapper->bind_key_double<"drive", "harmonics">(settings); diff --git a/src/bass_enhancer_ui.cpp b/src/bass_enhancer_ui.cpp index 4f5b0263..2217457c 100644 --- a/src/bass_enhancer_ui.cpp +++ b/src/bass_enhancer_ui.cpp @@ -37,6 +37,8 @@ struct Data { struct _BassEnhancerBox { GtkBox parent_instance; + GtkBox* lv2_ui_box; + GtkScale *input_gain, *output_gain; GtkLevelBar *input_level_left, *input_level_right, *output_level_left, *output_level_right; @@ -81,6 +83,13 @@ void setup(BassEnhancerBox* self, std::shared_ptr bass_enhancer, c bass_enhancer->set_post_messages(true); + if (self->lv2_ui_box == NULL) { + std::cerr << "LV2 UI BOX IS NULL" << std::endl; + } + + GtkWidget* lv2_widget = bass_enhancer->lv2_wrapper->get_ui_widget(); + gtk_box_append(self->lv2_ui_box, lv2_widget); + self->data->connections.push_back(bass_enhancer->input_level.connect([=](const float left, const float right) { util::idle_add([=]() { if (get_ignore_filter_idle_add(serial)) { @@ -171,6 +180,7 @@ void bass_enhancer_box_class_init(BassEnhancerBoxClass* klass) { gtk_widget_class_set_template_from_resource(widget_class, tags::resources::bass_enhancer_ui); + gtk_widget_class_bind_template_child(widget_class, BassEnhancerBox, lv2_ui_box); gtk_widget_class_bind_template_child(widget_class, BassEnhancerBox, input_gain); gtk_widget_class_bind_template_child(widget_class, BassEnhancerBox, output_gain); gtk_widget_class_bind_template_child(widget_class, BassEnhancerBox, input_level_left); diff --git a/src/lv2_wrapper.cpp b/src/lv2_wrapper.cpp index 5441c11b..5cda8964 100644 --- a/src/lv2_wrapper.cpp +++ b/src/lv2_wrapper.cpp @@ -73,6 +73,14 @@ Lv2Wrapper::Lv2Wrapper(const std::string& plugin_uri) : plugin_uri(plugin_uri), } Lv2Wrapper::~Lv2Wrapper() { + // Clean up Suil resources + if (suil_instance) { + suil_instance_free(suil_instance); + } + if (suil_host) { + suil_host_free(suil_host); + } + if (instance != nullptr) { lilv_instance_deactivate(instance); lilv_instance_free(instance); @@ -234,6 +242,72 @@ auto Lv2Wrapper::create_instance(const uint& rate) -> bool { return true; } +void Lv2Wrapper::initialize_ui() { + if (ui_initialized) { + return; + } + + // Initialize Suil + suil_host = suil_host_new(nullptr, nullptr, nullptr, nullptr); + if (!suil_host) { + std::cerr << "Failed to initialize Suil host" << std::endl; + return; + } + + // Get the UI URI for the plugin + LilvNodes* ui_uris = lilv_plugin_get_uis(plugin); + const LilvNode* ui_uri = nullptr; + LILV_FOREACH(nodes, i, ui_uris) { + const LilvNode* node = lilv_nodes_get(ui_uris, i); + if (lilv_node_is_uri(node)) { + ui_uri = node; + break; + } + } + + if (!ui_uri) { + std::cerr << "No suitable UI found for the plugin" << std::endl; + return; + } + + // Create a Suil instance for the plugin UI + suil_instance = suil_instance_new(suil_host, nullptr, LV2_UI__Gtk4UI, lilv_node_as_uri(lilv_plugin_get_uri(plugin)), + lilv_node_as_uri(ui_uri), lilv_instance_get_descriptor(instance)->URI, + lilv_node_as_string(ui_uri), nullptr, nullptr); + + if (!suil_instance) { + std::cerr << "Failed to create Suil instance for the plugin UI" << std::endl; + return; + } + + SuilWidget tmp = suil_instance_get_widget(suil_instance); + + + // Get the GtkWidget for the plugin UI + plugin_ui = (GtkWidget*)suil_instance_get_widget(suil_instance); + + // Mark the UI as initialized + ui_initialized = true; + + std::cerr << "Lv2 UI initialized okay" << std::endl; +} + +void Lv2Wrapper::show_ui() {} + +GtkWidget* Lv2Wrapper::get_ui_widget() { + if (!ui_initialized) { + initialize_ui(); + } + + return plugin_ui; +} + +void Lv2Wrapper::hide_ui() { + if (plugin_ui) { + gtk_widget_hide(plugin_ui); + } +} + void Lv2Wrapper::connect_control_ports() { for (auto& p : ports) { if (p.type == PortType::TYPE_CONTROL) { diff --git a/src/meson.build b/src/meson.build index 414ad2d9..0a4ce317 100644 --- a/src/meson.build +++ b/src/meson.build @@ -125,6 +125,8 @@ cxx = meson.get_compiler('cpp') zita_convolver = cxx.find_library('zita-convolver', required: true) +add_project_arguments('-DLV2_UI__Gtk4UI="https://lv2plug.in/ns/extensions/ui#Gtk4UI"', language : [ 'c', 'cpp' ]) + # always require these libraries if the respective meson option is enabled, so they can't be accidentally left out rnnoise = dependency('rnnoise', include_type: 'system', required: get_option('enable-rnnoise')) @@ -143,6 +145,8 @@ if get_option('enable-libportal') status += 'Using libportal to handle autostart files.' endif +suil_h_dir = include_directories('/g/Programs/zrythm/src/plugins/lv2') + easyeffects_deps = [ dependency('libpipewire-0.3', version: '>=0.3.58', include_type: 'system'), dependency('glib-2.0', version: '>=2.56', include_type: 'system'), @@ -151,6 +155,7 @@ easyeffects_deps = [ dependency('sigc++-3.0', version: '>=3.0.6', include_type: 'system'), dependency('lilv-0', version: '>=0.22', include_type: 'system'), dependency('lv2', version: '>=1.18.2', include_type: 'system'), + dependency('suil-0', version: '>=0.10.18', include_type: 'system'), dependency('libbs2b', include_type: 'system'), dependency('sndfile', include_type: 'system'), dependency('fftw3f', include_type: 'system'), @@ -173,7 +178,7 @@ easyeffects_deps = [ executable( meson.project_name(), easyeffects_sources, - include_directories : [include_dir,config_h_dir], + include_directories : [include_dir,config_h_dir,suil_h_dir], dependencies : easyeffects_deps, install: true, link_args: link_args ```
alex-tee commented 1 year ago

Zrythm doesn't use suil anymore. It uses carla as a library for plugin hosting. See:

Carla has an option to run plugins in "bridge mode" (i.e., out of process). this fixes issues like with calf plugins and also offers crash protection

awfulcooking commented 1 year ago

Thanks @alex-tee

wwmm commented 1 year ago

Most of the LSP native ui controls can be used now. The widgets in our window synchronize with what was done there when the native ui is closed.

Now of the major problems related to the LSP ui the only one remaining is this weird memory leak when the native ui is closed.

PowerUser64 commented 2 months ago

Would it be possible to add something like https://github.com/falkTX/Carla as an effect and allow crafting a custom effect chain inside it? Carla is a plugin host that itself can run as a plugin or as a standalone JACK application. It supports all major plugin formats and provides a patchbay UI for connecting them together.

wwmm commented 2 months ago

Would it be possible to add something like https://github.com/falkTX/Carla as an effect and allow crafting a custom effect chain inside it? Carla is a plugin host that itself can run as a plugin or as a standalone JACK application. It supports all major plugin formats and provides a patchbay UI for connecting them together.

Even if there is a way to insert an external host pipeline inside EE pipeline I doubt it will be possible to properly integrate the plugins that are there with the rest of EE code. Our presets for example.

PowerUser64 commented 2 months ago

Could you add it like you add LSP plugins though? Carla itself runs as a VST/LV2 plugin or JACK application, much like LSP.

wwmm commented 2 months ago

Could you add it like you add LSP plugins though? Carla itself runs as a VST/LV2 plugin or JACK application, much like LSP.

I have no idea. The LSP plugins we use dp not try to act as a host for other plugins. And we are not using its standalone version but the traditional LV2 plugin. It is not the same as putting a whole host in there.