Any plans to implement a VST host in rust?

[ZECTBynmo]

I would LOVE to see that happen!

[overdrivenpotato]

This is definitely doable, it would essentially involve wrapping a dynamically loaded library (the plugin) in a some type of struct that handles passing VST opcodes between the host rust program and the plugin.

If you (or anybody else) wants to try this, it'd probably look something like this:

struct Plugin {...}
impl Plugin {
    pub fn load(...) -> Plugin {...}

    // For example    
    pub fn init(&mut self) {
        // Send OpCode::Initialize to loaded library via vst2::api::AEffect::dispatcher
        let opcode = vst2::plugin::OpCode::Initialize.into();
        self.effect.dispatcher(self.effect_ptr, opcode, 0, 0, ptr::null_mut(), 0.0);
    }

    // Rest of VST opcodes ...
}

I don't have any plans at the moment to support this but if I have some free time I might be able to cook up a prototype

[ZECTBynmo]

What are your thoughts about the process of showing the editors of hosted plugins from the DAW? Do you think it's possible? Easy? What would your instincts be about the first steps?

[overdrivenpotato]

I think it'd be actually fairly simple as the VST API only really requires a window pointer to be passed to the plugin. It would have to be a platform specific window pointer though, e.g. X11, WinAPI or Carbon on OS X (optionally Cocoa if the plugin and host both support it). Pretty sure this library would only have to open the platform specific editor window and pass it to the plugin for that to work.

It gets a little bit more complex if you're building a plugin, as to actually interact with the editor window you need to use platform-specific APIs. That is a bit of a pain especially in rust, so there could potentially be another library built off this one that handles the platform specific stuff

[ZECTBynmo]

I'm actually only interested in the VST host part, so that sounds awesome! Any chance you might take a swing at it? I'd be happy to donate a few bucks!

[overdrivenpotato]

Yeah for sure, if I get some time this week/next week I'll try and get a working implementation

[ZECTBynmo]

Hey @overdrivenpotato if you can get a rust VST host working that can show client VST GUIs and process audio, I'll send you $50 via PayPal

[overdrivenpotato]

@ZECTBynmo any example plugins you'd like to load?

[ZECTBynmo]

@overdrivenpotato how about: iZotope Ozone 5 (the most important one - and likely the most difficult) NI Omnisphere Guitar Rig 5

[overdrivenpotato]

Alright that seems doable, I'll update you when I get something running

[ZECTBynmo]

Awesome! Looking forward to it

[overdrivenpotato]

Just a quick update: most of the VST-related programming for this to work is finished, and I have been able to load and play around with Ozone 5. The only remaining problem is passing a window pointer to the plugin. I've gotten this to work via a forked version of glutin but I'm currently just writing a simpler stripped down window library to give more flexible options. It might just be a submodule in rust-vst2 as it is fairly specific but I'm debating just creating a new repository and pushing it to crates.io.

[ZECTBynmo]

Wow that is so cool @overdrivenpotato!!! I'm super excited to see it!

[Boscop]

Hey @overdrivenpotato, I'm also very interested in a VST host in Rust! Which window library are you using? Are you using https://github.com/RustAudio/dsp-chain for the processing?

[overdrivenpotato]

I was using glutin initially as the window library, but with most of the functionality removed and with modifications to allow plugins to properly draw. I'm currently just writing a really small module to do exactly what glutin did but tailored to the VSTAPI. Processing is up to the plugin to implement as custom functionality so at the moment dsp-chain would be more useful for plugin developers.

[ZECTBynmo]

@overdrivenpotato do I owe you $50 yet? :)

[overdrivenpotato]

@ZECTBynmo everything except the editor should be functional. It ended up being a lot of work to get a window working (cross platform, OpenGL, key/mouse events, etc). If anybody is interested in resurrecting this project with me, I could pick it back up again but I haven't had the need to get an editor open in my own projects.

[Boscop]

@overdrivenpotato: I'm interested in this as well. I haven't done a host yet but I've worked on the plugin editor GUI. It would be useful if you can put your host on github, so we can collaborate.. (I didn't know how to solve the problems I ran into with my vstscanner so I didn't continue working on the host yet.) I've gotten an editor working with my fork of glutin but sometimes it crashes when I have many plugin instances' GUI open, when removing one plugin instance... I'm still trying to debug this, I think it's related to the way the message queue for the window are removed when the instance is removed: https://github.com/Boscop/glutin/blob/master/src/api/win32/mod.rs#L468 Since all plugin instances' windows must run in the same thread (messages from parent window to child window can't cross thread boundaries) they all have to store a message queue. The original glutin lib spawns a new thread for each window and doesn't support child windows. One of the things I did in my glutin fork is make sure it spawns child windows in the same thread. This also means they have to store the msg queue ThreadLocalData in the same thread, so I changed the CONTEXT_STASH from just ThreadLocalData to HashMap<winapi::HWND, ThreadLocalData>, using the each window's HWND handle as key: https://github.com/Boscop/glutin/blob/master/src/api/win32/callback.rs#L23 So when a message is sent, it stores it in the message queue associated with that window: https://github.com/Boscop/glutin/blob/master/src/api/win32/callback.rs#L43 I think the bug might have to do with the fact that the Destroy message might be sent after drop is called? But that shouldn't crash since if there is no such HWND key in the hashmap, it just returns.

I would really appreciate if someone could look into this so we can debug this, if we can debug this, we can use glutin with glium to render beautiful opengl UIs with shaders etc. We could also use conrod on top of my glutin fork with the glutin_glium backend!

[overdrivenpotato]

The WIP window handling that I have is very rough and has no support for win32. I figured I'd get it working on OSX first, and then focus on win32 / X support.

Using glutin I was able to load up a few iZotope plugins, but the module I began writing was never completed and I left it in a dysfunctional state. The documentation gets a little hazy when it comes to native window APIs, and Glutin had some issues with screen refreshes so the UI was very inconsistent. I guess we could proceed by using a glutin fork, but I have no easy access to a windows machine right now so I can't help out with debugging any errors that aren't OSX.

@Boscop It seems that you're looking into creating UIs for plugins, the fork I mentioned was supposed to create windows for loaded plugins to use.

As an aside: I'm still not sure how viable conrod is as a UI library, my personal goal would be to integrate CEF using something like cef-rs. I don't think the performance implications would be bad considering the chromium UI would only be active while the plugin UI is open (usually only a few on screen at a time), and the DSP code would still be entirely rust.

[Boscop]

Usually the host runs the UI in a separate thread from the audio anyway, because ASIO needs the lowest latency (some say, don't even allocate in the ASIO thread, but the way we did the midi event handling allocates... Usually a preallocated buffer is used that should be large enough for all midi events that could happen in a frame). I'm not familiar with CEF, is it like NW.js? I think it would be interesting to go that route, as long as it runs in its own thread. One could design GUIs in CSS etc. Btw, VSTGUI uses a XML description of the plugins to describe the interface of the plugin.

[zyvitski]

Maybe there is a way of using Electron from GitHub to do the gui?

[overdrivenpotato]

Something like electron would be perfect, but I don't think electron is designed to be embedded on it's own. If we could get CEF working with a rust-vst API to communicate control changes and audio information, that would allow plugin developers to design professional UIs really quickly with existing tooling.

I have a test project with electron that looks like this at the moment: But I have a feeling that integrating CEF will be a lot of work. Perhaps we could create another library which uses the exposed editor trait in this one, to keep the VSTAPI guts separated from a CEF UI.

Let's move discussion over to #9

[ZECTBynmo]

Having gone pretty far down this road for C++ VST plugins, embedding CEF in a useful way as a pure control surface is easy, but getting hooks into the chromium audio loop is very difficult. Chromium has no hooks to give external applications access to lower level audio streams, and there's no reliable way to capture page audio via injected client side JS. If there's a workaround or some new audio hooks I'd love to hear about it, but IMHO this is more difficult than it sounds.

On Thu, Jan 26, 2017 at 8:52 PM, Marko Mijalkovic notifications@github.com wrote:

Something like electron would be perfect, but I don't think electron is designed to be embedded on it's own. If we could get CEF working with a rust-vst API to communicate control changes and audio information, that would allow plugin developers to design professional UIs really quickly with existing tooling.

I have a test project with electron that looks like this at the moment: [image: electron] https://camo.githubusercontent.com/4585d84b75e52221a8de57d88f21015a6b6a976b/687474703a2f2f692e696d6775722e636f6d2f6f4268323464712e706e67 But I have a feeling that integrating CEF will be a lot of work. Perhaps we could create another library which uses the exposed editor trait in this one, to keep the VSTAPI guts separated from a CEF UI.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/overdrivenpotato/rust-vst2/issues/7#issuecomment-275570155, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKo2E6YOsuIgN0kyfhoH4O6WwFOAPGPks5rWU3wgaJpZM4GDmE8 .

[zyvitski]

You don't need to get hooks into the chromium audio loop though. You just use a library that will talk directory to the machines audio. Something like PortAudio which is a well know audio API and has bindings for Rust. The CEF window would serve only as a control surface for an underlying application. Or in the case of VST plugins you would only have to have the window communicate with the plugin because the VST is being hosted by a program that takes care of the audio io for you.

[ZECTBynmo]

@zyvitski Unfortunately hooking into system audio is also difficult to do (at least on OSX). Things like PortAudio can monitor samples for existing audio inputs/outputs, but they can't get a hook into the global system output (which is where Electron audio would go). Products like SoundFlower exist to create an additional output device that acts as you'd want, but they work by patching the kernel, so it's a very high-touch solution.

[overdrivenpotato]

I think there are 2 different discussions going on here. CEF would only be useful for plugin creators.

When it comes to DAW development, we only need a window opened via native system APIs, no CEF involved. The rest is up to the loaded plugin. This library should be able to integrate into any DAW without imposing UI restriction​s.

Regarding creating plugins, if we use CEF we would need some sort of channel to communicate control changes, but the actual audio processing would still be written in rust. It would be up to the DAW loading the plugin to provide us with an audio buffer we have to fill.

On Fri, Jan 27, 2017, 3:22 PM Mike Vegeto notifications@github.com wrote:

@zyvitski https://github.com/zyvitski Unfortunately hooking into system audio is also difficult to do (at least on OSX). Things like PortAudio can monitor samples for existing audio inputs, but they can't get a hook into the global system output. Products like SoundFlower exist to create an additional sound output that acts as you'd want, but they work by patching the kernel, so it's a very high-touch solution.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/overdrivenpotato/rust-vst2/issues/7#issuecomment-275764904, or mute the thread https://github.com/notifications/unsubscribe-auth/AE_5eop6wRO4VX0RLQ_S7dg3yOH-9BrVks5rWlH0gaJpZM4GDmE8 .

[ZECTBynmo]

Agreed on all points - CEF isn't relevant to opening existing plugin UIs, and if a CEF UI library is built keeping processing in Rust would dodge questions about the Chromium audio loop.

On the topic of opening Ozone, I'm almost positive that iZotope plugins (and likely others) will crash if they're not instantiated on the host's main thread. That may have been one of the issues preventing it from working as expected.

[zyvitski]

My point though is that in regards to the creation of a DAW or even a minimal VST host. You wouldn't need to be able to create it destroy virtual audio devices, all you would need is the ability to interface with an existing device. If the user wanted to interact with a different device then they could use something like sound flower to create one which should be recognized by a library like portaudio as a valid device and thus giving the ability to connect to it.

[piedoom]

Excuse me if I'm being dense (I'm still starting out with Rust & VST development), but wouldn't something like Electron cause a lot of unnecessary overhead? Apps that utilize HTML frameworks like Slack/Discord tend to feel very sluggish compared to native stuff. I might be wrong, though! Screenshot looks really cool @overdrivenpotato :)

[zyvitski]

@piedoom there would most likely be extra overhead using electron. But the important things to note would we that the overhead would not be tied to the audio processing thread so there wouldn't be any performance differences to the actual audio and that the using web UI tech would make creating GUI's much easier. So it's somewhat of a trade off. It's also worth noting that the scale of the application being written does factor in, for a simple UI Electron may perform just as well as a native UI. It all depends on how complex the UI is and how much animation is being used, etc...

[Boscop]

Maybe integrate servo, to shift the rendering to the GPU ;) Is there something like CEF / Electron for servo? There should be! EDIT: oh look! They are implementing the CEF interface in rust, with servo as backend. So it should render on the GPU.

From my experience, it's much easier to make nice GUIs with web technologies than with hardcoding stuff. Separation of concerns. You can iterate much faster if you develop the GUI outside of the plugin because building and linking the plugin dll takes forever. But with browserSync you hit save in the editor and the Browser reloads the page.. It also makes it possible to easily have resizable / scalable vector GUIs like Meldaproduction. Or you could go one step further and define an extension to the VST standard where plugins have a non-graphical API that exposes all functionality that the GUI would, and then the host opens one browser window where all plugin GUIs are displayed (the host can let the user decide if they should be shown as a graph like Jeskola Buzz or in channels like Ableton or whatever! This would really reduce the overhead since you'd only have one browser environment, not as many as plugins you have open! And you could remote control the plugins over LAN etc. from the DJ booth, delegating to different machines to parallelize, so you can have hundrets of plugins working together in a live performance.. This API could work over OSC (it also has binary blob messages, so you could transfer audio buffers for visualization). (This API would also make the plugins more accessible for blind people etc.)

As for the audio, the host needs to open the audio device, not the plugin. The host can use something like portaudio, RtAudio, or any platform-specific api like ASIO directly to open devices. The plugins are given a buffer to read/write by the host. If audio should be visualized in the plugin, it has to pass the audio buffer to the GUI, so if the GUI and audio thread are different, it should probably do that in a sync way (i.e. mutex or message passing). In my current plugin I don't do it in a sync way and it still works (but I haven't tested it in a lot of hosts).

[cdbattags]

Any updates on this? I've been looking around for something like https://github.com/teragonaudio/MrsWatson in Rust!

[klingtnet]

@cdbattags I think this project is orphaned, take at look at this fork github.com/rust-dsp/rust-vst that is actively developed.