WebAudioPlugin as VST

[gcperrin]

Hey all -- been heavily investigating this technology, have a thought / question.

Ultimately it will take time for the Web-based DAWs to come into stability and production ready builds. Is it worth considering a WebAudioPlugin bridge to back-connect to existing DAWs as a VST?

Been thinking about how this could be accomplished...perhaps a native vst bridge that takes incoming audio from the browser as an input, and outputs to the DAW? Planning to test an implementation with JUCE and a WASM module, hopefully though a shared memory buffer.

This could be separate to the current spec, but would be curious to hear any thoughts or feedback on integration, design, et. cetera.

[micbuffa]

Hi Jari Kleimola demoed this as a proof of concept at the WebAudio Conference 2018. Look at his paper(https://www.researchgate.net/profile/Jari_Kleimola/publication/331990070_Native_Web_Audio_API_Plugins/links/5c9a16d1a6fdccd4603b85a5/Native-Web-Audio-API-Plugins.pdf) + video presentation (https://www.youtube.com/watch?v=Kpv6_EU9YmE&list=PLQ9S5oqvpyWm2-HDGnQWWlX30pqYlb_Rz&index=22)

The idea was to compile Chrome Embedded Framework a sa VST that would load the WebAudio applications. This worked pretty well in Ableton for example, the video shows our virtual pedalboard web app that is a WebAudioPlugin host, loaded into Ableton. Jari showed that it adds just 3ms of latency. I think he did not go further that the proof of concept. I have this running on my machine. I can load different WebAudio application in the WebAudio VST wrapper and use them in Ableton.

[micbuffa]

This is a bit old, but it presents this idea back in 2018. See the slides near the end... https://docs.google.com/presentation/d/1BCvYXa57Zcc9jtKfGmEwyd9-e8jvunvDcNrg8nWGQmY/edit?usp=sharing

[gcperrin]

Thanks for the reply!

I'd like to do some testing...is the WebAudio VST wrapper openly available?

Also, would love to contribute to the spec/implementation in any way I can.

[micbuffa]

Ah, Jari Kleimola has the source code, it's not publically available. I'll do a visio with him and ask. Can you tell us who you are / what are your motivations / what do you do. I'm a professor at University Côte d'Azur in France, I'm the supervisor of a national research project with WebAudio as core technology. Jari was a researcher in Sweden, and he co-created AmpedStudio, an online commercial DAW (ampedstudio.com).

[Catsvilles]

Hi everyone, I'm also investigating this option currently, would be cool if we would be able to do cross-platform audio plugins (instruments, effects) just with Web Audio and JavaScript. Funny coincidence but a few days ago I also was reading and going through Jari Kleimola's published works and video presentation. In the end of the video he sends us to this link, where source code should reside: https://github.com/jariseon/web-audio-vst

[jariseon]

hi, would be cool to proceed past the proof-of-concept stage with this. the repo still only contains cef and chromium patches though. i seriously need to find time to push the vst plugin part as well. sorry about that.

browser wrapping might actually be more straight forward with the new chromium audio stack: chromium's native audio part runs nowadays in a separate process, so stabbing that codebase might drop the requirement for CEF.

microsoft's chromium-based webview is in developer preview. anyone know if their full chromium fork is open sourced?

[gcperrin]

@micbuffa @jariseon no worries, thank you for the help.

I'm out of Boulder, Colorado...Electrical Engineer (CU Boulder) specializing in DSP, web-based technology, and audio synthesis/sound design.

Ultimately I'm working to make a push on a production-ready vst bridge for audio from chrome, perhaps to integrate with your spec as a solution to pipe audio to a native DAW via the VST bridge with Web Audio Plugins as an audio source.

I'd like to do so without wrapping anything in an embedded browser...realistically I think the overhead is to high to run at scale (20+ instances inside a DAW).

@jariseon I'm going to look deep into the chrome audio proc & get back to you in this thread about the findings. This seems like a much more elegant solution (assuming the audio stream is accessible through the thread) than the one below.

The other architecture concept in mind involves streaming the audio via uWebSocket (c++) compiled to a WASM (WebAssembly) module, which could work decently (although even the best Websocket implementation is half as fast as Unix domain sockets).

If anyone has additional ideas on potential scalable architectures I'd love to hear, thank you again for the feedback.