nicroto
commented
8 years ago I am aware of, pretty much, all of these options. wdl-ol is what I'm using, primarily because of its licensing.
I tried to approach this from the most naive perspective - I've modified WebKit to enable rendering audio to the DAW(https://github.com/nicroto/webkit/commit/309358b78007205fb1f41cd464be3d08b7a9b279), but after a month of linking (and what not) issues, I've "frozen" my work. Right now, I am at the stage where I can link to the custom built WebKit in wdl-ol project (modifying the IGraphicsMac implementation to create a window with my custom built WebView), but I had series of issues with passing arguments between C/C++/Objective-C/Objective-C++ and that's where I, kind of, gave up... At least for now.
But even if I get this working, at some point, there are issues that arise with the threading model of the browser I'll be embedding. The page would run in a separate thread from the one the DAW creates for my plugin. And the Web Audio graph would be running on yet another one. There is an issue particularly for sample accurate rendering, which seems impossible in this implementation. I think the best way to go about it, is to have a JavaScript context/vm (using the JavaScriptCore, for example), running in the same thread (the plugin thread), with bindings for WebAudio API initialized manually initialized by you, have a WebAudio library (which currently don't exist as a separate project) to be called from these bindings and have a regular WebView for the UI and a JSON communication bridge between the 2 JS contexts (UI and audio). This way sample accurate rendering would be achievable (audio JS code as well as the actual Web Audio graph it manipulates, runs in the same thread), there will be no need to translate the code to C++ and the view can be implemented in web, too.
I believe Web Audio is an awesome engine to be used entirely from C++, too, since it provides the graph API, the awesome scheduling of parameter changes and the many different node types. I haven't extracted it from WebKit or Chromium, though. And since I took a look at both, when I considered which should I use in my project, I would say that WebKit's implementation might be a bit behind, but should be easier to extract from the project. In the Chromium project I got the feel it uses too many shared dependencies, which you will need to extract, too. Chromium is a lot bigger project, too. Blink (WebKit's alternative) is mashed up in the project's source and it isn't a separate lib you can just take out. And if you develop under OS X, Chromium's project for Xcode simply doesn't work - navigation is virtually non-existent. What's worse, sometimes Xcode would find a header (for example) with the same name, but it's not the one that's linked to during compilation. You will have to work entirely from the terminal, including debugging. And, finally, a month ago, Chromium's build didn't work with latest Xcode - I had to downgrade to latest 6-x-x.
I'm interested in your concerns about putting a WebView for the UI. Why isn't this a good idea, in your opinion?
I read your post elsewhere regarding the feasibility of running CEF/Electron in VSTi in order to attain a VSTi compatible build using your web implementation. I have come across projects that have utilized CEF in VSTi (I am pretty sure, to some extent). Another option is a project that guts the WebAudio engine from Webkit for re-use elsewhere. This might be the most minimal method of essentially making the WebAudio Graph API available to you, but would of course leave you with re-implementing the UI and logic/DSP code in C++ (translation to C++ from JS is not out of the question though).
In the long run though, especially if you want to commercialize the VSTi version, it would be worth re implementing your graphs design as a pure C++ plugin perhaps using a library for DSP/Audio processing which provides a graph-like API to make things easier. Once you got the hang of it I think things would end up going smoother and less time consuming then dealing with the maintenance of getting/keeping CEF/WebAudio/JS running smoothly in a VSTi scenario (sounds a bit of a nightmare).
As a foundational building block for audio plugins that run efficiently, offer the simplest API, and enable cross platform on many different types of plugin hosts I highly recommend https://github.com/olilarkin/wdl-ol (the current/maintained WDL which superceeds the original). Oli also offers wdl-ol packaged with Maximilian DSP: https://github.com/olilarkin/IPlugMaxi
This is my DSP building block of choice: https://github.com/micknoise/Maximilian which has a JS/Processing/JAVA counterpart: https://github.com/micknoise/Maxim which in theory you could use to rewrite the web version of your synth to use Maxim which I think should enable easier translation from Maxim into a C++ equivilent ala Maximillian. Just some food for thoughts, not sure how they taste.
I have similar goals (sorta) where I want to be able to write efficient DSP code for the web, while also fully automatically generating a C++ version of the code-base along with the scaffolding to allow compilation into PNaCl/NaCl for Chromium/Blink browsers, in a manner that would act something like progressive enhancement, replacing the JS implementation with the Native Client version on supported clients.
Other than that I also desire to automate the generation of yet another build for audio plugin hosts via wdl-ol. The code-generation aspect I think is fairly smooth sailing, the part I worry about is how to translate the UI/logic code into an identical audio plugin version without having to embed a browser in the plugin host. I have theories on how I might do this by creating a bit of an audio UI framework declaratively implemented in SVG, perhaps even translatable into GLSL code, so that the design should be feasibly possible to implement on a non-browser based host only leaning on a minimal vector graphics API like NanoVG and/or OpenGL/GLSL.
Other avenues I am considering is using the Faust language/engine as a intermediate DSL for the DSP code, since it is already equipped to target many audio engines, plugin hosts, as well as WebAudio/JS and is far more mature than my crazy ideas about code-generation. Of course Faust does nothing to help with the looming UI problem.
Any thoughts, ideas, or insults you may have on these subjects would be great fun.