Make a plugin/addon API

[MainCharacteroftheProblem]

This is a bit of a pipe dream and would require a lot of effort on our part (effort that would be better spent elsewhere). However, I think that if anyone could add this to Beepbox, it would be us.

[AstroScraps]

what is meant by plugin/addon api? like what would fall under a plugin? first thought that comes to mind is a vst, but iirc u guys said that's not really gonna be happening any time soon

[leovoel]

Needs a lot of design work!

There's a bunch of stuff that could be called a plugin. The things I'm aware of that could be dealt with are:

• Audio or song data processors and generators: This is what people are usually thinking of when they say "VST"s or "VSTi"s. Their inputs and outputs are usually audio data, though some can operate instead on MIDI, but in our case we'd maybe not bother with that since it'd require a conversion step. It'd allow people to implement new instrument types and effects. The main API for this would be some way to operate on blocks of audio coming out of the synthesizer. For song data/MIDI I don't actually know what is commonly done, needs research.
• Scripts: I don't know if that's always the name used for this (some would call these "macros" I guess?), but there is usually also a way to automate actions on the editor, and this would be how. The APIs here would generally mirror buttons and other actions available on the editor.

Performance is an important constraint here, and is the main complicating factor here I feel. Not only the plugin can be its own perf can-of-worms but we also have extra data copies and such to worry about on our end. Probably we'd also want to figure out the threading story first before doing anything here. Performance issues also interact badly with my next point...

Security is another important constraint, perhaps more so in our case. Ideally we'd be able to "sandbox" plugins well, but it's difficult to do that while keeping things very fast, which we need for audio.

As for the implementation language we'd open up for plugin authors, it could be JavaScript as the rest of the editor uses, but maybe WebAssembly would be the better option here. We can't load shared libraries (.dll or .so or whatever) in browsers, only in offline versions where we control the executable, so "VST" support can only happen there.

The first actual step here would probably be to look at plugin formats out there and see what we could borrow good ideas from.