search

spotify / pedalboard

🎛 🔊 A Python library for audio.
https://spotify.github.io/pedalboard
GNU General Public License v3.0
5.23k stars 262 forks source link

Midi, instrument support for vst3 plugins, including in AudioStream. #220

Closed CameronChurchwell closed 1 year ago

CameronChurchwell commented 1 year ago

I wanted to use Pedalboard for a school project involving VST3 instrument plugins, so I have added that functionality in as sensible a manner as I could think of.

external plugins store an internal juce::MidiMessageSequence which can be manipulated via instance methods (e.g. midi_note_on). This message sequence then gets split into blocks according to how many samples are passed in, meaning that it is viable both in real-time and otherwise.

I also made adjustments to ExternalPlugin methods to allow VST instrument plugins which have no input bus.

In working with this I discovered what may be a bug in AudioStream, in that certain plugin methods need to be called from the MessageThread, however the current Pedalboard implementation calls plugin->prepare from the ChangeObserver thread, while the main thread is the MessageThread. I have fixed this by setting the new thread to be the MessageThread and everything works perfectly.

I cannot figure out how to create the .pyi typing files as that functionality seems to be broken right now (running the mypy test command on a fresh clone yields an error relating to overload decorators). Additionally, the post processing file for the stubs does not properly write to the supplied output directory (in fact, much of this file seems suspect to me). I tried adding these manually but it did not work so I have commented those stubs out.

I have not thought of a good way to test VST3 instrument processing without including a VST3 instrument plugin in the repository, which seems like it could become a license issue if not resolved carefully.

I also would like to add more Midi helper methods to ExternalPlugin including a way to pass raw Midi data, but I would like these changes reviewed and ideally accepted before I put that time in, as what I have already more than fulfills my personal needs.

I've never used pybind before, nor do I have a ton of C++ experience, so I may have made mistakes along the way, but all tests (save those run by mypy) are passing right now on my local install of this branch. I'll include the file I've been using to test instrument support for clarity on the interface since the type hinting and function signatures are not working without the bindings. bbc.py.zip

psobot commented 1 year ago

Hey @CameronChurchwell,

Thanks so much for getting started on this! I appreciate the effort - this has spurred me to put together a draft pull request that adds full MIDI instrument support from end-to-end. Feel free to take a peek at that PR and let me know if that addresses the same needs you had when opening this PR.

I'll be making a separate PR to address the bugs you've discovered as part of this PR (re: methods being called from the wrong threads in AudioStream).

psobot commented 1 year ago

Hey @CameronChurchwell,

Just FYI that I've merged my MIDI support PR; check out the documentation of pedalboard.VST3Plugin.process for details on how to use the new functionality. I'll close out this PR for now and create a new issue to track the AudioStream threading issue. 👍🏻

CameronChurchwell commented 1 year ago

@psobot Very cool! Glad to see this feature being implemented