Ability to generate .dll (VST2) or .vst3 for use within a DAW?

[GavinRay97]

This project is really impressive!

I am curious whether it's possible to compile the Python project to a dynamic library to load as an audio plugin into a DAW?

Thank you =)

[kunitoki]

It should be definitely possible, with some caveats. We should be able to compile juce as plugin library, and it all boils down to how each plugin format might need to receive the plugin implementation (there are also configuration parts of juce for plugins that are build/compile time, so difficult if not impossible to script at runtime, so each popsicle plugin dll might end up being the same conceptual plugin from the point of view of hosts, unless you recompile it with different name and uid)

Need investigate how we could achieve this.

[GavinRay97]

Thanks for responding =D

I've been trying to port JUCE to other languages in my spare time. (Automation header translation from clang-based tools).

(I actually saw your post on the JUCE forums in the thread about building JUCE into a shared library where JUCE devs said "this isn't a good idea" and you were like "some of us have usecases for this") 🤣

It's a huge pain because of JUCE's build process and number of compile flags, you're right. Not sure if it would work for JUCE, but I did manage to use Python to write a REAPER native extension as a .dll using

https://cffi.readthedocs.io/en/latest/embedding.html

Basically just exposing Python methods to C/C++ as exported symbols, and letting CFFI bootstrap the python Runtime.

/* plugin.h */
typedef struct
{
    int caller_version;
    void* hwnd_main;
    int (*Register)(const char* name, void* infostruct);
    void* (*GetFunc)(const char* name);
} reaper_plugin_info_t;

#ifndef CFFI_DLLEXPORT
#    if defined(_MSC_VER)
#        define CFFI_DLLEXPORT extern __declspec(dllimport)
#    else
#        define CFFI_DLLEXPORT extern
#    endif
#endif

CFFI_DLLEXPORT int ReaperPluginEntry(void* hInstance, reaper_plugin_info_t* rec);

# plugin_build.py
import cffi

ffibuilder = cffi.FFI()

with open('plugin.h') as f:
    # read plugin.h and pass it to embedding_api(), manually
    # removing the '#' directives and the CFFI_DLLEXPORT
    data = ''.join([line for line in f if not line.startswith('#')])
    data = data.replace('CFFI_DLLEXPORT', '')
    ffibuilder.embedding_api(data)

ffibuilder.set_source("my_plugin", r'''
    #include "plugin.h"
''')

with open("plugin_source.py") as f:
    ffibuilder.embedding_init_code(f.read())

ffibuilder.compile(target="reaper_plugin_python_example.*", verbose=True)
# or: ffibuilder.emit_c_code("my_plugin.c")

# my_plugin.py
# Fake export for IDE
def ffi():
  return dict()

# plugin_source.py
# This is a fake import, just to make the IDE happy
# (so we can use @ffi.def_extern() annotation without errors)
from my_plugin import ffi

@ffi.def_extern()
def ReaperPluginEntry(h_instance, rec):
    if not rec or not rec.GetFunc: return 0
    show_console_msg_ptr = rec.GetFunc(b"ShowConsoleMsg")
    show_console_msg = ffi.cast("void (*)(const char*)", show_console_msg_ptr)
    show_console_msg(b"Hello from Python DLL!")
    return 1

In the process of trying to convert JUCE headers to other language bindings (Rust/D/Nim, etc), I've had to figure out a good set of preprocessor flags.

This seems to work okay-ish for an audio-plugin (on Windows + VST3 SDK):

DEFINES =   -D_WIN32=1 -D_WIN64=1 -D_MSC_VER=2000 \
            -DJUCE_MSVC=1 -DJUCE_WINDOWS=1 -DJUCE_STRING=1_UTF_TYPE=16 \
            -DJUCE_PLUGINHOST_VST3=1 -DJUCE_GLOBAL_MODULE_SETTINGS_INCLUDED=1

# Not sure if these are needed
UNDEFINES = -UJUCE_MAC -UJUCE_IOS -UJUCE_LINUX -UJUCE_WEB -UJUCE_BSD -UDOXYGEN

But I can see from your git that you have been using JUCE longer than most anyone, so I'd trust your judgement better.

I dunno much about JUCE or C++ to be honest.

[kunitoki]

Yeah, it really depends on the level of portability you want to achieve. If you want to just rapidly prototype locally, i think it can be done much easier than packaging a dll plugin with python, cppyy and cling in a self contained and deployable dynamic library to any other machine.

Thanks for the pointer on cffi embedding, wasn't aware of that. The problem here is we will need to modify juce as it is already providing the implementation of the plugins entry points. One of the purpose of popsicle was to not require juce forks (hate maintaining just another one).

[GavinRay97]

The problem here is we will need to modify juce as it is already providing the implementation of the plugins entry points.

Ah yeah this is not a viable option at all.

I just wasn't sure how it would be possible to embed/pack the Python code properly so that it could be loaded by the DAW, as a shared lib.

Yeah, it really depends on the level of portability you want to achieve. If you want to just rapidly prototype locally, i think it can be done much easier than packaging a dll plugin with python, cppyy and cling in a self contained and deployable dynamic library to any other machine.

I was thinking of the cppyy + cling route actually.

(I built LLVM + Clang 13 for MSVC because I'm trying to get cppyy bindings to LibTooling and Clang's ASTMatchers. Why has nobody done this yet? Want to build translation/header bindgen tools with LibTooling in Python)

Is it possible with cpppy to create portable distributions/libs (at least, per OS and set of -D flags)? Was reading on their readthedocs yesterday, I think they may have a CMake function which can generate bindings that have some degree of portability:

https://cppyy.readthedocs.io/en/latest/cmake_interface.html

cppyy_add_bindings(
    pkg
    pkg_version
    author
    author_email
    [URL url]
    [LICENSE license]
    [LANGUAGE_STANDARD std]
    [LINKDEFS linkdef...]
    [IMPORTS pcm...]
    [GENERATE_OPTIONS option...]
    [COMPILE_OPTIONS option...]
    [INCLUDE_DIRS dir...]
    [LINK_LIBRARIES library...]
    [H_DIRS H_DIRSectory]
    H_FILES h_file...)

But I dunno. I did build popsicle from source, to produce the JUCE application .dll, maybe it's possible to have a secondary CMakeLists build which uses the VST3 flags and cppyy_add_bindings()?

[kunitoki]

I just wasn't sure how it would be possible to embed/pack the Python code properly so that it could be loaded by the DAW, as a shared lib.

Making a hosted plugin library is different than building a binding library for python, they work in the opposite direction. When building bindings for python (in our case with cppyy), we will access the library from a running python interpreter, and cppyy will generate the python glue code dynamically as we access the library. When building a plugin, first the host will opening a library which should contain the bootstrap code that will return an object following an interface, and if we need to access the library via bindings, we need an interpreter first, so we will need to bundle it with the dll itself as we wouldn't be able to start cppyy in the first place.

Is it possible with cpppy to create portable distributions/libs (at least, per OS and set of -D flags)? Was reading on their readthedocs yesterday, I think they may have a CMake function which can generate bindings that have some degree of portability:

https://cppyy.readthedocs.io/en/latest/cmake_interface.html

This is just automating a bunch of steps that popsicle is doing (and finetuning) manually to be able to build a dynamic library that contains the bindings only. The generated library will still need a python interpreter (and the interpreter still need to have the cppyy dependency installed) to be able to function correctly, you can't load the dylib from a host, and expect it will just work, as it will need the interpreter (which the host is not providing).

A solution would be to create a self contained dll with:

• a python interpreter
• cppyy wheels
• the new entry point for the plugin tech which will:

  start the interpreter install the cppyy wheels in a newly created python home start the python script containing the overloaded plugin class, which will instantiate a python inherited instance from the c++ base class and return it

We might have other issues here with threading. As the host could potentially call different parts of the plugin from the message thread and audio thread, and we execute all python code from the interpreter, we might need to carefully unlock the GIL or the python code executed from the audio thread could block for very long time (or creating deadlocks on the ui for example) or we could create priority inversion issues for the audio thread. It's a very delicate area, and likely to be prone to subtle breaks.

But I dunno. I did build popsicle from source, to produce the JUCE application .dll, maybe it's possible to have a secondary CMakeLists build which uses the VST3 flags and cppyy_add_bindings()?

it wouldn't help much. we need the glue code for the interpreter as well, isn't enough to build bindings. Bindings alone they don't work if you don't load them via python.

[kunitoki]

Not likely to be investigated until we settle the whole project.