Plans on VST3 support

[lentferj]

Now with Ardour/Mixbus finally supporting VST3 and Bitwig doing it already for a longer time, any plans on supporting Windows VST3 with yabridge? Some vendors already stopped releasing VST2, e.g. Melodyne.

[robbert-vdh]

I've been holding off on supporting VST3 because it's just so ridiculously complicated compared to VST2.4 and more or less all plugins are still available as VST2. Aside from Melodyne 5 I've only run into a single other plugin that was only available as VST3. But since Steinberg has stopped allowing new developers to support VST2 in 2018, there will probably be more to follow. Aside from the name, VST2 and VST3 sadly have nothing in common so it's probably going to a multiple month effort to support it (that's also why I've been avoiding it), but I actually did start making some small steps towards it last weekend in the feature/vst3 branch.

So far I've mostly been refactoring things to make parts of yabridge a bit more generic towards different plugin formats and to factor out common functionality that would be needed for both VST2 and VST3 support. I've also spent all Sunday getting the VST3 SDK to even build under winegcc, but I did manage to make it work in the end. And since no real Wine VST3 bridges exist yet this might also involve a bit of trial and error to figure out the best way to approach some things. So at the moment I cannot yet put an ETA on when VST3 in yabridge will be in a usable state and I'm pretty busy at the moment, but I'll keep this issue open as a tracking issue for VST3 support!

[robbert-vdh]

As a small status update, I'm starting to get fairly close to implementing all mandatory VST 3.0 interfaces. It's still very far from being usable, and there are still a lot of optional and newer interfaces left to implement, but it's something! After implementing all of that I'll still have to add support for this to yabridgectl and update all documentation, but there's some progress at least (and believe me, you don't want to set up VST3 plugins by hand without yabridgectl). Right now every audio/parameters/events processing related feature works, and I just got to the point where you can actually open the editor (but most of that interface including closing and resizing editors is still filled with stubs right now):

[robbert-vdh]

I just finished implementing all VST 3.0.0 features and I've merged feature/vst3 into master. There's still lots of things to implement before we have full VST 3.7.1 support though (although not as much as you might think because they've skipped most versions in between those two). And of course there will be lots of testing to be done when we get to that point. But, it's something!

Sadly VST3 support does appear to vary greatly between hosts. Bitwig, REAPER, and Carla (so any JUCE based host) seem to behave perfectly, while some implementation issues in Ardour and Qtractor are causing certain plugins to misbehave or segfault.

[CarlColglazier]

Super cool. I just tested this on my system and was able to run a few VST3 plugins pretty well (Valhalla Shimmer, VVV, SWAM Brass). There did seem to be a noticeable lag on the UI compared to the VST2 implementation. Are you experiencing that as well?

[robbert-vdh]

I have not, but from those plugins I've only tested some Valhalla plugins (I tried SuperMassive and VintageVerb, but they should all be the same). With VST3 plugins are responsible for refreshing on their own, so it could be that their UIs refresh at a lower rate in the VST3 versions. How did you install this version of yabridge (e.g. using the CI builds, the yabridge{,ctl}-git AUR packages, or by manually building from source)? If you built from source, then make sure you passed --buildtype release to meson setup/meson configure.

[CarlColglazier]

Built yabridge using AUR yabridge-git. Here's a video of the issue:

https://user-images.githubusercontent.com/9517688/103324832-f8de8280-4a16-11eb-8443-558ddac3bdbf.mp4

Kind of weird in that it's not 100% consistent. Seems worst when a new element is selected. Only effect seems to be on the UI interface. Controlling parameters within the BitWig interface has no such freezing effect.

[robbert-vdh]

Weird. Yabridge uses the exact same window handling code for VST2 and VST3 plugins, it just passes the Win32 windows ID to a different function. Are there suspicious Wine warnings or errors in ~/.BitwigStudio/log/engine.log (or $YABRIDGE_DEBUG_FILE if you have that set) when this happens? And what version of Wine do you use? For the record, this is what it looks like with ValhallaSuperMassive and wine-tkg 6.0-rc4 on my system.

[robbert-vdh]

Another small update on the VST3 support status. We now support all VST 3.5.0 features (the full list of interfaces from the VST3 SDK can be found here. I haven't gotten around to testing some of the features just yet because a lot of them are very rarely used by plugins and hosts, but in theory it should all work fine. I've also done a lot of small responsiveness improvements that should affect both VST2 and VST3 plugins.

[cameronleger]

Hello Robbert! This is some amazing progress, and I'm pretty excited to use this. I'd even play with it soon with the VST3s I have currently installed, but I had an issue.

I was previously using yabridge-git-2.2.1.r2.gb7ae661-1 for quite some time without issues. Last night I did some updates and was then using yabridge-git-2.2.1.r564.g3ac64ec-1, but it seemed to be completely broken. I had swapped all my SOs to the vst2 version, but every single plugin would immediately crash the Bitwig Audio Engine. I didn't see much in the Wine STDERR, other than the last line being an unexpected closure.

I also like a stable setup, so I went back to the other version, but I'm happy to help debug this (if we should bother) too!

[robbert-vdh]

@cameronleger I know you don't use yabridgectl, but have you triple checked to make sure you updated your copies correctly? It's been working fine for me and for everyone else on the yabridge Discord. The plugin scanning errors (or the logs in ~/.BitwigStudio/log/engine.log/$YABRIDGE_DEBUG_FILE once Bitwig has finished scanning) will probably contain at least some useful output.

[cameronleger]

I have, and here's some info from me just swapping back over to test it. Since I started a new/empty project to add the plugins, it now actually just doesn't add the device and shows an error notification. If the plugin was saved with a project, opening that project is what crashes the audio engine.

[I] ➜ pacman -Ql yabridge-git 
yabridge-git /usr/
yabridge-git /usr/bin/
yabridge-git /usr/bin/yabridge-group-32.exe
yabridge-git /usr/bin/yabridge-group-32.exe.so
yabridge-git /usr/bin/yabridge-group.exe
yabridge-git /usr/bin/yabridge-group.exe.so
yabridge-git /usr/bin/yabridge-host-32.exe
yabridge-git /usr/bin/yabridge-host-32.exe.so
yabridge-git /usr/bin/yabridge-host.exe
yabridge-git /usr/bin/yabridge-host.exe.so
yabridge-git /usr/lib/
yabridge-git /usr/lib/libyabridge-vst2.so
yabridge-git /usr/lib/libyabridge-vst3.so

[I] ➜ ls -lah ~/.local/share/yabridge/bx_*
Permissions Size User    Date Modified Name
lrwxrwxrwx   124 cameron 29 Oct  2020  /home/cameron/.local/share/yabridge/bx_console SSL 4000 E.so -> /mnt/games/PlayOnLinux/wineprefix/VSTsBX/drive_c/Program Files/Steinberg/VstPlugins/Plugin Alliance/bx_console SSL 4000 E.so

[I] ➜ ls -lah "/mnt/games/PlayOnLinux/wineprefix/VSTsBX/drive_c/Program Files/Steinberg/VstPlugins/Plugin Alliance/bx_console SSL 4000 E.so"
Permissions Size User    Date Modified Name
.rwxr-xr-x  952k cameron  8 Jan 11:35  /mnt/games/PlayOnLinux/wineprefix/VSTsBX/drive_c/Program Files/Steinberg/VstPlugins/Plugin Alliance/bx_console SSL 4000 E.so

[I] ➜ ls -lah /usr/lib/libyabridge-vst2.so
Permissions Size User Date Modified Name
.rwxr-xr-x  952k root  7 Jan 20:24  /usr/lib/libyabridge-vst2.so
~ 

I basically get the same logging information in Bitwig, from the terminal running it, and the YABRIDGE_DEBUG_FILE, and essentially every plugin has the same output (setup, Wine starting, exited), so I'm only including this one. Swapping back to the older package, and swapping all the files to the /usr/lib/libyabridge.so file, and everything immediately works again. Of course, Bitwig/Wine are completely closed between the tests.

[I] ➜ grep OTT yb.log 
11:41:00 [OTT_x64-5ALRcmou] Initializing yabridge version 2.2.1-564-g3ac64ec
11:41:00 [OTT_x64-5ALRcmou] host:         '/usr/bin/yabridge-host.exe'
11:41:00 [OTT_x64-5ALRcmou] plugin:       '/mnt/games/PlayOnLinux/wineprefix/Serum/drive_c/Program Files/VST/OTT_x64.dll'
11:41:00 [OTT_x64-5ALRcmou] plugin type:  'VST2'
11:41:00 [OTT_x64-5ALRcmou] realtime:     'yes'
11:41:00 [OTT_x64-5ALRcmou] sockets:      '/run/user/1000/yabridge-OTT_x64-5ALRcmou'
11:41:00 [OTT_x64-5ALRcmou] wine prefix:  '/mnt/games/PlayOnLinux/wineprefix/Serum'
11:41:00 [OTT_x64-5ALRcmou] wine version: '5.6 (Staging)'
11:41:00 [OTT_x64-5ALRcmou] 
11:41:00 [OTT_x64-5ALRcmou] config from:   '<defaults>'
11:41:00 [OTT_x64-5ALRcmou] hosting mode:  'individually, 64-bit'
11:41:00 [OTT_x64-5ALRcmou] other options: '<none>'
11:41:00 [OTT_x64-5ALRcmou] 
11:41:00 [OTT_x64-5ALRcmou] Enabled features:
11:41:00 [OTT_x64-5ALRcmou] - bitbridge support
11:41:00 [OTT_x64-5ALRcmou] - VST3 support
11:41:00 [OTT_x64-5ALRcmou] 
11:41:00 [OTT_x64-5ALRcmou] [Wine STDERR] 000b:fixme:winediag:__wine_start_process Wine Staging 5.6 is a testing version containing experimental patches.
11:41:00 [OTT_x64-5ALRcmou] [Wine STDERR] 000b:fixme:winediag:__wine_start_process Please mention your exact version when filing bug reports on winehq.org.
11:41:01 [OTT_x64-5ALRcmou] [Wine STDERR] 0026:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
11:41:01 [OTT_x64-5ALRcmou] [Wine STDERR] 0026:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
11:41:01 [OTT_x64-5ALRcmou] [Wine STDERR] 0026:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
11:41:01 [OTT_x64-5ALRcmou] [Wine STDERR] 0026:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
11:41:01 [OTT_x64-5ALRcmou] [Wine STDERR] 0026:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
11:41:02 [OTT_x64-5ALRcmou] [Wine STDERR] 000b:fixme:msg:pack_message msg 14 (WM_ERASEBKGND) not supported yet
11:41:02 [OTT_x64-5ALRcmou] [Wine STDERR] 000b:fixme:ver:GetCurrentPackageId (0x32faa0 (nil)): stub
11:41:02 [OTT_x64-5ALRcmou] The Wine host process has exited unexpectedly. Check the output above for more information.

[robbert-vdh]

I thought maybe Wine 5.6 was the issue (I had to change some things to get yabridge to build with Wine 6.0 because of some internal changes to Wine), but the combination of yabridge's master branch (2.2.1-581-g07a9940), Wine 5.6, OTT and Bitwig Studio 3.3.1 is working without issues here, and I've also tested on older Wine versions in the last few weeks without any issues. Could you try running the following:

yabridge-host.exe
yabridge-host.exe a b c

The first command without any arguments should print:

Usage: yabridge-host.exe <plugin_type> <plugin_location> <endpoint_base_directory>

And with three arguments it should print (with a different version):

Initializing yabridge host version 2.2.1-581-g07a9940
Preparing to load <unknown> plugin at 'b'
Unknown plugin type 'a'

[cameronleger]

Well...

[I] ➜ /usr/bin/yabridge-host.exe
000b:fixme:winediag:__wine_start_process Wine Staging 5.6 is a testing version containing experimental patches.
000b:fixme:winediag:__wine_start_process Please mention your exact version when filing bug reports on winehq.org.
001d:err:module:import_dll Library WDFLDR.SYS (which is needed by L"C:\\windows\\system32\\drivers\\ssdevfactory.sys") not found
001d:err:ntoskrnl:ZwLoadDriver failed to create driver L"\\Registry\\Machine\\System\\CurrentControlSet\\Services\\ssdevfactory": c0000142
000f:fixme:service:scmdatabase_autostart_services Auto-start service L"ssdevfactory" failed to start: 1114
0023:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
0023:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
0023:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
0023:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002d:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
002d:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
000f:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
000f:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
zsh: segmentation fault (core dumped)  /usr/bin/yabridge-host.exe

vs the previously working:

[I] ➜ /usr/bin/yabridge-host.exe                                                    
000b:fixme:winediag:__wine_start_process Wine Staging 5.6 is a testing version containing experimental patches.
000b:fixme:winediag:__wine_start_process Please mention your exact version when filing bug reports on winehq.org.
001d:err:module:import_dll Library WDFLDR.SYS (which is needed by L"C:\\windows\\system32\\drivers\\ssdevfactory.sys") not found
001d:err:ntoskrnl:ZwLoadDriver failed to create driver L"\\Registry\\Machine\\System\\CurrentControlSet\\Services\\ssdevfactory": c0000142
000f:fixme:service:scmdatabase_autostart_services Auto-start service L"ssdevfactory" failed to start: 1114
0023:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
0023:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
0023:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
0023:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002f:fixme:ntdll:server_ioctl_file Unsupported ioctl 24000 (device=2 access=1 func=0 method=0)
002d:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
002d:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
000f:fixme:ntdll:EtwEventRegister ({1a3c3605-aa85-4b19-aa24-bb74bc365059}, 0x180007fe4, 0x18000f120, 0x18000f720) stub.
000f:fixme:ntdll:EtwEventUnregister (deadbeef) stub.
Usage: yabridge-host.exe <vst_plugin_dll> <endpoint_base_directory>

[robbert-vdh]

Make sure to do a completely clean build (although the yabridge-git AUR package should already remove the build directory for you if it still exists). Wine versions >= 5.7 and < 6.0 require a special setting automatically gets set by the build definition when it detects those Wine versions. It builds and runs correctly on my system with Wine 5.6 when I just tried it. Alternatively just downgrade to Wine Staging 5.4, build, and then go back to your Wine 5.6 again. Binaries built with never versions won't work with Wine 5.6, but ones built with older versions should.

[cameronleger]

Interesting thought! I do use that package, and I just tried again, but it's the same thing. I definitely removed any files it would've been using, but I got the same result. I see in the build where it would check the version and add the flag, but my wine is returning 5.6 and I don't see that flag appearing anywhere in the build output, so I'm not sure that's what's happening.

[robbert-vdh]

With Wine 5.6 it should not add __cdecl. But maybe you're using some Wine 5.6 that's actually almost Wine 5.7? You can try adding __cdecl manually in individiual-host.cpp and group-host.cpp.

[cameronleger]

Well, I'm back on wine-staging-git 5.6.1.r0.g4c160ec8+wine.5.6.r0.g4cdb7ec829-1 and yabridge-git 2.2.1.r2.gb7ae661-1, but I've learned a bit since then.

• An up-to-date wine-staging-git and yabridge-git will work fine
• However, I haven't updated wine in order to limit license reactivations, and since I had a broken wine the last time I tried
• Now I know why it's broken, though. I'm having the issues described here, and my setup is pretty similar: https://bbs.archlinux.org/viewtopic.php?id=261004&p=2
• Basically, something about multi-nvidia/bumblebee, wine, and X are having an issue right now. An up-to-date wine will only work inside optirun/primusrun lol

So, not sure why yabridge isn't ok building against this version, but I'm sticking on this version for the time-being so I'll hold off until the other issue is resolved.

[robbert-vdh]

Even though everything seems to work fine on the CI and for the people using Arch and Manjaro on the yabridge Discord, I'm also getting the same segfault building with an older Wine version in an Ubuntu 20.04 docker image (the same one I use for the CI). I managed to narrow down the cause of the segfault to linking to some specific .cpp files in src/serialization/vst3/, but I still have to do some more investigation to understand what exactly is going on. I'll get back to you later.

[robbert-vdh]

@cameronleger So this was super bizarre. I could reproduce the segfault you were getting, but only when doing a unity build with older Wine versions (as will become apparent later, the fact that some things didn't immediately segfault with non-unity builds is likely a fluke). It also happens regardless of what the actual program is supposed to be doing (so it still happens if you replace the main function with a function that just prints Hello, world! and exits). I ended up bisecting all definitions in the .cpp files and I could trace it back to the use of the VST3 SDK static variable Steinberg::Vst::IEventList::iid. Once that specific variable is used somewhere (these *::iids are used all over the place to identify specific VST3 interfaces) a unity build with older Wine versions will segfault (and after thinking about it, this should happen when including any of these iids). I have absolutely no explanation for why this may be happening, and I would be surprised if this wasn't a winegcc bug. So a unity build that only links to the VST3 SDK and only compiles the following file will segfault:

#include <pluginterfaces/vst/ivstevents.h>

int main(int argc, char* argv[]) {
    Steinberg::FUID whatisevenhappening = Steinberg::Vst::IEventList::iid;
}

After debugging Wine itself a bit I traced the actual segfault down to a call to the Windows function CoCreateGuid which gets called during static initialization after Wine loads the library. Earlier I noted that the segfault didn't happen with non-unity builds, but when running with GDB attached it happens both with and without unity builds, so that was very likely a fluke. I haven't spent enough time to track down an old bug report on WInHQ for CoCreateGuid segfaulting, but after testing every Wine version between 5.0 and 6.0-rc6 this only seems to happen with Wine versions below 5.7 (so 5.6 is the first affected version).

Since this is a Wine bug, you have two options:

1. Disable VST3 support by adding the -Dwith-vst3=false option to the meson setup command line, or
2. Upgrade your Wine installation, since this bug only affects versions 5.6 and lower (although I haven't tested anything below 5.0 for obvious reasons)

I've added a check to the Meson build file that will just throw an error if you're trying to build with VST3 support with Wine 5.6 or below to prevent this from causing more confusion in the future.

[cameronleger]

Wow, I love the debugging effort you went through. I'd love to update for the VST3 support (Reason Rack isn't VST2), but I'm essentially locked on my Wine version due to that bug with some X11 changes in Wine not playing well with what appears to be some Xorg settings related to multi-GPU systems. I'll watch both closely!

[CarlColglazier]

Reason Rack isn't VST2

Just so you know: Reason Rack isn't working yet even with VST3+yabridge. It runs, but it bugs out when trying to save the state. I'm not quite sure what the issue is, but the standalone also seems to have issues accessing the filesystem through WINE.

[robbert-vdh]

I don't own Reason Rack so I can't check it out for myself, but if it's the actual saving that's not working (so if it doesn't return during a call to IComponent::getState()/IEditController::getState() if you run with YABRIDGE_DEBUG_LEVEL=1), then it's almost definitely just a Wine issue.

[robbert-vdh]

So I finally managed to implement all VST 3.7.1 features! That of course doesn't mean that yabridge 3.0.0 is ready yet. There's still lots of testing to be done, and before that I still have a lengthy list of things to either fix or investigate a bit further, but we're getting somewhere!

[robbert-vdh]

Another small but kind of important change worth mentioning, I just reworked a bunch of things to fix some final outstanding VST3 bugs on REAPER. REAPER's very particular about from what thread functions that perform GUI options may be called. Things like plugin-drive resizes and opening context menus might otherwise either not work as expected or they might completely crash REAPER. It took me a day work this out in the neatest way I possibly could and it still holds the crown for the nastiest code in yabridge, but at least now these things work as expected.

[robbert-vdh]

Built yabridge using AUR yabridge-git. Here's a video of the issue: yabridge.mp4

Kind of weird in that it's not 100% consistent. Seems worst when a new element is selected. Only effect seems to be on the UI interface. Controlling parameters within the BitWig interface has no such freezing effect.

@CarlColglazier Turns out Bitwig and yabridge's input focus grabbing had some weird interaction that was causing that focus grabbing to happen recursively. Some other people reported it in the Discord, and I could reproduce it on my laptop as well. It's fixed as of https://github.com/robbert-vdh/yabridge/commit/9788f21e0e1a8dbcbcc713b6fceb6e6eaa2b7d12.

[robbert-vdh]

:warning: The latest master branch commits contain backwards incompatible changes that will cause your old projects containing yabridged VST3 plugins to stop loading, see below for how to fix this :warning:

As was pointed out to me, projects containing VST3 plugins saved under Windows would not load with yabridge under Linux. It turned out that directly copying the unique identifiers reported by VST3 plugins resulted in different identifiers than what you'd get with the native, non-bridged version of same plugin running under either Windows or Linux. I've modified yabridge to change those identifiers around so they do match up with the correct identifiers, but this does mean that after updating your host will no longer recognize VST3 plugins that were running through yabridge in old projects.

I've spend all day writing migration scripts that can rewrite existing Ardour, Bitwig, REAPER and Renoise project files to account for these changes. You can find them along with instructions on how to use them here: https://github.com/robbert-vdh/yabridge/tree/master/tools/migration

Sorry for this major inconvenience, but I'm very glad that we at least learnt of this before an actual release.

[robbert-vdh]

Yabridge 3.0.0 has now been released with full VST 3.7.1 support! If anyone runs into issues with VST3 plugins, then feel free to open a new bug report.