Implement audio/realtime thread for vst-bridge host.

kramlie commented 5 years ago

The problem with realtime processing in vst-bridge is that although the host calls process on the plugin from a realtime thread, the thread which is in the VST host process is not realtime, and hence the audio thread is stuck waiting for a non-realtime thread. Solving this comes with some challenges, because we only have one pipe to communicate through, and this is difficult to synchronize between multiple threads.

The solution in this patch splits the whole host communication part in two, by separating out the relevant components into their own struct, and then providing two instances of the communication pipe instead of one: One for the realtime thread, and one for the normal thread. Then we use the process calls as a guide to figure out which thread is the realtime thread on the plugin side, and any call made using this thread is forwarded to the VST host process through the realtime pipe instead of the normal pipe.

We also need to actually set up a realtime thread on the host side, and this is done by providing a new opcode for setting the thread scheduler and priority. We just use the same values as the calling thread uses.

It may be difficult to achieve true realtime performance in wine, for a variety of reasons, but this patch goes a surprisingly long way, and I've been able to have hour long sessions with no xruns at all, whereas before they would occur several times per minute.

A couple of things to look out for if performance is still poor after applying this patch:

Hosts that create their own threads. This plugin is not able to apply the realtime scheduler on threads it doesn't own, which means the host will still be stuck waiting for non-realtime threads. At the time of writing, Wine does not honor Windows thread priorities (I've checked the source code), so even if the VST plugin tries to do the right thing and increase the thread priority, it will fail to do so. I may try to submit a patch to the Wine project at some point, if I get the time. However, in many hosts, thread creation is configurable under "Number of cores" or some similar setting, and turning it off may give better performance. At least it did for me.
/tmp/.wine-<UID> should be a on tmpfs. While wine is communicating with wineserver, it uses some temporary files here, and these should be on tmpfs, IOW a RAM disk, to make sure the thread doesn't stall on I/O.