RemoteVstPlugin.exe.so crashes lmms on "incompatible Qt library"

[dvzrv]

Hi, I'm maintaining lmms for Arch Linux and someone reported a crash, that is caused by the Vestige VST plugin host.

In short: lmms 1.2.1-1 was built against qt5-base 5.13.1. Then that person upgraded to 5.13.2 and now lmms crashes on loading a VST plugin. I could reproduce this with TAL noisemaker:

VST sync support disabled in your configuration
VST_PATH not set, defaulting to /home/dave/vst:/usr/local/lib/vst:/usr/lib/vst
"Cannot load library /usr/lib/ladspa/lsp-plugins-ladspa.so: (/usr/lib/ladspa/lsp-plugins-ladspa.so: undefined symbol: _ZN3lsp17builtin_resourcesE)"
"Cannot load library /usr/lib/ladspa/lsp-plugins-ladspa.so: (/usr/lib/ladspa/lsp-plugins-ladspa.so: undefined symbol: _ZN3lsp17builtin_resourcesE)"
"Cannot load library /usr/lib64/ladspa/lsp-plugins-ladspa.so: (/usr/lib64/ladspa/lsp-plugins-ladspa.so: undefined symbol: _ZN3lsp17builtin_resourcesE)"
Starting using X11Embed protocol.
RemotePluginClient::shmget: No such file or directory
RemoteVstPlugin.cpp::shmget: No such file or directory
RemoteVstPlugin.cpp: Failed to initialize shared memory for VST synchronization.
 (VST-host synchronization will be disabled)
unique ID: ncut
RemotePlugin::DebugMessage: inputs: 2  output: 2
RemotePlugin::DebugMessage: creating editor
002b:fixme:ver:GetCurrentPackageId (0xbbfefc (nil)): stub
0009:fixme:win:RegisterTouchWindow (0x10060 00000000): stub
0009:fixme:msg:ChangeWindowMessageFilterEx 0x10060 233 1 (nil)
0009:fixme:msg:ChangeWindowMessageFilterEx 0x10060 4a 1 (nil)
0009:fixme:msg:ChangeWindowMessageFilterEx 0x10060 49 1 (nil)
0009:err:ole:CoGetClassObject class {4ce576fa-83dc-4f88-951c-9d0782b4e376} not registered
0009:err:ole:create_server class {4ce576fa-83dc-4f88-951c-9d0782b4e376} not registered
0009:err:ole:CoGetClassObject no class object {4ce576fa-83dc-4f88-951c-9d0782b4e376} could be created for context 0x6
RemotePlugin::DebugMessage: editor successfully created
Cannot mix incompatible Qt library (version 0x50d01) with this library (version 0x50d02)
[1]    1242429 abort (core dumped)  lmms

Problematically, afterwards, there will be an instance of /usr/lib/lmms/RemoteVstPlugin.exe.so running in the background.

I have rebuild lmms against qt5-base (and the other components) in version 5.13.2 and the problem disappers.

I'm wondering, whether this can be fixed in a better way, than to rebuild lmms even against patch level releases of Qt5 (note, no symbol changes or anything).

[lukas-w]

@dvzrv This problem is tracked at https://github.com/lukas-w/qt5-x11embed/issues/3, unfortunately we know no way to solve this. RemoteVstPlugin uses qt5-x11embed, a port of QX11EmbedContainer, a feature that was part of Qt4 but was removed in Qt5. This feature depends on Qt-internal private headers making it tied to a specific Qt version, which Qt enforces by aborting with the error message you see.

There could be ways to improve the situation from a packaging perspective though. Maybe qt5-x11embed can be added to Arch as a separate package? That way, we can link to it dynamically and only qt5-x11embed (not lmms) would need to be recompiled when Qt is updated.

[opekope2]

I have Manjaro 20 KDE with QT 5.14.2 and wine 5.9. LMMS 1.2.1 shows QT 5.14.0. This crashes. The AppImage variant shows QT 5.9.7. This works fine for me and does not crash. The Flatpak variant does not have VeSTige and shows QT 5.14.2.

[patryk-tech]

Same problem after upgrading to Qt 5.15.

@lukas-w maybe it would be a good idea to add a qt5-x11embed package to the AUR, and tie it to the Qt version (e.g. qt5-x11embed-5.15.0). Then it would be possible to update the version in the PKGBUILD file whenever Qt gets updated, and users could install the latest version.

For anyone who came here from googling the error message, a temporary workaround I am using is under Edit/Settings/General/Plugin Embedding, changing the option from XEmbed to Embed using Qt API. The VST GUIs break (but that may just be my machine), but they are at least usable and LMMS doesn't crash.

[lukas-w]

@patryk-tech Sure, I'd be happy to submit the package to the AUR, but I don't think we can have an AUR qt5-x11embed dependency in the non-AUR package lmms. So I think it may make more sense to add it as a package to the community repo. Maybe as an alternative we can modify LMMS so that we can have qt5-x11embed be an optional dependency. Waiting on @dvzrv's input on this.

[dvzrv]

@lukas-w sorry for the long delay. ETOOMANYPACKAGES ;-)

Hm, tough cookie. An alternative could be indeed to link dynamically against a qt5-x11embed as it would probably be faster to rebuild than lmms. Given that this library would only be in use for lmms defeats the purpose a little though.

AFAICS the cleaner and more forward compatible solution would probably be to get rid of this dependency altogether (if possible). Is there another way with given qt5 (or soon qt6) that can achieve this natively? Staying in sync with qt upstream and additionally providing a separate library plus releases could prove to be very fragile (and time consuming).

[dvzrv]

@lukas-w After consulting the qt5 maintainer for Arch Linux on this matter I think the "separate package to link against" option is not really worth it. Qt5 is feature frozen and qt6 will be coming in November.

I think it would make more sense to find ways of replacing qt5-x11embed with something native, which will ease the maintenance burden on both sides and increase the long term stability of lmms.

FYI: Currently lmms is on a rebuild list for qt5 related packages so a rebuild is triggered for it if the qt5 libs are being (re)built.

[lukas-w]

I think it would make more sense to find ways of replacing qt5-x11embed with something native

@dvzrv I think we're out of luck here, qt5-x11embed was created precisely because of the removal of this feature in Qt5. It's a port of the QtX11EmbedContainer class from Qt4 which I wrote because no replacement was available.

[0x0015]

@dvzrv Another temporary fix for me that allows me to stay on the one from the package manager was in the lmms settings under plugin embedding select "Embed using Qt API" instead of "Embed using XEmbed protocol". Note that this will mess up the mouse input for some plugins (eg. IL Autogun). it also spits a whole bunch of messages in the console, but you're most likely not worried about that.