Open kmturley opened 1 year ago
hey, thanks for trying this out 🙂
what CPU architecture are you building for? For macOS, I did find that FL studio's plugin validator gets false-negatives on verifying ARM-native JUCE plugins. I had to built it as a Universal Binary in order to pass validation, because I think it only considers x86 plugins valid.
the other thing that might be needed for a macOS distribution is code-signing? not sure what error you'd get and when though.
but I think the most likely thing… did you build it in one container, copy it elsewhere, validate it in a different container?
because that's a 3.31MB .vst3 file. I bet you're dynamically-linking, since a statically-linked distribution (i.e. all libraries burned into one file) would be 12.1MB as a release build.
if you're dynamically-linking: I bet you've linked to a .dylib which doesn't exist in the container which is vaildating it.
try printing the load commands to see which dylibs it's linked to: otool -L juicysfplugin-mac.vst3
are those libraries actually on the computer that's doing the validation?
if that does turn out to be the problem: here's how to make a portable distribution of juicysfplugin:
https://birchlabs.co.uk/blog/alex/juicysfplugin/synth/cpp/2019/01/05/a-soundfont-vst-for-macos.html
you could try debugging the load-time linker:
DYLD_PRINT_LIBRARIES=1 validator "juicysfplugin-mac.vst3"
this will print every dynamic load command it attempts. I think. if there's failures, it might tell you.
Would this be a feature you would be interested in using? I can submit a PR?
CI would be nice, but I fear that I have a lot of requirements that are hard to meet. I want distribution/installation to be easy on all platforms. I thought static linking was the solution to this, but it's super hard (I failed to finish getting it working for Linux… just too many libraries to compile-from-source). so maybe for Linux the answer is distribution via a package manager. but that sounds like a lot of work too…
the best answer for Linux would probably be a statically-linked image built on muslc
. I think this would work on every Linux distribution. so it'd make sense to do the work in an Alpine Linux image, which gives you access to muslc-based libraries.
for macOS… dynamic or static linking could be fine (I've done both in the past), but it needs to be a Universal Binary in order to pass FL Studio's plugin validation. nobody distributes Universal Binary libraries of fluidsynth and its dependencies, so that adds some complexity in terms of automating it. it probably also means you can't use a solution based on brew, because you wouldn't be able to express a dependency on the fluidsynth they publish.
I don't have a business Apple Developer account, so I believe I cannot produce a signing key for CI to sign macOS artifacts. this means anything built by CI would be hard for users to install.
VST2 cannot be done in CI (I cannot distribute the Steinberg VST2 SDK). I dunno whether anybody still needs VST2 though.
Windows is the biggest problem. if you dynamically-link, you need to put the .dlls somewhere. there's not really good options; you can't put them next to the VST2 dll, because the DAW requires that every dll in the VST2 folder be a VST2 plugin.
for all platforms, I'd want licenses of every dependency included in the distribution.
I got the Windows build working pretty well -- it's a Dockerized, Linux-based cross-compiler which builds a statically-linked Windows binary via LLVM-MinGW toolchain:
docker build . -f win32.Dockerfile --tag=llvm-mingw
docker run -it --rm --name llvm-mingw llvm-mingw
https://github.com/Birch-san/juicysfplugin/blob/master/win32.Dockerfile
I tried to extend the Docker image to do an even more ambitious cross-compile (target 32-bit, 64-bit, ARM, x86, Windows, Linux, do everything with static linking):
https://github.com/Birch-san/juicysfplugin/commits/linux
but I didn't finish getting Linux working. tried for weeks but static linking is just too hard. dynamic linking probably an easier way to do it, but then you need to solve "well how will the user get or install the libraries it depends on".
a big reason why I went for a solution based on Docker — instead of GitHub CI — is that it also solves the problem of "how do I build the artifact on my developer workstation", for any host or target platform.
so, all-in-all it's a worthy cause but it's very hard. I think shortcuts can be taken (i.e. dynamic linking) but then it creates a distribution/installation problem instead. handling issues is very hard when there's no guarantee that the user installed it correctly. so I'm not sure there's a good option here.
What CPU architecture are you building for?
macOS distribution could be missing code-signing?
I thought it was a different error corrupted vs unsigned, maybe i'm wrong here
Did you build it in one container, copy it elsewhere, validate it in a different container?
Yes built in GitHub CI, created artifacts here: https://github.com/studiorack/juicysf/actions/runs/2976408057 Download to my local machine to validate. This could be the issue!
I bet you're dynamically-linking
You can see the commands i'm using above in the workflows yml file. I see both
-dynamiclib
and [ 65%] Linking CXX static library JuicySFPlugin_artefacts/Release/libjuicysfplugin_SharedCode.a
in the CI output. I presume this means I am using dynamic linking...
I looked into it more and found users with the same issue here: https://github.com/bcpierce00/unison/issues/422
It appears the issue is to do with the binary not being signed.
Running the MacOS command to bypass security allows the plugin file to be opened by MacOS locally.
xattr -cr "test/tool/VST3/juicysfplugin-mac-remote.vst3"
otool -L juicysfplugin-mac-remote.vst3/Contents/MacOS/juicysfplugin > remote.txt otool -L juicysfplugin-mac-local.vst3/Contents/MacOS/juicysfplugin > local.txt
Comparing a diff shows only version differences:
DYLD_PRINT_LIBRARIES=1 validator "test/tool/VST3/juicysfplugin-mac-remote.vst3/Contents/MacOS/juicysfplugin" > remote.txt DYLD_PRINT_LIBRARIES=1 validator "test/tool/VST3/juicysfplugin-mac-local.vst3/Contents/MacOS/juicysfplugin" > local.txt
Outputs are the same, except for the plugin path:
However plugin still does not load in Ableton Live, where as your official build does.
Comparing the generated artifacts I see some differences:
And the final binary size as you mentioned which might be due to static generation:
Going to try static linking!
In https://github.com/Birch-san/juicysfplugin/blob/master/building.macos.md it says:
git clone git@github.com:juce-framework/JUCE.git
cd JUCE
git checkout 6.1.4
...
git clone git@github.com:FluidSynth/fluidsynth.git
cd fluidsynth
git checkout v2.2.4
However in https://github.com/Birch-san/juicysfplugin/tree/master/win32_cross_compile it says:
git clone --branch 6.1.5 --depth 1 https://github.com/juce-framework/JUCE.git
git clone --branch v2.2.5 --depth 1 https://github.com/FluidSynth/fluidsynth.git
Is there a difference between those versions?
I get errors when using those versions:
CMake Error at fluidsynth/CMakeLists.txt:158 (include):
include could not find requested file:
DefaultDirs
CMake Error at fluidsynth/CMakeLists.txt:162 (include):
include could not find requested file:
CheckSTDC
CMake Error at fluidsynth/CMakeLists.txt:192 (include):
include could not find requested file:
TestInline
CMake Error at fluidsynth/CMakeLists.txt:193 (include):
include could not find requested file:
TestVLA
CMake Error at fluidsynth/CMakeLists.txt:527 (include):
include could not find requested file:
UnsetPkgConfig
CMake Error at fluidsynth/CMakeLists.txt:573 (unset_pkg_config):
Unknown CMake command "unset_pkg_config".
regarding your otool
output, there's one more thing worth checking…
there's a load command there that says to look for libfluidsynth.3.dylib in /usr/local/opt/…
but does the referenced file actually exist on this machine?
and if you run otool -L
on the dylib, it'll tell you where libfluidsynth's dependencies are. are those also on this machine?
well, it's generally easier to build & run the standalone app as a starting point, rather than going straight to trying to integrate VST into a DAW. once .app
works fine, I'd move onto the plugin targets.
for macOS that JUCE version difference isn't consequential. both work.
both those versions of fluidsynth will work, but the newer one has a bug fix for sample looping:
https://github.com/Birch-san/juicysfplugin/releases/tag/3.1.0
so pick the newer if you have a choice. neither of them are deliberately kept low, they were just the newest available at the time of writing.
Upgrading fluidsynth version v2.2.4 -> v2.2.8 fixed the errors with UnsetPkgConfig
on Mac
Upgrading JUCE v6.1.4 -> v6.1.6, fixes errors with specification cannot appear in a typedef declaration
in Windows:
https://forum.juce.com/t/juceaide-fails-to-build-on-windows-plugin-demo-master-branch/48007/12
I would suggest updating your files to those versions to ensure compatibility cross-platform!
I wasn't using the exact same flags as you use in https://github.com/Birch-san/juicysfplugin/blob/master/building.macos.md so have added those
I was getting a new error with your flags:
/Users/runner/work/juicysf/juicysf/Source/FluidSynthModel.h:8:10: fatal error: 'fluidsynth.h' file not found
Changing -Denable-framework=off
to -Denable-framework=on
fixed that error
Good point regarding getting the standalone version to run. It opens, but then crashes when I try to load a SF2 file
Error message doesn't give much info:
-------------------------------------
Translated Report (Full Report Below)
-------------------------------------
Process: juicysfplugin [95072]
Path: /Users/USER/Sites/*/juicysfplugin.app/Contents/MacOS/juicysfplugin
Identifier: com.Birchlabs.JuicySFPlugin
Version: 3.0.0 (3.0.0)
Code Type: X86-64 (Native)
Parent Process: launchd [1]
User ID: 501
Date/Time: 2022-09-04 18:27:48.8395 -0700
OS Version: macOS 12.5.1 (21G83)
Report Version: 12
Bridge OS Version: 3.0 (14Y910)
Anonymous UUID: B27D1ACD-8DDC-233C-08FC-5D3CA8D6D30B
Sleep/Wake UUID: CD3BA0EB-9343-4C74-B68C-4FCDB38F6ADA
Time Awake Since Boot: 170000 seconds
Time Since Wake: 4194 seconds
System Integrity Protection: enabled
Crashed Thread: 0 JUCE Message Thread Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Codes: 0x0000000000000001, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Reason: Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process: exc handler [95072]
VM Region Info: 0 is not in any region. Bytes before following region: 4354777088
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 10390a000-103b32000 [ 2208K] r-x/r-x SM=COW ...juicysfplugin
Thread 0 Crashed:: JUCE Message Thread Dispatch queue: com.apple.main-thread
0 CoreFoundation 0x7ff81e2ae8dd CFDataGetBytePtr + 6
1 juicysfplugin 0x10391c114 FilePicker::filenameComponentChanged(juce::FilenameComponent*) + 148
2 juicysfplugin 0x103a4e737 juce::FilenameComponent::handleAsyncUpdate() + 87
3 juicysfplugin 0x1039920e6 juce::MessageQueue::deliverNextMessage() + 246
4 juicysfplugin 0x103991f9e juce::MessageQueue::runLoopSourceCallback(void*) + 14
5 CoreFoundation 0x7ff81e3121ab __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
6 CoreFoundation 0x7ff81e312113 __CFRunLoopDoSource0 + 180
7 CoreFoundation 0x7ff81e311e8d __CFRunLoopDoSources0 + 242
8 CoreFoundation 0x7ff81e3108a8 __CFRunLoopRun + 892
9 CoreFoundation 0x7ff81e30fe6c CFRunLoopRunSpecific + 562
10 HIToolbox 0x7ff826fbe5e6 RunCurrentEventLoopInMode + 292
11 HIToolbox 0x7ff826fbe213 ReceiveNextEventCommon + 283
12 HIToolbox 0x7ff826fbe0e5 _BlockUntilNextEventMatchingListInModeWithFilter + 70
13 AppKit 0x7ff820d49fad _DPSNextEvent + 927
14 AppKit 0x7ff820d4866a -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1394
15 AppKit 0x7ff820d3ad19 -[NSApplication run] + 586
16 juicysfplugin 0x10391ba8f main + 191
17 dyld 0x10bb6052e start + 462
Thread 1:
0 libsystem_pthread.dylib 0x7ff81e246f48 start_wqthread + 0
Thread 2:
0 libsystem_pthread.dylib 0x7ff81e246f48 start_wqthread + 0
Thread 3:
0 libsystem_pthread.dylib 0x7ff81e246f48 start_wqthread + 0
Thread 4:: AMCP Logging Spool
0 libsystem_kernel.dylib 0x7ff81e20e9b6 semaphore_wait_trap + 10
1 caulk 0x7ff826d4a2e6 caulk::mach::semaphore::wait_or_error() + 16
2 caulk 0x7ff826d32148 caulk::concurrent::details::worker_thread::run() + 36
3 caulk 0x7ff826d31e0c void* caulk::thread_proxy<std::__1::tuple<caulk::thread::attributes, void (caulk::concurrent::details::worker_thread::*)(), std::__1::tuple<caulk::concurrent::details::worker_thread*> > >(void*) + 41
4 libsystem_pthread.dylib 0x7ff81e24b4e1 _pthread_start + 125
5 libsystem_pthread.dylib 0x7ff81e246f6b thread_start + 15
Thread 5:: com.apple.audio.IOThread.client
0 libsystem_kernel.dylib 0x7ff81e20e97a mach_msg_trap + 10
1 libsystem_kernel.dylib 0x7ff81e20ece8 mach_msg + 56
2 CoreAudio 0x7ff81fe91c07 HALB_MachPort::SendSimpleMessageWithSimpleReply(unsigned int, unsigned int, int, int&, bool, unsigned int) + 111
3 CoreAudio 0x7ff81fd1c729 HALC_ProxyIOContext::IOWorkLoop() + 3931
4 CoreAudio 0x7ff81fd1b205 invocation function for block in HALC_ProxyIOContext::HALC_ProxyIOContext(unsigned int, unsigned int) + 63
5 CoreAudio 0x7ff81fee7616 HALB_IOThread::Entry(void*) + 72
6 libsystem_pthread.dylib 0x7ff81e24b4e1 _pthread_start + 125
7 libsystem_pthread.dylib 0x7ff81e246f6b thread_start + 15
Thread 6:
0 libsystem_pthread.dylib 0x7ff81e246f48 start_wqthread + 0
Thread 7:: JUCE Timer
0 libsystem_kernel.dylib 0x7ff81e2113ea __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x7ff81e24ba6f _pthread_cond_wait + 1249
2 libc++.1.dylib 0x7ff81e1a9d93 std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) + 93
3 juicysfplugin 0x103946d42 juce::WaitableEvent::wait(int) const + 130
4 juicysfplugin 0x103995568 juce::Timer::TimerThread::run() + 200
5 juicysfplugin 0x103955fb1 juce::threadEntryProc(void*) + 481
6 libsystem_pthread.dylib 0x7ff81e24b4e1 _pthread_start + 125
7 libsystem_pthread.dylib 0x7ff81e246f6b thread_start + 15
Thread 8:: com.apple.NSEventThread
0 libsystem_kernel.dylib 0x7ff81e20e97a mach_msg_trap + 10
1 libsystem_kernel.dylib 0x7ff81e20ece8 mach_msg + 56
2 CoreFoundation 0x7ff81e31239d __CFRunLoopServiceMachPort + 319
3 CoreFoundation 0x7ff81e310a28 __CFRunLoopRun + 1276
4 CoreFoundation 0x7ff81e30fe6c CFRunLoopRunSpecific + 562
5 AppKit 0x7ff820eb79ce _NSEventThread + 132
6 libsystem_pthread.dylib 0x7ff81e24b4e1 _pthread_start + 125
7 libsystem_pthread.dylib 0x7ff81e246f6b thread_start + 15
Thread 9:
0 libsystem_pthread.dylib 0x7ff81e246f48 start_wqthread + 0
Thread 10:
0 libsystem_pthread.dylib 0x7ff81e246f48 start_wqthread + 0
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x0000000000000000 rcx: 0x00000000e1e63012 rdx: 0x00000000e206380e
rdi: 0x0000000000000000 rsi: 0x0000000000000008 rbp: 0x00007ff7bc5f42d0 rsp: 0x00007ff7bc5f4238
r8: 0x0000000000000410 r9: 0x0000000000000050 r10: 0x00000000000007fb r11: 0x00000000000000cb
r12: 0x00007fb7b0824f20 r13: 0x00007fb7b0824e30 r14: 0x00006000029480a0 r15: 0x00007fb7b0824f20
rip: 0x00007ff81e2ae8dd rfl: 0x0000000000010246 cr2: 0x0000000000000000
Logical CPU: 2
Error Code: 0x00000004 (no mapping for user data read)
Trap Number: 14
Thread 0 instruction stream:
00 48 8b 5d 80 48 85 db-74 0f 48 8d 3d 92 cd ca .H.].H..t.H.=...
42 e8 1a 79 fe ff 48 89-03 31 db 48 8b 7d 88 e8 B..y..H..1.H.}..
b7 e8 fe ff 48 8b 05 80-fc af 42 48 8b 00 48 3b ....H.....BH..H;
45 d0 75 14 89 d8 48 81-c4 88 00 00 00 5b 41 5c E.u...H......[A\
41 5d 41 5e 41 5f 5d c3-e8 20 61 1a 00 90 90 90 A]A^A_].. a.....
90 90 90 90 90 90 90 90-90 90 40 f6 c7 01 75 35 ..........@...u5
[48]8b 07 48 85 c0 74 1f-48 8b 0d 54 9b 72 41 48 H..H..t.H..T.rAH <==
39 c8 74 13 a8 01 74 1d-48 8b 15 dc 05 b0 42 48 9.t...t.H.....BH
23 02 48 39 c8 75 0e 48-8b 47 08 a8 04 75 13 48 #.H9.u.H.G...u.H
8b 7f 28 eb 15 48 8b 35-6f 2a 72 41 ff 25 59 06 ..(..H.5o*rA.%Y.
b0 42 48 83 c7 3f 48 83-e7 f0 48 89 f8 c3 90 90 .BH..?H...H.....
90 90 90 90 90 90 90 90-90 90 90 90 90 55 41 57 .............UAW
Binary Images:
0x7ff81e292000 - 0x7ff81e794fff com.apple.CoreFoundation (6.9) <c39123bf-9d62-3577-a11f-0a97cc4d9991> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
0x10390a000 - 0x103b31fff com.Birchlabs.JuicySFPlugin (3.0.0) <e46ecced-2502-38d0-944a-8386d9d21b68> /Users/USER/Sites/*/juicysfplugin.app/Contents/MacOS/juicysfplugin
0x7ff826f90000 - 0x7ff827283fff com.apple.HIToolbox (2.1.1) <95eab511-8fc1-353a-ac8a-26aa19032e2e> /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox
0x7ff820d0b000 - 0x7ff821b9afff com.apple.AppKit (6.9) <bcb46e9b-acd5-3b26-878f-a3988bc5df7e> /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
0x10bb5b000 - 0x10bbc6fff dyld (*) <f71fb3ca-5fcc-3577-9457-b047888a46d1> /usr/lib/dyld
0x7ff81e245000 - 0x7ff81e250fff libsystem_pthread.dylib (*) <f32b6d06-b156-3da0-b086-a31cf011362b> /usr/lib/system/libsystem_pthread.dylib
0x7ff81e20d000 - 0x7ff81e244fff libsystem_kernel.dylib (*) <792406fe-2224-3c14-ba9f-f076fd7839d2> /usr/lib/system/libsystem_kernel.dylib
0x7ff826d30000 - 0x7ff826d51fff com.apple.audio.caulk (1.0) <bfafc6ea-ebeb-3e0e-8142-e73436d78d67> /System/Library/PrivateFrameworks/caulk.framework/Versions/A/caulk
0x7ff81fb62000 - 0x7ff820297fff com.apple.audio.CoreAudio (5.0) <e59404d5-8edd-32fe-bbc7-940b167ea342> /System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio
0x7ff81e19e000 - 0x7ff81e1f6fff libc++.1.dylib (*) <08aaa9d6-930c-3b45-a5ae-2905f436b834> /usr/lib/libc++.1.dylib
External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
VM Region Summary:
ReadOnly portion of Libraries: Total=1.0G resident=0K(0%) swapped_out_or_unallocated=1.0G(100%)
Writable regions: Total=638.7M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=638.7M(100%)
Will keep working on it
I spent some time trying to build your plugin with the following changes:
The resulting file is:
https://github.com/studiorack/juicysf/blob/master/.github/workflows/release.yml
Pipeline and jobs:
https://github.com/studiorack/juicysf/actions/runs/2976408057
However the generated binaries are corrupted. When I try to validate them with pluginval I get:
And same with Steinberg validator:
Where I am going wrong with my build commands?
Would this be a feature you would be interested in using? I can submit a PR?