JuliaLang / PackageCompiler.jl

Compile your Julia Package
https://julialang.github.io/PackageCompiler.jl/dev/
MIT License
1.42k stars 189 forks source link

macOS shared library fails during `init_julia` #662

Open victor-shepardson opened 2 years ago

victor-shepardson commented 2 years ago

Hi, I'm trying to use PackageCompiler to build a shared library which I can link into a relocatable plugin for a complex piece of C++ software, specifically, scsynth (https://supercollider.github.io/).

OS: macOS 10.14.6 (x86_64-apple-darwin18.7.0) cpp: Apple LLVM version 10.0.1 (clang-1001.0.46.4) Julia: 1.7.1 .app release / 1.7.1 debug built locally with git checkout v1.7.1; make debug

Currently my plugin fails during the init_julia call, which is in a class constructor not in main since this is a plugin.

int argc_ = 1;
const char* arg = "";
char** argv_ = const_cast<char**>(&arg);
init_julia(argc_, argv_);

here's the backtrace:

Executable module set to "/Applications/SuperCollider.app/Contents/Resources/scsynth".
Architecture set to: x86_64h-apple-macosx-.
(lldb) c
Process 54584 resuming
Process 54584 stopped
* thread #7, name = 'com.apple.audio.IOThread.client', stop reason = signal SIGABRT
    frame #0: 0x00007fff7127e2c2 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff7127e2c2 <+10>: jae    0x7fff7127e2cc            ; <+20>
    0x7fff7127e2c4 <+12>: movq   %rax, %rdi
    0x7fff7127e2c7 <+15>: jmp    0x7fff71278453            ; cerror_nocancel
    0x7fff7127e2cc <+20>: retq
Target 0: (scsynth) stopped.
(lldb) thread backtrace
* thread #7, name = 'com.apple.audio.IOThread.client', stop reason = signal SIGABRT
  * frame #0: 0x00007fff7127e2c2 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff71339bf1 libsystem_pthread.dylib`pthread_kill + 284
    frame #2: 0x00007fff711e86a6 libsystem_c.dylib`abort + 127
    frame #3: 0x00007fff6e3c1641 libc++abi.dylib`abort_message + 231
    frame #4: 0x00007fff6e3c1704 libc++abi.dylib`default_terminate_handler() + 48
    frame #5: 0x00007fff6e3cd19e libc++abi.dylib`std::__terminate(void (*)()) + 8
    frame #6: 0x00007fff6e3cd225 libc++abi.dylib`std::terminate() + 69
    frame #7: 0x00007fff6e372960 libc++.1.dylib`std::__1::thread::~thread() + 16
    frame #8: 0x00007fff711e93cf libsystem_c.dylib`__cxa_finalize_ranges + 319
    frame #9: 0x00007fff711e96b3 libsystem_c.dylib`exit + 55
    frame #10: 0x000000010b19d64b libjulia-internal.1.7.dylib`jl_exit + 27
    frame #11: 0x000000010b0d5b3d libjulia-internal.1.7.dylib`jl_vexceptionf + 285
    frame #12: 0x000000010b0d5bd2 libjulia-internal.1.7.dylib`jl_errorf + 146
    frame #13: 0x000000010764a44c libscjulia.0.1.0.dylib`init_julia + 300
    frame #14: 0x0000000107296977 JuliaUGen_scsynth.scx`JuliaUGen::JuliaUGen::JuliaUGen(this=0x000000012734aad4) at JuliaUGen.cpp:16:9
    frame #15: 0x0000000107296a85 JuliaUGen_scsynth.scx`JuliaUGen::JuliaUGen::JuliaUGen(this=0x000000012734aad4) at JuliaUGen.cpp:11:24
    frame #16: 0x0000000107296d65 JuliaUGen_scsynth.scx`void detail::constructClass<JuliaUGen::JuliaUGen>(unit=0x000000012734aad4) at SC_PlugIn.hpp:242:98
    frame #17: 0x0000000101565c4b scsynth`Graph_FirstCalc(inGraph=0x000000012734a580) at SC_Graph.cpp:440:9 [opt]
    frame #18: 0x000000010156e5c7 scsynth`::Group_Calc(inGroup=<unavailable>) at SC_Group.cpp:69:9 [opt]
    frame #19: 0x000000010156e5c7 scsynth`::Group_Calc(inGroup=<unavailable>) at SC_Group.cpp:69:9 [opt]
    frame #20: 0x0000000101563efa scsynth`void SC_CoreAudioDriver::Run<true>(this=0x0000000167b47000, inInputData=0x00007fc8048be6b0, outOutputData=0x00007fc7fff05f50, oscTime=<unavailable>) at SC_CoreAudio.cpp:1370:13 [opt]
    frame #21: 0x0000000101562881 scsynth`int appIOProc<true>(device=<unavailable>, inNow=<unavailable>, inInputData=0x00007fc805002700, inInputTime=<unavailable>, outOutputData=0x00007fc7fff05f50, inOutputTime=0x000000010c63d0e0, defptr=0x0000000167b47000) at SC_CoreAudio.cpp:0:10 [opt]
    frame #22: 0x00007fff44c181b2 CoreAudio`HALC_ProxyIOContext::IOWorkLoop() + 4908
    frame #23: 0x00007fff44c16cd4 CoreAudio`HALC_ProxyIOContext::IOThreadEntry(void*) + 122
    frame #24: 0x00007fff44c16836 CoreAudio`HALB_IOThread::Entry(void*) + 72
    frame #25: 0x00007fff713372eb libsystem_pthread.dylib`_pthread_body + 126
    frame #26: 0x00007fff7133a249 libsystem_pthread.dylib`_pthread_start + 66
    frame #27: 0x00007fff7133640d libsystem_pthread.dylib`thread_start + 13

looks like maybe somewhere in here: https://github.com/JuliaLang/PackageCompiler.jl/blob/v2.0.4/src/julia_init.c#L27-L38 ?

I'm not sure if this will turn out to be a PackageCompiler issue, scsynth issue or user error:

code is here: https://github.com/victor-shepardson/sc-julia

victor-shepardson commented 2 years ago

I notice that when using PackageCompiler with the 1.7.1 release build of Julia, libjulia.dylib and the entry point built by PackageCompiler (in my case libscjulia.dylib) end up in a lib directory, but everything else e.g. libjulia-internal.dylib are in lib/julia. Whereas with the debug build, everything is together in lib. However, I made sure to add both as rpaths when building my plugin:

$ otool -L build/JuliaUGen_scsynth.scx
build/JuliaUGen_scsynth.scx:
    @rpath/libjulia.dylib (compatibility version 1.0.0, current version 1.7.1)
    @rpath/libscjulia.0.1.0.dylib (compatibility version 0.0.0, current version 0.1.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

$ otool -l build/JuliaUGen_scsynth.scx | grep RPATH -A 2
          cmd LC_RPATH
      cmdsize 56
         path @loader_path/../SCJulia/build/scjulia/lib/ (offset 12)
--
          cmd LC_RPATH
      cmdsize 64
         path @loader_path/../SCJulia/build/scjulia/lib/julia/ (offset 12)

So I can't see how this difference would cause the failure.

I also notice that libscsynth uses @executable_path unlike my .scx plugin and the other Julia .dylibs:

$ otool -L SCJulia/build/scjulia/lib/libscjulia.dylib
SCJulia/build/scjulia/lib/libscjulia.dylib:
    @rpath/libscjulia.0.1.0.dylib (compatibility version 0.0.0, current version 0.1.0)
    @rpath/libjulia-debug.dylib (compatibility version 1.0.0, current version 1.7.1)
    @rpath/libjulia-internal-debug.dylib (compatibility version 1.0.0, current version 1.7.1)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

$ otool -l SCJulia/build/scjulia/lib/libscjulia.dylib | grep RPATH -A 2
          cmd LC_RPATH
      cmdsize 32
         path @executable_path (offset 12)
--
          cmd LC_RPATH
      cmdsize 40
         path @executable_path/julia (offset 12)

$ otool -l SCJulia/build/scjulia/lib/libjulia-internal-debug.dylib | grep RPATH -A 2
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)

this is true for both debug and release Julia. AFAICT the dynamic linker wouldn't find libjulia-internal-debug.dylib using @executable_path, which would point to the location of scsynth which loads JuliaUGen_scsynth.scx which loads libscjulia.dylib which loads libjulia*.dylib.

If I use install_name_tool to change @executable_path to @loader_path here, nothing changes. debug still works and release still fails.

man dyld says:

The run path stack is built from the LC_RPATH load commands in the dependency chain that lead to the current dylib load.

so I think it's the use of @loader_path in JuliaUGen.scx which is actually enabling libscjulia to find further dependencies?

victor-shepardson commented 2 years ago

after more debugging, it definitely fails here: https://github.com/JuliaLang/PackageCompiler.jl/blob/ab5d6cf008b25a0868c61b21287d46ca2104ef05/src/julia_init.c#L27

where libname is "libscjulia.0.1.0.dylib", and was compiled into libscjulia itself here https://github.com/JuliaLang/PackageCompiler.jl/blob/94fdba2b99b7f0d3be9906de831c1ca6a0091488/src/PackageCompiler.jl#L583

if I start scsynth with DYLD_PRINT_RPATHS=1 DYLD_PRINT_LIBRARIES=1:

<initially loading all plugins>
...
dyld: loaded: /Users/victor/sc-julia/build/../SCJulia/build/scjulia/lib/libscjulia.dylib
RPATH successful expansion of @rpath/libscjulia.dylib to: /Users/victor/sc-julia/build/../SCJulia/build/scjulia/lib/libscjulia.dylib
...
<starting plugin, triggering init_julia call>
RPATH failed to expanding     libscjulia.dylib to: /Users/victor/sc-julia/SCJulia/build/scjulia/lib/julia/libscjulia.dylib
ERROR: julia: Failed to load library at libscjulia.dylib

so the directory structure does seem to explain the debug/release difference:

when using PackageCompiler with the 1.7.1 release build of Julia, libjulia.dylib and the entry point built by PackageCompiler (in my case libscjulia.dylib) end up in a lib directory, but everything else e.g. libjulia-internal.dylib are in lib/julia. Whereas with the debug build, everything is together in lib.

I still don't understand why /Users/victor/sc-julia/SCJulia/build/scjulia/lib wouldn't be in the search path at this point, nor why it even tries to expand RPATH since there's no @rpath/ in the name passed to dlopen:

RPATH failed to expanding     libscjulia.dylib to: /Users/victor/sc-julia/SCJulia/build/scjulia/lib/julia/libscjulia.dylib
                             ^
                            there is no @rpath ???
victor-shepardson commented 2 years ago

like the locally built julia-debug 1.7.1, locally built 1.7.1 in release mode also works as expected and causes PackageCompiler to place everything under lib/ and not lib/julia.

this directory structure appears to be inherited from the Julia install. reflects <repo>/usr/lib in the locally built case, /Applications/Julia-1.7.app/Contents/Resources/julia/lib in the installed case.