Open victor-shepardson opened 2 years ago
I notice that when using PackageCompiler with the 1.7.1 release build of Julia, libjulia.dylib
and the entry point built by PackageCompiler (in my case libscjulia.dylib
) end up in a lib
directory, but everything else e.g. libjulia-internal.dylib
are in lib/julia
. Whereas with the debug build, everything is together in lib
. However, I made sure to add both as rpaths when building my plugin:
$ otool -L build/JuliaUGen_scsynth.scx
build/JuliaUGen_scsynth.scx:
@rpath/libjulia.dylib (compatibility version 1.0.0, current version 1.7.1)
@rpath/libscjulia.0.1.0.dylib (compatibility version 0.0.0, current version 0.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
$ otool -l build/JuliaUGen_scsynth.scx | grep RPATH -A 2
cmd LC_RPATH
cmdsize 56
path @loader_path/../SCJulia/build/scjulia/lib/ (offset 12)
--
cmd LC_RPATH
cmdsize 64
path @loader_path/../SCJulia/build/scjulia/lib/julia/ (offset 12)
So I can't see how this difference would cause the failure.
I also notice that libscsynth
uses @executable_path
unlike my .scx
plugin and the other Julia .dylib
s:
$ otool -L SCJulia/build/scjulia/lib/libscjulia.dylib
SCJulia/build/scjulia/lib/libscjulia.dylib:
@rpath/libscjulia.0.1.0.dylib (compatibility version 0.0.0, current version 0.1.0)
@rpath/libjulia-debug.dylib (compatibility version 1.0.0, current version 1.7.1)
@rpath/libjulia-internal-debug.dylib (compatibility version 1.0.0, current version 1.7.1)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
$ otool -l SCJulia/build/scjulia/lib/libscjulia.dylib | grep RPATH -A 2
cmd LC_RPATH
cmdsize 32
path @executable_path (offset 12)
--
cmd LC_RPATH
cmdsize 40
path @executable_path/julia (offset 12)
$ otool -l SCJulia/build/scjulia/lib/libjulia-internal-debug.dylib | grep RPATH -A 2
cmd LC_RPATH
cmdsize 32
path @loader_path/ (offset 12)
this is true for both debug and release Julia. AFAICT the dynamic linker wouldn't find libjulia-internal-debug.dylib
using @executable_path
, which would point to the location of scsynth
which loads JuliaUGen_scsynth.scx
which loads libscjulia.dylib
which loads libjulia*.dylib
.
If I use install_name_tool
to change @executable_path
to @loader_path
here, nothing changes. debug still works and release still fails.
man dyld
says:
The run path stack is built from the LC_RPATH load commands in the dependency chain that lead to the current dylib load.
so I think it's the use of @loader_path
in JuliaUGen.scx
which is actually enabling libscjulia
to find further dependencies?
after more debugging, it definitely fails here: https://github.com/JuliaLang/PackageCompiler.jl/blob/ab5d6cf008b25a0868c61b21287d46ca2104ef05/src/julia_init.c#L27
where libname
is "libscjulia.0.1.0.dylib"
, and was compiled into libscjulia
itself here https://github.com/JuliaLang/PackageCompiler.jl/blob/94fdba2b99b7f0d3be9906de831c1ca6a0091488/src/PackageCompiler.jl#L583
if I start scsynth
with DYLD_PRINT_RPATHS=1 DYLD_PRINT_LIBRARIES=1
:
<initially loading all plugins>
...
dyld: loaded: /Users/victor/sc-julia/build/../SCJulia/build/scjulia/lib/libscjulia.dylib
RPATH successful expansion of @rpath/libscjulia.dylib to: /Users/victor/sc-julia/build/../SCJulia/build/scjulia/lib/libscjulia.dylib
...
<starting plugin, triggering init_julia call>
RPATH failed to expanding libscjulia.dylib to: /Users/victor/sc-julia/SCJulia/build/scjulia/lib/julia/libscjulia.dylib
ERROR: julia: Failed to load library at libscjulia.dylib
so the directory structure does seem to explain the debug/release difference:
when using PackageCompiler with the 1.7.1 release build of Julia, libjulia.dylib and the entry point built by PackageCompiler (in my case libscjulia.dylib) end up in a lib directory, but everything else e.g. libjulia-internal.dylib are in lib/julia. Whereas with the debug build, everything is together in lib.
I still don't understand why /Users/victor/sc-julia/SCJulia/build/scjulia/lib
wouldn't be in the search path at this point, nor why it even tries to expand RPATH since there's no @rpath/
in the name passed to dlopen
:
RPATH failed to expanding libscjulia.dylib to: /Users/victor/sc-julia/SCJulia/build/scjulia/lib/julia/libscjulia.dylib
^
there is no @rpath ???
like the locally built julia-debug 1.7.1, locally built 1.7.1 in release mode also works as expected and causes PackageCompiler to place everything under lib/
and not lib/julia
.
this directory structure appears to be inherited from the Julia install. reflects <repo>/usr/lib
in the locally built case, /Applications/Julia-1.7.app/Contents/Resources/julia/lib
in the installed case.
Hi, I'm trying to use PackageCompiler to build a shared library which I can link into a relocatable plugin for a complex piece of C++ software, specifically, scsynth (https://supercollider.github.io/).
OS: macOS 10.14.6 (x86_64-apple-darwin18.7.0) cpp: Apple LLVM version 10.0.1 (clang-1001.0.46.4) Julia: 1.7.1 .app release / 1.7.1 debug built locally with
git checkout v1.7.1; make debug
Currently my plugin fails during the
init_julia
call, which is in a class constructor not inmain
since this is a plugin.here's the backtrace:
looks like maybe somewhere in here: https://github.com/JuliaLang/PackageCompiler.jl/blob/v2.0.4/src/julia_init.c#L27-L38 ?
I'm not sure if this will turn out to be a PackageCompiler issue, scsynth issue or user error:
init_julia
code in a minimal C++ program (https://github.com/victor-shepardson/sc-julia/blob/master/plugins/JuliaUGen/test.cpp) works as expected, so the scsynth environment may be relevant.code is here: https://github.com/victor-shepardson/sc-julia