Closed mfkl closed 6 years ago
Maybe this can give you ideas for your specific case, https://github.com/garuma/GuiUnitNg/blob/master/GuiUnitNg/GuiUnit/MonoMacMainLoopIntegration.cs#L31 this is how GuiUnitNg initializes Xamarin.Mac.
But if you just want to dlopen
into a library and friends you can just use
[DllImport ("/usr/lib/libSystem.dylib")]
static extern IntPtr dlopen (string path, int mode);
and this should not require Xamarin.Mac at all.
About DllMap config file I have not used one myself but according to docs http://www.mono-project.com/docs/advanced/pinvoke/dllmap/ it should work, also this one is really handy http://www.mono-project.com/docs/advanced/pinvoke/
Hope this helps, feel free to reopen if you have additional questions.
Thanks for your message Alex.
Maybe this can give you ideas for your specific case, https://github.com/garuma/GuiUnitNg/blob/master/GuiUnitNg/GuiUnit/MonoMacMainLoopIntegration.cs#L31 this is how GuiUnitNg initializes Xamarin.Mac.
So what you're saying is that all Xamarin.Mac applications that do P/Invoke need to bundle libxammac.dylib
, load it and Init
it first before P/Invoke calls?
But if you just want to
dlopen
dlopen
on my dylibs is fine, but the following P/Invoke calls fail. I don't want to dlsym
manually all symbols as this is a crossplatform binding based on P/Invoke.
Are you saying bundling, loading and initializing libxammac.dylib
(which does not seem to be in the .app bundle out of the box) is the way I should do things to be able to perform P/Invoke calls?
feel free to reopen.
Looks like I can't.
So what you're saying is that all Xamarin.Mac applications that do P/Invoke need to bundle
libxammac.dylib
, load it and Init it first before P/Invoke calls?
Not at all, I wasn't sure if you wanted to use Xamarin.Mac on your own tooling, P/Invoke is .NET feature so Xamarin.Mac is not needed at all for it.
Ok lets go one step back, from your bug report you have this
Steps to Reproduce
- Generate package from https://github.com/mfkl/libvlc-nuget/pull/5 (not on nuget yet)
- Add it to a Xamarin.Mac project
- Try loading the dylib at runtime and P/Invoking into it (CLI launch for interesting debug logs).
So if you are creating a Xamarin.Mac App with VisualStudio for Mac you do not need to bundle libxammac.dylib
at all since our tooling should do the right thing for it.
Now from your actual issue
Unhandled Exception:
System.DllNotFoundException: Could not find the runtime library libxammac.dylib
at ObjCRuntime.Runtime.LookupInternalFunction[T] (System.String name) [0x0010a] in /Library/Frameworks/Xamarin.Mac.framework/Versions/4.4.1.193/src/Xamarin.Mac/ObjCRuntime/Runtime.mac.cs:107
at ObjCRuntime.Runtime.EnsureInitialized () [0x00030] in /Library/Frameworks/Xamarin.Mac.framework/Versions/4.4.1.193/src/Xamarin.Mac/ObjCRuntime/Runtime.mac.cs:117
at AppKit.NSApplication.Init () [0x00016] in /Library/Frameworks/Xamarin.Mac.framework/Versions/4.4.1.193/src/Xamarin.Mac/AppKit/NSApplication.cs:56
I am unsure if having a dllMap
file might be confusing somehow our logic, @chamons does this rings a bell?
@mfkl is there any chance you could provide us with a test case so we can reproduce this ourselves and we have the exact project setup you have??
On a side note there are other projects like SkiaSharp and Urho3d that are doing the same thing you want to achieve so you might want to take a look at them too :)
Looks like SkiaSharp uses a Xamarin.Mac binding project1 and some #if directives0 to drive the P/Invoke lib they use, neat trick.
You could try setting MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll to see where mono is trying to load libxammac.dylib from.
As @dalexsoto noted, having a sample showing your issue would be very useful, there isn't a lot we can do beyond guess from here.
if you are creating a Xamarin.Mac App with VisualStudio for Mac you do not need to bundle libxammac.dylib at all since our tooling should do the right thing for it.
Ok, thanks for the confirmation.
On a side note there are other projects like SkiaSharp and Urho3d that are doing the same thing you want to achieve so you might want to take a look at them too :) Looks like SkiaSharp uses a Xamarin.Mac binding project1 and some #if directives0 to drive the P/Invoke lib they use, neat trick.
I did have a look but I am not using objective sharpie to auto generate binding from Obj-c so I guess that's a bit different. Using #if directives for P/Invoke on iOS too btw :) https://github.com/videolan/libvlcsharp/blob/11a72ffee462d407c83b89ed94d0e4d7bf8cffd4/LibVLCSharp/Shared/Core.cs#L106
having a sample showing your issue would be very useful, there isn't a lot we can do beyond guess from here.
My bad, here is a minimal repro: https://github.com/mfkl/LibVLC.PInvoke.Mac
You could try setting MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll to see where mono is trying to load libxammac.dylib from.
Martz-2:MonoBundle Martz$ mono LibVLC.PInvoke.Mac.exe
Mono: DllImport attempting to load: '/usr/lib/libSystem.dylib'.
Mono: DllImport loaded library '/usr/lib/libSystem.dylib'.
Mono: DllImport searching in: '/usr/lib/libSystem.dylib' ('/usr/lib/libSystem.dylib').
Mono: Searching for 'dlopen'.
Mono: Probing 'dlopen'.
Mono: Found as 'dlopen'.
140273887815600 # this is me printing the result of dlopen on my 2 dylibs
140273889895568 # this is me printing the result of dlopen on my 2 dylibs
Mono: DllImport attempting to load: '/usr/lib/libobjc.dylib'.
Mono: DllImport loaded library '/usr/lib/libobjc.dylib'.
Mono: DllImport searching in: '/usr/lib/libobjc.dylib' ('/usr/lib/libobjc.dylib').
Mono: Searching for 'sel_registerName'.
Mono: Probing 'sel_registerName'.
Mono: Found as 'sel_registerName'.
Mono: DllImport searching in: '/usr/lib/libobjc.dylib' ('/usr/lib/libobjc.dylib').
Mono: Searching for 'objc_getClass'.
Mono: Probing 'objc_getClass'.
Mono: Found as 'objc_getClass'.
Mono: DllImport searching in: '/usr/lib/libSystem.dylib' ('/usr/lib/libSystem.dylib').
Mono: Searching for 'dlsym'.
Mono: Probing 'dlsym'.
Mono: Found as 'dlsym'.
You are using dlopen without a full path, retrying by prepending /usr/lib
Unhandled Exception:
System.DllNotFoundException: Could not find the runtime library libxammac.dylib
at ObjCRuntime.Runtime.LookupInternalFunction[T] (System.String name) [0x000a6] in <91a95489a34f4ae887dadc916f0228c3>:0
at ObjCRuntime.Runtime.EnsureInitialized () [0x00030] in <91a95489a34f4ae887dadc916f0228c3>:0
at AppKit.NSApplication.Init () [0x00016] in <91a95489a34f4ae887dadc916f0228c3>:0
at LibVLC.PInvoke.Mac.MainClass.Main (System.String[] args) [0x00049] in <10a0767b1d2d469b880d78cded2947b0>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.DllNotFoundException: Could not find the runtime library libxammac.dylib
at ObjCRuntime.Runtime.LookupInternalFunction[T] (System.String name) [0x000a6] in <91a95489a34f4ae887dadc916f0228c3>:0
at ObjCRuntime.Runtime.EnsureInitialized () [0x00030] in <91a95489a34f4ae887dadc916f0228c3>:0
at AppKit.NSApplication.Init () [0x00016] in <91a95489a34f4ae887dadc916f0228c3>:0
at LibVLC.PInvoke.Mac.MainClass.Main (System.String[] args) [0x00049] in <10a0767b1d2d469b880d78cded2947b0>:0
I'd assume that once loaded with ObjCRuntime.Dlfcn.dlopen
, the librairies are ready for P/Invoke but I guess that's a wrong assumption (as P/Invoke does a dlopen anyway).
@mfkl I think SkiaSharp is using the binding project just so the tooling places the .dylib in the right spot see https://github.com/mono/SkiaSharp/blob/master/binding/SkiaSharp.OSX/Properties/AssemblyInfo.cs#L5
I'm taking a look at your sample shortly.
I am using both a Xamarin.Mac binding project and a normal net45 project.
https://github.com/mono/SkiaSharp/blob/master/binding/SkiaSharp.OSX/SkiaSharp.OSX.csproj
For the net45 project, I use a targets file to copy the dylib next to the app.
For both cases I pinvoke libSkiaSharp.dylib directly, https://github.com/mono/SkiaSharp/blob/master/binding/Binding/SkiaApi.cs#L82
I think SkiaSharp is using the binding project just so the tooling places the .dylib in the right spot see https://github.com/mono/SkiaSharp/blob/master/binding/SkiaSharp.OSX/Properties/AssemblyInfo.cs#L5
hmm, I am not using that LinkWith
attribute.
Hi Matthew,
I am using both a Xamarin.Mac binding project and a normal net45 project.
Ok. Personally, I'm not using a Xamarin.Mac binding project. I see that you're using the new sdk style csproj, probably should use that as well on OSX.
I use a targets file to copy the dylib next to the app.
I've filed a PR against your sample with a fix to your project plus a small work around:
https://github.com/mfkl/LibVLC.PInvoke.Mac/pull/1
The full explanation will take a bit, which I'll type soon.
While researching this, I found this cute bug: https://github.com/xamarin/xamarin-macios/issues/4434
So there are three parts to my PR, two are legit and one is working around a bug. We'll go in order:
The first bit, MonoBundlingExtraArgs, is working around https://github.com/xamarin/xamarin-macios/issues/4435
We should be doing that for you, but aren't so you have to add that. That way the native libraries can find each other.
The second part with NativeReference is adding the two native libraries in question to your native references. This means MMP processes them, copies them into your final bundle, and sets it up so the launcher references them. This is needed and expected.
The third part is a bit of msbuild magic to run two ln -s
commands post building your bundle. The reason we need this is that your have two native libraries, with numbers, but the dependencies between them reference the name without the numbers. Your package dependency had copies without the next of text files, but I tried copying them in and it didn't help.
That happened to trigger #4434 , but on macOS the standard pattern I believe is to use sym links to handle this, so I added a step to do that for now.
If this makes sense we can close this and use #4435 to track the one bug this found.
Thanks Chris for taking the time to write all this up, it's really useful!
So, I pulled out your branch and the P/Invoke call works fine (but returns 0x0
) when launching the app from VS, but for some reason I get the same error from CLI. System.DllNotFoundException: Could not find the runtime library libxammac.dylib
The second part with NativeReference is adding the two native libraries in question to your native references. This means MMP processes them, copies them into your final bundle, and sets it up so the launcher references them. This is needed and expected.
Yes.
The reason I did not use NativeReference
(like in libvlc for iOS) is because those 2 dylibs must be in a lib
folder, which must be in the same directory as the plugins
folder which contains the bundle dylibs which the core will load/unload. That relative location must be respected for the plugins to be found.
AFAIK NativeReference
does not allow you to put your dylib in the location of your choice, and the root of the bundle sadly does not work in the case of libvlc.
I tried modifying your PR by adding /lib
to the MonoBundlingExtraArgs
and Target
but no luck so far.
Hmm, how are you launching it from the command line? I'm doing this just fine?
./LibVLC.PInvoke.Mac/bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MacOS/LibVLC.PInvoke.Mac
You are correct NativeReference makes a number of simplifying assumptions on file locations. To do something custom would require a bit of hacking. Let me poke around for a few...
To do something custom would require a bit of hacking. Maybe something like this:
You might need to add more linker commands to get your launcher to link against that new location, but that is a start.
Hmm, how are you launching it from the command line? I'm doing this just fine?
Yep that works, was doing it wrong:
mono --debug ./bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MonoBundle/LibVLC.PInvoke.Mac.exe
You might need to add more linker commands to get your launcher to link against that new location, but that is a start.
Thanks for the gist, looking into this now. Unsure about the syntax to add multiple rpath args though. Also, the dylibs in plugins
are Mach-O 64-bit bundle x86_64
unlike the ones in lib
which are Mach-O 64-bit dynamically linked shared library x86_64
.
Do you think that difference matters regarding the linker options needed?
This is the directory structure of the official VLC app for macOS.
Contents
└───MacOS
└───lib
└───plugins
└───VLC
└───Resources
└───Frameworks
Is the best practice for MonoMac apps to put the dylibs in the MonoBundle directory (as NativeReference
seems to be doing)?
Again, thanks for your help.
Yeah, you can't just run the executable, you have to run the bundle launcher (the reason we bundled things up).
MonoBundle is the general "everything" directory in XM bundles, we dump managed assemblies, the debug symbols, native references, and a few more things. There are a number of places in users code that assume things live next to their assemblies and it makes things easier for them.
If you got your managed p/invoke s right, there is no reason you couldn't add them to MacOS, or even a top level folder in Contents if you want (like I did with plugins/lib).
I believe you can just add multiple rpath params via something like:
<MonoBundlingExtraArgs>--link_flags="-rpath '@executable_path/../plugins/lib' -rpath '@executable_path/../Foo'"</MonoBundlingExtraArgs>
Hi Chris,
Thanks.
Still trying to understand why the plugins are not found, though they are in the right location. I just copy them with
<Copy SourceFiles="@(PluginsSource)" DestinationFiles="@(PluginsSource->'$(AppBundleDir)/Contents/MacOS/plugins/%(RecursiveDir)%(Filename)%(Extension)')" />
But it's like they are not there when P/Invoking libvlc.dylib
... (libvlc_new
returns null
)
I don't think referencing them with NativeReference
is an option ld: can't link with bundle (MH_BUNDLE) only dylibs (MH_DYLIB) file
. But my guess is that, before that ld
error, xamarin-macos tooling, through NativeReference
, calls
xcrun -sdk macosx install_name_tool -id @executable_path/../MonoBundle/libpsychedelic_plugin.dylib /Users/Mtz/LibVLC.PInvoke.Mac/LibVLC.PInvoke.Mac/bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MonoBundle/libpsychedelic_plugin.dylib
on each bundle dylibs, which my guess is needed and likely the fix to my issue... Here again, I cannot use the NativeReference
I'm afraid.
Still investigating, will keep you posted.
Can you update your sample with what you are trying, or post something. It's a tad hard to follow with the long discussion.
Wait, i see it here (in the branch, I was looking in master) - https://github.com/mfkl/LibVLC.PInvoke.Mac/pull/1#issuecomment-405909734
I'm seeing this when I try to build. Are all of your changes landed there?
EXEC : error : /Applications/Xcode94.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: (No such file or directory)
/Users/donblas/Programming/Projects/LibVLC.PInvoke.Mac/LibVLC.PInvoke.Mac/LibVLC.PInvoke.Mac.csproj(121,5): error MSB3073: The command "xcrun -sdk macosx install_name_tool -id @executable_path/../MonoBundle/plugins/"" """ exited with code 1.```
Sorry about that, should be clearer now. Only 1 branch and added latest package on nuget (with empty targets file to not conflict with the testing csproj).
I think my final issue is with the dylibs rpath.
Consider
otool -L libremap_plugin.dylib
libremap_plugin.dylib:
@rpath/libvlccore.dylib (compatibility version 10.0.0, current version 10.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
and
otool -L libvlc.5.dylib
libvlc.5.dylib:
@rpath/libvlc.dylib (compatibility version 12.0.0, current version 12.0.0)
@rpath/libvlccore.dylib (compatibility version 10.0.0, current version 10.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
If @rpath
is the current location of the dylib passed as a parameter, then the plugins dylib have a wrong path to libvlccore.dylib
? Unsure whether (and how) @loader_path
fits in the picture as well though...
This looks relevant https://github.com/videolan/vlckit/blob/8c6d7425770273671fb3e8fb6a01797230f2be2e/Resources/VLCKit/installPluginsVLCKit.sh#L99-L118
Additionally, from the official VLC.app
otool -L VLC
VLC:
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1450.15.0)
/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
@executable_path/../Frameworks/Breakpad.framework/Versions/A/Breakpad (compatibility version 1.0.0, current version 1.0.0)
@rpath/libvlc.dylib (compatibility version 12.0.0, current version 12.0.0)
@rpath/libvlccore.dylib (compatibility version 10.0.0, current version 10.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 1561.20.106)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1450.15.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
I miss those 2 @rpath
it seems, in my Xamarin.Mac app...
Should note that VLC
is the binary and located in the same directory than the plugins/
and lib/
folders. In our case, if I'm not mistaken, paths start from inside MonoBundle
.
So @rpath doesn't work exactly that way. Let me give you the 15 second version:
@
@rpath
says go look through the list of rpath that someone setsWe set up some rpaths for you with the additional mmp arguments:
--link_flags="-rpath '@executable_path/../MonoBundle/lib' -rpath '@executable_path/../MonoBundle/plugins'" -v
LibVLC.PInvoke.Mac.app/Contents/MacOS/
../MonoBundle/lib
to the rpath which is LibVLC.PInvoke.Mac.app/Contents/MonoBundle/lib
. LibVLC.PInvoke.Mac.app/Contents/MonoBundle/plugins
.I'm looking at your sample now, but this is already long enough...
If you dump the loader commands on your executable:
otool -lv bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MacOS/LibVLC.PInvoke.Mac
Load command 91
cmd LC_RPATH
cmdsize 48
path @executable_path/../MonoBundle/lib (offset 12)
Load command 92
cmd LC_RPATH
cmdsize 56
path @executable_path/../MonoBundle/plugins (offset 12)
Both rpaths are being set.
Ok, now that that looks good, let's try running it and look for errors. The magic incantation to get mono to scream when you it can't load libraries is:
MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll LibVLC.PInvoke.Mac/bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MacOS/LibVLC.PInvoke.Mac
Which gives us output too large to paste inline:
https://gist.github.com/chamons/1a8b66d3941e110da8e77780f959b045
It starts with:
info: DllImport attempting to load: 'libvlc.5'.
likely from your p/invoke.
The next few lines makes you think it failed but then we hit:
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: Searching for 'libvlc_new'.
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: Probing 'libvlc_new'.
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: Found as 'libvlc_new'.
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: DllImport searching in: 'libvlc.5' ('libvlc.5.dylib').
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: Searching for 'libvlc_errmsg'.
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: Probing 'libvlc_errmsg'.
2018-07-20 09:52:30.598 LibVLC.PInvoke.Mac[22308:1162264] info: Found as 'libvlc_errmsg'.
and we're not getting an exception. However, your LibVLCNew call is returning null.
I hacked LibraryName path to be an absolute path to the library and that did not change behavior.
I'd attach a native debugger:
$ lldb LibVLC.PInvoke.Mac/bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MacOS/LibVLC.PInvoke.Mac
(lldb) target create "LibVLC.PInvoke.Mac/bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MacOS/LibVLC.PInvoke.Mac"
Current executable set to 'LibVLC.PInvoke.Mac/bin/Debug/LibVLC.PInvoke.Mac.app/Contents/MacOS/LibVLC.PInvoke.Mac' (x86_64).
(lldb) b libvlc_new
Breakpoint 1: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.
(lldb) r
and try to figure out what's going on inside your native library. You may need to obtain symbol files for them and re-run, since everything is in assembly.
In the end when I built libvlc for mac with the available script, I ended up with all the dylibs statically linked into a .framework. It's simpler and works, though the initial way with many dylibs offered more flexibility (for things like this). Maybe later when I have more time!
Thanks for your help and time though!
Hello!
I'm currently attempting to add macOS support to LibVLCSharp (libvlc binding for xamarin). After some issues regarding the packaging/bundling of dylibs, I managed to copy them all in the final .app bundle. My next (and hopefully final) issue to get this working is making the
DllImport
calls work.Steps to Reproduce
Expected Behavior
Can perform P/Invoke calls on a dylib after loading it with
ObjCRuntime.Dlfcn.dlopen
Actual Behavior
Calling
dlopen
on them beforedoes a return a not null pointer, but dllimport calls fail with
I have a dllMap file if that matters
Environment
I could bundle the
libxammac.dylib
in my .app bundle but... that does not feel right at all (and crashes for another reason, anyway).Any help or insights greatly appreciated.