Loading libMoltenVK.dylib instead of libvulkan.dylib on iOS

[jhvst]

Hi there,

I saw there is an old commit made here: 594a476fdb7a9fe211a7257f353dbf9807e87786, which renamed the dynamic library load path on iOS to be libvulkan.dylib instead of libMoltenVK.dylib. However, it seems to cause crashes on iOS if the standard MoltenVK build instructions are followed.

In specific, it seems that building MoltenVK somehow internally assumes that the file is named in this specific way. When any code where ash instance is created using the load() function, it searches for libvulkan.dylib in the iOS Frameworks folder. The unexpected behaviour is that if this file does exists, an error is produced which says that libMoltenVK.dylib does not exist. If copies of the two files are added inside the Frameworks folder, everything works.

I can put sample applications online in a few days and submit a PR if this seems something you want to fix. Alternatively, I could try to find a way to fix the naming scheme and make instructions on how to do so. If you already know how to fix this, maybe reference that in the README?

[Ralith]

ash is designed to be used with the Vulkan loader ("libvulkan"), not directly targeting a specific ICD. Is there a reason you can't use that?

[jhvst]

I was not aware that there could be some other loader program that could be used instead of MoltenVK on iOS. Because of this I was thinking that maybe the assumption could be made, but I see the reasoning why this is not the case now.

Thanks for the prompt reply, I will close this ticket for now.

[MarijnS95]

I also wasn't aware that libMoltenVK could be loaded through the standard Vulkan loader (does MacOS ship with one at all? I've never physically touched this platform) [EDIT: There's even a reference to its json ICD file in our readme]. There is however ash-molten which statically links against libMoltenVK which might do what you need @jhvst?

Perhaps we should reference this project from the README, if someone can write up how it differs from the instructions in there already.

[jhvst]

In my initial description both libvulkan.dylib and libMoltenVK.dylib are the same file that is produced by the MoltenVK build process for iOS. I now recognize how you might have misread this, but libvulkan.dylib is not a Vulkan loader, it's the MoltenVK library renamed. Since there is no other way currently than MoltenVK to use Vulkan bindings on Mac and iOS, I thought that changing the default on iOS makes sense -- I use MacOS as my development machine and this I control with the ICD files as environment paths. My problem with iOS is that I do not know how to set up similar environment paths (I will look into this eventually, but it's not my priority right now), nor did it even occur to me at the time of opening this ticket that I had setup my Mac a long time ago with the ICD paths. With the iOS in particular being more constrained environment, I was thinking if it would make sense to revert the referenced change for iOS.

ash-molten is also something I completely missed, the process seems generalized version of mine. I can see why it does not run into the same problem, because it's not using dynamic linking. I think I might as well move into that crate later on. Thanks for the heads-up and sorry about the fuss!

[Ralith]

I don't have personal experience with MacOS either, but I do know it doesn't ship with the loader; applications would have to bundle it themselves.

[MarijnS95]

In my initial description both libvulkan.dylib and libMoltenVK.dylib are the same file that is produced by the MoltenVK build process for iOS. I now recognize how you might have misread this, but libvulkan.dylib is not a Vulkan loader, it's the MoltenVK library renamed.

I have not misread this, I have simply subsumed that - if there exists a MacOS loader - it's probably called something like libvulkan.dylib (with .1 actually) which would be a plausible explanation for 594a476fdb7a9fe211a7257f353dbf9807e87786 to exist. I clearly understood that you renamed it in hopes of tricking ash into loading libMoltenVK through the "ICD loader" library name (this is possible with some drivers, as long as they expose the vkGetInstanceProcAddr symbol).

Do Mac dylib files have something similar to SONAME that might trip up a load function somewhere, resulting in a search for libMoltenVK.dylib at all? Are you sure your replaced libvulkan.dylib in the right place, that ash really isn't accidentally poking the ICD loader library which subsequently tries to find libMoltenVK.dylib through a JSON ICD file?

[jhvst]

Maybe this will help to clarify the issue:

So, the dylibs are linked and copied on compilation to the Frameworks folder. When ash is called, this Frameworks folder is automatically accessed (its added to the LD path), and as such the dylibs are automatically loaded from this folder.

iOS debug console reports the following:

Metal API Validation Enabled
objc[80232]: Class MVKBlockObserver is implemented in both /private/var/containers/Bundle/Application/ECE67327-672E-41AC-967F-D66B09FBA774/rivi-mobile.app/Frameworks/libMoltenVK.dylib (0x102380ee0) and /private/var/containers/Bundle/Application/ECE67327-672E-41AC-967F-D66B09FBA774/rivi-mobile.app/Frameworks/libvulkan.dylib (0x105cd0ee0). One of the two will be used. Which one is undefined.

If I place the files anywhere else I am greeted with the following:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: LibraryLoadFailure(DlOpen { desc: "dlopen(libvulkan.dylib, 0x0005): tried: \'/usr/lib/system/introspection/libvulkan.dylib\' (no such file), \'/private/var/containers/Bundle/Application/4A47814B-60C0-42AE-85FB-939CEA564131/rivi-mobile.app/Frameworks/libvulkan.dylib\' (no such file), \'/private/var/containers/Bundle/Application/4A47814B-60C0-42AE-85FB-939CEA564131/rivi-mobile.app/Frameworks/libvulkan.dylib\' (no such file), \'libvulkan.dylib\' (no such file), \'/usr/local/lib/libvulkan.dylib\' (no such file), \'/usr/lib/libvulkan.dylib\' (no such file), \'/usr/lib/system/introspection/libvulkan.dylib\' (no such file), \'//libvulkan.dylib\' (no such file)" })', src/lib.rs:15:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
libc++abi: terminating with uncaught foreign exception
dyld4 config: DYLD_LIBRARY_PATH=/usr/lib/system/introspection DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libBacktraceRecording.dylib:/Developer/usr/lib/libMainThreadChecker.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib
terminating with uncaught foreign exception
(lldb)

If I now provide the libvulkan.dylib but not the MoltenVK one, I get the following:

dyld[80318]: Library not loaded: @rpath/libMoltenVK.dylib
  Referenced from: /private/var/containers/Bundle/Application/837532B9-C04A-47BA-BD7F-2865015911B1/rivi-mobile.app/rivi-mobile
  Reason: tried: '/usr/lib/system/introspection/libMoltenVK.dylib' (no such file), '/private/var/containers/Bundle/Application/837532B9-C04A-47BA-BD7F-2865015911B1/rivi-mobile.app/Frameworks/libMoltenVK.dylib' (no such file), '/private/var/containers/Bundle/Application/837532B9-C04A-47BA-BD7F-2865015911B1/rivi-mobile.app/Frameworks/libMoltenVK.dylib' (no such file), '/usr/local/lib/libMoltenVK.dylib' (no such file), '/usr/lib/libMoltenVK.dylib' (no such file)
Library not loaded: @rpath/libMoltenVK.dylib
  Referenced from: /private/var/containers/Bundle/Application/837532B9-C04A-47BA-BD7F-2865015911B1/rivi-mobile.app/rivi-mobile
  Reason: tried: '/usr/lib/system/introspection/libMoltenVK.dylib' (no such file), '/private/var/containers/Bundle/Application/837532B9-C04A-47BA-BD7F-2865015911B1/rivi-mobile.app/Frameworks/libMoltenVK.dylib' (no such file), '/private/var/containers/Bundle/Application/837532B9-C04A-47BA-BD7F-2865015911B1/rivi-mobile.app/Frameworks/libMoltenVK.dylib' (no such file), '/usr/local/lib/libMoltenVK.dylib' (no such file), '/usr/lib/libMoltenVK.dylib' (no such file)
(lldb) 

The source code of the project is here: https://github.com/periferia-labs/rivi-mobile/tree/main/ios (mind the non-sensical function names in Rust and Swift, I just wanted to see if loading ash works).

I'll debug further later on, by changing the ash source code and seeing if I can manage to provide an ICD file somehow on iOS to redirect the load path as I do on macOS. Before that, I thought to provide the error traces if you think the error is not about ash, but about MoltenVK instead.

[MarijnS95]

@jhvst All I can think about here is the Referenced from: explicitly printing your project name and not the "equally-built" libvulkan.dylib. Could this error originate from a dlopen call or is it implying dynamic linking in the executable? Does Mac have tools like readelf to inspect the dynamic linker section? If listed in there, something else could be forcing your application to link dynamically against libMoltenVK.dylib. I don't see how ash/build.rs could do that though.