search

KhronosGroup / MoltenVK

MoltenVK is a Vulkan Portability implementation. It layers a subset of the high-performance, industry-standard Vulkan graphics and compute API over Apple's Metal graphics framework, enabling Vulkan applications to run on macOS, iOS and tvOS.
Apache License 2.0
4.64k stars 402 forks source link

Invalid driver dylib generated since #2170 #2186

Closed spnda closed 3 months ago

spnda commented 3 months ago

Checking out 9082ca839f4eb83d183837fe656059ac12e4ee03, which is the last commit before the merge of #2170, and building the dylib with make macos my application runs just as it used to.

Now if I checkout eff757144614a20d7b452c8388019783d9511a25 (from #2170), build the driver with make macos I get this output:

install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/sean/Documents/git/spnda/MoltenVK/Package/Release/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib (for architecture x86_64)
install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/sean/Documents/git/spnda/MoltenVK/Package/Release/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib (for architecture arm64)
...
note: Run script build phase 'Package MoltenVK' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'MoltenVK-macOS' from project 'MoltenVKPackaging')

After changing the symlink to the new dylib location the application crashes with an EXC_BAD_ACCESS while loading the dylib file and this reason:

Termination Reason:    Namespace CODESIGNING, Code 2 Invalid Page

If I update past #2175, build it and run my application, instead of an application crash I now get this output (running with VK_LOADER_DEBUG=error,warn,info:

ERROR:             loader_scanned_icd_add: Attempt to retrieve either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD /usr/local/lib/libMoltenVK.dylib failed.
ERROR:             loader_scanned_icd_add: Attempt to retrieve either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD /usr/local/lib/libMoltenVK.dylib failed.
ERROR:             loader_scanned_icd_add: Attempt to retrieve either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD /usr/local/lib/libMoltenVK.dylib failed.
ERROR:             loader_scanned_icd_add: Attempt to retrieve either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD /usr/local/lib/libMoltenVK.dylib failed.
ERROR:             loader_scanned_icd_add: Attempt to retrieve either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD /usr/local/lib/libMoltenVK.dylib failed.
INFO:              Portability enumeration bit was set, enumerating portability drivers.
ERROR:             loader_scanned_icd_add: Attempt to retrieve either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD /usr/local/lib/libMoltenVK.dylib failed.
ERROR | DRIVER:    vkCreateInstance: Found no drivers!

The rest of my setup did not change and switching to any of the commits always gives the same exact result. I also can't find the section of the documentation anymore that talked about symlinking the dylib file from /usr/local/lib to the build output directly when developing, so did that perhaps change?

SRSaunders commented 3 months ago

I have seen similar issues as well. I see the following when building for macOS with Xcode 15.2:

  1. There is no longer a dylib located at MoltenVK/Package/Release/MoltenVK/dylib/macOS/libMoltenVK.dylib. The dylib now appears at MoltenVK/Package/Release/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib. This is unfortunate, as it breaks compatiblity with Vulkan SDK structure and thus application cmake scripts also break when doing development with local MoltenVK builds, i.e. you can no longer just change the VULKAN_SDK env variable to point at your local MVK build. You have to either change your cmake script, or manually create a symlink at MoltenVK/Package/Release/MoltenVK/dylib.
  2. Clean builds (i.e. clean then build) of MoltenVK produce a correct dylib at the new location, but incremental builds (i.e. make a change then build) now break, resulting in an incomplete and much smaller dylib which cannot load. I think this is the same issue as reported above. To work around this I have being doing a clean then build every time.
spnda commented 3 months ago

Hmm, interesting find about the size of the file, which I can confirm. I just checked out the latest again and compiled it and got a 86KB dylib instead of the 15.2MB one. And I can confirm that doing make clean && make macos then produces the 15.2MB dylib file again. Btw I am on Xcode 15.3.

richard-lunarg commented 3 months ago

This is unfortunate, as it breaks compatiblity with Vulkan SDK structure and thus application cmake scripts also break when doing development with local MoltenVK builds, i.e. you can no longer just change the VULKAN_SDK env variable to point at your local MVK build. You have to either change your cmake script, or manually create a symlink at MoltenVK/Package/Release/MoltenVK/dylib.

FWIW, the next SDK is using v1.2.8 (which will be out next week). Vulkan-Tools cmake files have also been updated to reflect the new location of the .dylib.

SRSaunders commented 3 months ago

Hi @richard-lunarg, will cmake’s find_package(Vulkan OPTIONAL_COMPONENTS MoltenVK) (using the FindVulkan.cmake module) continue to work with this change, assuming you first do source setup-env.sh inside the Vulkan SDK directory? Many cross platform apps depend on this working.

Or put another way, will the Vulkan SDK layout change in any way as a result of this MoltenVK change? Hopefully not...

richard-lunarg commented 3 months ago

libMoltenVK.dylib will be in the same place in the SDK for quite some time moving forward. It's just moved in the MoltenVK build artifacts.

billhollings commented 3 months ago

install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/sean/Documents/git/spnda/MoltenVK/Package/Release/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib (for architecture x86_64)

and

I just checked out the latest again and compiled it and got a 86KB dylib instead of the 15.2MB one.

These should be fixed in PR #2175. Try running from the SDK release v.1.2.8 tag.

SRSaunders commented 3 months ago
  1. Checkout 1d98babb721c03667f65e7f6197ca2a356d19198
  2. ./fetchDependencies --macos
  3. Launch MoltenVKPackaging.xcodeproj (Xcode 15.2)
  4. Build --> successful 15.2 MB build at MoltenVK/Package/Release/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib
  5. Change something trivial (e.g. #define MVK_HIDE_VULKAN_SYMBOLS 1)
  6. Build --> build completes, but generates a non-functional dylib of 101 KB in size.
  7. Build clean
  8. Build --> successful 15.2 MB build

I'm not sure, but perhaps incremental builds are not linking in some missing pieces. Kind of annoying since you have to build clean and rebuild with every change to get a functional dylib.

spnda commented 3 months ago

install_name_tool: warning: changes being made to the file will invalidate the code signature in: /Users/sean/Documents/git/spnda/MoltenVK/Package/Release/MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib (for architecture x86_64)

and

I just checked out the latest again and compiled it and got a 86KB dylib instead of the 15.2MB one.

These should be fixed in PR #2175. Try running from the SDK release v.1.2.8 tag.

As stated in the original comment, I tested past #2175, which fixed the first issue, but was the one which brought up the issue with the tiny binary which contains no Vulkan symbols. So, no, that PR did not fix the second issue.

billhollings commented 3 months ago

PR #2196 should fix this. Please retest with latest, and close this issue if the problem has been fixed.