search

gfx-rs / wgpu-native

Native WebGPU implementation based on wgpu-core
Apache License 2.0
850 stars 98 forks source link

Broken MacOS distribution #329

Closed lisyarus closed 9 months ago

lisyarus commented 9 months ago

In the latest release (v0.18.1.3) the macos-aarch64 dynamic binaries (libwgpu_native.dylib) fail to load when linking a program to them, stating 

dyld[63381]: Library not loaded: /Users/runner/work/wgpu-native/wgpu-native/target/aarch64-apple-darwin/release/deps/libwgpu_native.dylib
  Referenced from: <C5F30B84-AB4C-368B-A82E-E3BA70A07556> /Users/lisyarus/webgpu-demo/cmake-build-debug/webgpu-demo
  Reason: tried: '/Users/runner/work/wgpu-native/wgpu-native/target/aarch64-apple-darwin/release/deps/libwgpu_native.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/wgpu-native/wgpu-native/target/aarch64-apple-darwin/release/deps/libwgpu_native.dylib' (no such file), '/Users/runner/work/wgpu-native/wgpu-native/target/aarch64-apple-darwin/release/deps/libwgpu_native.dylib' (no such file), '/usr/local/lib/libwgpu_native.dylib' (no such file), '/usr/lib/libwgpu_native.dylib' (no such file, not in dyld cache)

As you can see, there is a very suspicious path /Users/runner/work/wgpu-native/wgpu-native/target/aarch64-apple-darwin/release/deps/libwgpu_native.dylib in there. (Needless to say, I don't have a runner user on my mac.)

If we check the dependencies of libwgpu_native.dylib, we get

lisyarus@i111354133 ~/webgpu-demo/cmake-build-debug $ otool -L ../wgpu-native/libwgpu_native.dylib
../wgpu-native/libwgpu_native.dylib:
    /Users/runner/work/wgpu-native/wgpu-native/target/aarch64-apple-darwin/release/deps/libwgpu_native.dylib (compatibility version 0.0.0, current version 0.0.0)
    /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)
    /System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 306.3.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
    /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1690.3.3)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.255.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)

Everything here seems fine, except for the first entry, which coincides with the error above. I guess there's an error in the packaging process somewhere?

Update: I checked a few other releases, and it seems that all releases are affected by this, since the very first one (v0.5.2).

rajveermalviya commented 9 months ago

it seems that all releases are affected by this

Oof, seems like it has been the case for far too long! Thanks for filing!

Looks like we need to set an install_name for the dylibs that we provide. It can be done via rpath = true in Cargo.toml or RUSTFLAGS="-C link-args=-Wl,-install_name,@rpath/libwgpu_native.dylib" at build time. rpath = true may change configs on other platforms as well so I think we should avoid that.

@almarklein I am not sure if python would break if we add the install_name, IIRC it does dlopen so it shouldn't really be affected. I have only been using static libs, so that's why didn't hit this earlier either.

BTW @lisyarus you can update current install_name using the following command:

install_name_tool -id @rpath/libwgpu_native.dylib libwgpu_native.dylib

Can you give it a try and report back, if it indeed does fix the issue for you?

lisyarus commented 9 months ago

@rajveermalviya Thanks for a quick response! Yes, this does solve the problem.

After running install_name_tool -id @rpath/libwgpu_native.dylib libwgpu_native.dylib otool produces

lisyarus@i111354133 ~/webgpu-demo/wgpu-native $ otool -L libwgpu_native.dylib
libwgpu_native.dylib:
    @rpath/libwgpu_native.dylib (compatibility version 0.0.0, current version 0.0.0)
    /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)
    /System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 306.3.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
    /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1690.3.3)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.255.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)

and the executable linking it runs with no problems.

almarklein commented 9 months ago

Thanks for bringing this to our attention!

In wgpu-py we statically bind to the library (using cffi) which is why we have not seen it either. And which is also why its most probably safe (from our end) to change the install_name 👍

rajveermalviya commented 9 months ago

New archives ~will be soon~ are now available in https://github.com/gfx-rs/wgpu-native/releases/tag/v0.18.1.4

lisyarus commented 9 months ago

@rajveermalviya @almarklein Awesome, thank you!