Error building with latest dawn

[stackotter]

Hi Henry,

I am currently working on setting up a homebrew formula for dawn to allow people to more easily try out swift-webgpu (because building from source can take over 45 minutes on my laptop).

It seems like swift-webgpu might not quite be compiling with the latest dawn though. Here's the error I get:

➜  swift-webgpu git:(main) swift build -c release
Building for production...
In file included from /Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/CDawnNative/dawn_native.cpp:4:
/usr/local/include/dawn/native/DawnNative.h:25:15: warning: nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]
namespace dawn::platform {
              ^~~~~~~~~~
               { namespace platform
warning: nested namespace definition is a C++17 extension; define each namespace separately [-Wc++17-extensions]
namespace dawn::native {
              ^~~~~~~~
               { namespace native
2 warnings generated.
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:841:48: error: missing arguments for parameters 'timestampWriteCount', 'timestampWrites' in call
        var cStruct = WGPUComputePassDescriptor(
                                               ^
CWebGPU.WGPUComputePassDescriptor:3:12: note: 'init(nextInChain:label:timestampWriteCount:timestampWrites:)' declared here
    public init(nextInChain: UnsafePointer<WGPUChainedStruct>!, label: UnsafePointer<CChar>!, timestampWriteCount: UInt32, timestampWrites: UnsafePointer<WGPUComputePassTimestampWrite>!)
           ^
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1586:51: error: missing argument for parameter 'clearValue' in call
            clearColor: cStruct_clearColor.pointee
                                                  ^
                                                  , clearValue: <#WGPUColor#>
CWebGPU.WGPURenderPassColorAttachment:3:12: note: 'init(view:resolveTarget:loadOp:storeOp:clearColor:clearValue:)' declared here
    public init(view: WGPUTextureView!, resolveTarget: WGPUTextureView!, loadOp: WGPULoadOp, storeOp: WGPUStoreOp, clearColor: WGPUColor, clearValue: WGPUColor)
           ^
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1635:59: error: missing arguments for parameters 'depthClearValue', 'stencilClearValue' in call
        var cStruct = WGPURenderPassDepthStencilAttachment(
                                                          ^
CWebGPU.WGPURenderPassDepthStencilAttachment:3:12: note: 'init(view:depthLoadOp:depthStoreOp:clearDepth:depthClearValue:depthReadOnly:stencilLoadOp:stencilStoreOp:clearStencil:stencilClearValue:stencilReadOnly:)' declared here
    public init(view: WGPUTextureView!, depthLoadOp: WGPULoadOp, depthStoreOp: WGPUStoreOp, clearDepth: Float, depthClearValue: Float, depthReadOnly: Bool, stencilLoadOp: WGPULoadOp, stencilStoreOp: WGPUStoreOp, clearStencil: UInt32, stencilClearValue: UInt32, stencilReadOnly: Bool)
           ^
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1634:16: warning: no calls to throwing functions occur within 'try' expression
        return try self.view.withUnsafeHandle { handle_view in
               ^
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1689:47: error: missing arguments for parameters 'timestampWriteCount', 'timestampWrites' in call
        var cStruct = WGPURenderPassDescriptor(
                                              ^
CWebGPU.WGPURenderPassDescriptor:3:12: note: 'init(nextInChain:label:colorAttachmentCount:colorAttachments:depthStencilAttachment:occlusionQuerySet:timestampWriteCount:timestampWrites:)' declared here
    public init(nextInChain: UnsafePointer<WGPUChainedStruct>!, label: UnsafePointer<CChar>!, colorAttachmentCount: UInt32, colorAttachments: UnsafePointer<WGPURenderPassColorAttachment>!, depthStencilAttachment: UnsafePointer<WGPURenderPassDepthStencilAttachment>!, occlusionQuerySet: WGPUQuerySet!, timestampWriteCount: UInt32, timestampWrites: UnsafePointer<WGPURenderPassTimestampWrite>!)

And then if I regenerate the code from the current dawn.json file I get different errors:

Building for production...
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1601:137: error: consecutive statements on a line must be separated by ';'
    public init(view: TextureView? = nil, resolveTarget: TextureView? = nil, loadOp: LoadOp, storeOp: StoreOp, clearColor: Color = { NAN, NAN, NAN, NAN }, clearValue: Color) {
                                                                                                                                        ^
                                                                                                                                        ;
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1601:137: error: expected expression
    public init(view: TextureView? = nil, resolveTarget: TextureView? = nil, loadOp: LoadOp, storeOp: StoreOp, clearColor: Color = { NAN, NAN, NAN, NAN }, clearValue: Color) {
                                                                                                                                        ^
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1601:134: error: cannot find 'NAN' in scope
    public init(view: TextureView? = nil, resolveTarget: TextureView? = nil, loadOp: LoadOp, storeOp: StoreOp, clearColor: Color = { NAN, NAN, NAN, NAN }, clearValue: Color) {
                                                                                                                                     ^~~
/Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/Sources/WebGPU/Structs.swift:1656:126: error: cannot find 'NAN' in scope
    public init(view: TextureView, depthLoadOp: LoadOp = .undefined, depthStoreOp: StoreOp = .undefined, clearDepth: Float = NAN, depthClearValue: Float = 0, depthReadOnly: Bool = false, stencilLoadOp: LoadOp = .undefined, stencilStoreOp: StoreOp = .undefined, clearStencil: UInt32 = 0, stencilClearValue: UInt32 = 0, stencilReadOnly: Bool = false) {
                                                                                                                             ^~~
[1/7] Compiling WebGPU Aliases.swift

The dawn commit I am using is ed84058.

Do you get the same errors when you try to build swift-webgpu with the latest dawn commit? (there have been two newer commits but they're just dep rolls and the errors I'm getting seem to be more about breaking changes in dawn)

[stackotter]

Here's the repo with the script I used to build dawn. And here's the dawn homebrew tap I made which seems to be working. But swift-webgpu doesn't build with the tap installed or with the current way from the readme (same error both ways).

[henrybetts]

Hey,

Sounds good! A homebrew formula will definitely help make it more accessible. You can probably disable a few things like swiftshader, vulkan, angle/opengl to speed up the dawn build, if you know that you will be targeting metal.

Looks like they added a couple of new concepts to dawn.json, which I've implemented now in the swift-webgpu generator. If you pull the latest, you should find that it works with that dawn commit.

It's probably worth me documenting which dawn commit the swift files are generated from. Eventually, I would like to remove the generated swift files from this repo and have them generated as a custom build step. I believe this will be possible in an upcoming release of the swift package manager.

[stackotter]

Would you be able to point me in the right direction for how to disable the unnecessary components?

Thanks for fixing that!

I've just tried building with the latest commit and I'm getting a lot of strange errors. I've tried troubleshooting for quite a while and I'm only managing to create even weirder errors. The one time I managed to get it to compile, it gave me this error when running DemoTriangle:

dyld[5006]: Symbol not found: _wgpuAdapterEnumerateFeatures
  Referenced from: /Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/.build/x86_64-apple-macosx/release/DemoTriangle
  Expected in: /usr/lib/libSystem.B.dylib
[1]    5006 abort      .build/x86_64-apple-macosx/release/DemoTriangle

Here's what happens if I do it clean without trying to fix anything (I have attached the 896 lines of errors in a text file) output.txt.

The command I ran was:

swift build -c release \
-Xcc -I/path/to/dawn/src/include \
-Xcc -I/path/to/dawn/out/Release_x64/gen/include \
-Xlinker -L/path/to/dawn/out/Release_x64 \
-Xlinker -rpath -Xlinker /path/to/dawn/out/Release_x64

At the moment I'm just trying without using the version installed with homebrew, because I don't really have a chance of getting it to compile with those if it won't compile the normal way.

[henrybetts]

You can run gn args out/Release --list in the dawn directory and this should list all of the available options that you can add to your args.gn file. The most relevant ones should start with "dawn_". I currently use the following for development:

is_debug=false
dawn_use_angle=false
dawn_use_swiftshader=false

I suppose if you did want to target a Mac that doesn't support metal then you may want to leave them enabled, but I can't imagine that is very common these days?

Ah, they did reorganise the dawn folder structure recently and I need to update the readme. The first include path should now be /path/to/dawn/include instead of /path/to/dawn/src/include. That might explain those errors.

You may also run into an issue with the dawn libraries having incorrect install names. I'll add a note to the readme about that as well. Homebrew will likely fix these for you automatically though.

[stackotter]

Ok, thanks for the tips. I'll try changing that include path when I get back to my laptop. And I'll also see how's he quicker builds are with those settings disabled. The one time I did get it to compile (dunno how), the rpath for dawn_native was libnative.dylib and so naturally it couldn't find the dylib. Is that what you mean by install issues?

[stackotter]

And with the install names issue I tried using install_name_tool to correct it, but it couldn't find libdawn_native.dylib either even though it was in /usr/local/lib which is on the default search path according to the ld man page

[stackotter]

Just tried building again, and with the new include path it built. However now I think I'm running into the install name error you were talking about.

Here's the error I get when I run it:

dyld[11967]: Symbol not found: _wgpuAdapterEnumerateFeatures
  Referenced from: /Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/.build/x86_64-apple-macosx/release/DemoTriangle
  Expected in: /usr/lib/libSystem.B.dylib
[1]    11967 abort      ./DemoTriangle

I tried fixing the path for dawn_native with install_name_tool (otool -L showed me that it's trying to link against libnative.dylib and somehow doesn't throw an error about not finding the dylib).

install_name_tool -change @rpath/libnative.dylib /path/to/dawn/out/Release/libdawn_native.dylib DemoTriangle

However, I still get the same error when running. For some reason Swift seems to be telling the linker that all the dawn_native symbols are in libSystem.B.dylib? I'm on Swift 5.6 by the way, here's the exact version: swift-driver version: 1.44.2 Apple Swift version 5.6 (swiftlang-5.6.0.320.8 clang-1316.0.18.8). I'm on Xcode 13.3 beta 1.

[stackotter]

I just tried the new build instructions (including how to fix the linking issue) and it's finally running again, woohoo! Now I just need to figure out how to get it to link against the dawn installed by homebrew. It keeps giving me errors about not being able to find the library for -ldawn_native even though when I run ld manually it finds it correctly. Maybe Swift doesn't link to things from the system unless explicitly mentioned in package.swift or a module map?

Anyway, that's not part of this issue anymore, and I don't think it's an issue with swift-webgpu/dawn anymore (just the way swift does linking), so I'll close this issue.

Thanks for all the help :)

[stackotter]

Yep, guessed it:

Library search paths:
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/lib/swift
    /usr/local/Cellar/glfw/3.3.6/lib
    /Users/stackotter/Desktop/Projects/Swift/Experiments/DawnBuild/swift-webgpu/.build/x86_64-apple-macosx/release
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/lib

It only includes search paths for tools that it was explicitly told to link to, probably smart for portability of swift to do that. Hopefully I get some time this week to get the brew dawn working properly with swift-webgpu

[henrybetts]

Ah ok, brew can be added as a provider for dawn in the Package.swift, as is the case with glfw, which should work?

You should also be able to add the path manually with a linker flag - something like -Xlinker -L/usr/local/Cellar/dawn/lib.

Out of interest, how did you get that list of search paths?

[stackotter]

I think I've figured out what needs to be changed, my homebrew dawn formula needs to create a pkg config file so that swift knows where to find it (i think that's how it finds glfw, the provider thing is just to tell users what to do when it's missing from memory?). I can't try that right now though cause I don't have my laptop.

I got the linker search paths by add -Xlinker -v to the build command.

Yeah, it works when I add it to the search paths manually with the flag, but I don't want users to have to do that because then building from Xcode is cumbersome and everyone using derivative packages also has to add the flags.

[stackotter]

Hi again Henry, I've been juggling quite a few different open-source projects recently, so swift-webgpu has fallen down my priorities list again.

Feel free to try getting the homebrew package working though, I'd be happy to explain what I've done so far and what the current issues are. Just thought I should let you know that it'll probably be a while before I get around to working on this project again.