search

hexops / mach

zig game engine & graphics toolkit
https://machengine.org
Other
3.37k stars 160 forks source link

make static Dawn library easy to consume in other projects #109

Closed slimsag closed 2 years ago

slimsag commented 2 years ago

Spawned from Twitter convo https://twitter.com/rsms/status/1465073396861992961

We are already doing a fair amount of work to enable effortless compilation (and cross-compilation) of Google's Dawn WebGPU implementation, it makes sense to have this exist in a standalone project in which anyone interested in building static Dawn without all the hassle of Google tools, dependencies, etc. can consume.

This should also include the C shims needed to access the portions of dawn_native's API needed to e.g. bind Dawn to a GLFW window and similar from C.

cc @rsms

slimsag commented 2 years ago

@rsms a few questions for you:

(1) Preference for single big libdawn.a static lib, or one for each component? Right now we build one for each component:

ls zig-out/lib
libabseil-cpp.a     libdawn-native.a    libdawn-utils.a     libglfw.a       libspirv-tools.a
libdawn-native-mach.a   libdawn-platform.a  libdawn-wire.a      libgpu.a        libtint.a

(2) Clone the repo and build from source (would require zig and git), or download some prebuilt binary release?

(3) You mentioned on Twitter noticing Dawn's binary size being quite huge, I also noticed that - it was one of the reasons I switched to per-component static libs (was looking to debug it), right now we have:

% du -sh zig-out/lib/*
 18M    zig-out/lib/libabseil-cpp.a
2.4M    zig-out/lib/libdawn-native-mach.a
 66M    zig-out/lib/libdawn-native.a
940K    zig-out/lib/libdawn-platform.a
2.4M    zig-out/lib/libdawn-utils.a
 26M    zig-out/lib/libdawn-wire.a
1.6M    zig-out/lib/libglfw.a
903M    zig-out/lib/libgpu.a (all linked together)
539M    zig-out/lib/libspirv-tools.a
247M    zig-out/lib/libtint.a

Of course an actual triangle example binary linked against them is much smaller:

% du -sh zig-out/bin/dawn-example 
 20M    zig-out/bin/dawn-example

But curious if you have any thoughts/concerns around this

rsms commented 2 years ago

Thank you Stephen for making this effort!

(1) Preference for single big libdawn.a static lib, or one for each component? One for each component seems like a better approach. In this case it would be useful to include a text file documenting the dependencies (e.g. libdawn-native.a requires libspirv-tools.a) Separate lib files also makes distribution more streamlined.

(2) Build from source or download prebuilt binary release? Build from source seems like the most pragmatic solution. Zig being so readily available via binary dist itself makes it a small ask from users. Might be best to start that way and then consider adding github actions to produce release binaries later on if there's demand.

(3) You mentioned on Twitter noticing Dawn's binary size being quite huge Indeed quite thicc. It's from all the C++ and the fact that no stripping or DCE-esque optimizations can be done on generic object code. However I've got it in my todo list to experiment with -fvisibility and a linker version script to explicitly define public symbols which should in theory allow the compiler to spit out smaller object files.

Would you consider separating the mach code from the "static dawn lib" project? I.e. making mach a "consumer"/"user" of the dawn libraries?

slimsag commented 2 years ago

However I've got it in my todo list to experiment with -fvisibility and a linker version script to explicitly define public symbols which should in theory allow the compiler to spit out smaller object files.

Whoa that's a cool idea!

Would you consider separating the mach code from the "static dawn lib" project? I.e. making mach a "consumer"/"user" of the dawn libraries?

Absolutely, yes, that's the primary thing this issue is for. A few thoughts:

For Mach itself there are a lot of good reasons to prefer monorepo development, so here's how I envision this working in a way that is compatible with both your and my goals:

meshula commented 2 years ago

The dawn_utils part of the dawn library has various glfw interoperation bits in it.

Is dawn_utils part of the effort? If so it seems like it would be a good idea to have it be a third library, so that there's no hard dependency from dawn to glfw. dawn without glfw would be good...

slimsag commented 2 years ago

@meshula Yeah, I'd like for you to be able to choose (via build.zig, and CLI parameters) which of the components to build. You may still pay the cost of downloading GLFW (not sure), but it wouldn't build.

I can't say how easy Dawn is to use without dawn_utils, of course - it also implements the swap chain logic bindings to Metal/DirectX/Vulkan which you would likely want even if not using GLFW - that'd be an issue to file upstream on dawn.googlesource.com

slimsag commented 2 years ago

We've gotten a lot better here:

There's still more to do, documentation to add, etc. but overall you can consume this from other projects now without much trouble I believe.

I'll close this issue, feel free to create new ones with any feedback / issues, etc.

slimsag commented 2 years ago

Latest binary release: https://github.com/hexops/mach-gpu-dawn/releases/tag/release-6b59025

image