Rust-GPU support

[thedocruby]

What problem does this solve or what need does it fill?

rust-gpu is an admirable open-source effort to implement a rustc compiler backend for generating SPIR-V intermediate code. Embark Studios is already actively using this project to generate Vulkan shaders from Rust for an in-house project. Seeing that WebGPU also uses SPIR-V as an intermediate, support for WebGPU is a likely eventuality. This would allow a single language to be used for an entire project, with support for compiling to desktop, mobile, and web platforms. And not just any language, but Rust, the king of languages.

In the words of the developers:

Historically in games GPU programming has been done through writing either HLSL, or to a lesser extent GLSL. These are simple programming languages that have evolved along with rendering APIs over the years. However, as game engines have evolved, these languages have failed to provide mechanisms for dealing with large codebases, and have generally stayed behind the curve compared to other programming languages.

In part this is because it's a niche language for a niche market, and in part this has been because the industry as a whole has sunk quite a lot of time and effort into the status quo. While over-all better alternatives to both languages exist, none of them are in a place to replace HLSL or GLSL. Either because they are vendor locked, or because they don't support the traditional graphics pipeline. Examples of this include CUDA and OpenCL. And while attempts have been made to create language in this space, none of them have gained any notable traction in the gamedev community.

Our hope with this project is that we push the industry forward by bringing an existing, low-level, safe, and high performance language to the GPU; namely Rust. And with it come some additional benefits that can't be overlooked: a package/module system that's one of the industry's best, built in safety against race-conditions or out of bounds memory access, a wide range of tools and utilities to improve programmer workflows, and many others!

rust-gpu seems to fit perfectly with Bevy's mission and values. It would be terribly disheartening if Bevy never supports the project.

What solution would you like?

Rust-GPU might be a bit too early in development for Bevy to fully support it, but even if you decide not to implement it anytime soon, It is definitely something you'll want to do at some point. So, any restructuring of the Bevy internals that would need to be done to support the project should be done slowly and early-on, to reduce the amount of major changes needed to add support down the line.

Or, if you feel like rust-gpu support could be easily implemented starting now, without concerns for stability issues, then by all means please do so. I don't really know the structure of Bevy internals atm, so y'all probably are a better judge of that than me

What alternative(s) have you considered?

There is always the option to explicitly NOT support the project, but in my opinion, rust-gpu really looks like it is the perfect fit for the bevy project, and a great addition to the still-being-worked-on graphics and shader system. Of course, I'm not a Bevy developer, so I could be wrong.

Additional context

You may have heard of RLSL, an archived and abandoned project that had a similar mission. I'll have you know that the creator of RLSL is one of the minds behind rust-gpu, and has taken everything he learned from the previous project and applied it to this new incarnation of his original mission.

[alice-i-cecile]

We've been following these efforts actively. What exactly would you envision this looking like? I suspect that this is ultimately a question for wgpu, our upstream rendering dependency.

[bjorn3]

Doesn't wgpu already support SPIR-V as input?

[mockersf]

People have already played rust-gpu in Bevy on platforms that supports SPIR-V by using passthrough in Bevy. Since, Naga frontend for SPIR-V has improved, so maybe it could work on most platform.

But it was somewhat painful to use due to the need of doing two compilation passes or having a build script handling that for you

[thedocruby]

@alice-i-cecile @bjorn3 According to the wgpu README file:

wgpu supports shaders in WGSL, SPIR-V, and GLSL. Both HLSL and GLSL have compilers to target SPIR-V. All of these shader languages can be used with any backend, we will handle all of the conversion. Additionally, support for these shader inputs is not going away.

Since rust-gpu compiles rust code into SPIR-V, it shouldn't be impossible to feed the compiled code into Bevy's existing wgpu dependency. However, like @mockersf pointed out, this is not trivial for most users, and relying on the user to implement this manually sort of breaks up the simplicity of using a single language where everything meshes together.

Now, if all the gobbledygook were implemented in the engine itself, it could enhance the user experience greatly by unifying the graphics programming with everything else under one simple and powerful system.

This seems perfect for the model of "Rust supremacy" that Bevy proudly displays. (Which I totally agree with). Also, it enhances user experience and promotes indie game development by only requiring the developer to learn a single language.

[mockersf]

Bevy consider shaders as assets. Once we have asset preprocessing and baking, shader compilation will just be one of those and shouldn't need anything extra

[bjorn3]

Rust-GPU requires a specific nightly compiler. I don't think we can handle this transparently. At least not when rustup isn't used.

[thedocruby]

@mockersf I am not sure what you mean. There are plenty of cases, especially in 3D games where a developer would need to write custom shaders to achieve his vision. What are you trying to say? I'm worried I've completely misunderstood you.

[thedocruby]

@bjorn3 Notice how I specifically pointed out in the issue post that rust-gpu was likely not in enough of a finalized state for this to be feasible at the moment. It's probably best to wait until the project is on a more stable and polished foundation. Does this address your concerns?

[bjorn3]

Kind of. Yes it is probably best to wait, but I don't think Rust-GPU will ever ship with rustc and as such it will always require nightly rustc due to depending on unstable rustc apis by definition as alternative codegen backend.

[thedocruby]

@bjorn3 Those APIs won't always be unstable, will they? or do you mean unsafe?

Also, you mentioned using rustup. What's wrong with that? (forgive me, I haven't used rust in over a year. I'm a little... low on practice. don't really remember how the compiler system works.)

[bjorn3]

Those APIs won't always be unstable, will they?

They will always be unstable. Alternative codegen backends depend on the internal crates that make up rustc which regularily change. Stabilizing these interfaces would effectively require stopping development of rustc I think. Many changes are necessery either to implement new features or to fix bugs. The only thing that may allow codegen backends to work on stable would be to implement a new interface just for codegen backends which can stay the same even if the rustc internals change. The stable-mir initiative may result in such an interface, or it may be deemed infeasible to use this interface for codegen backends due to abstracting too much away.

[minecrawler]

I don't think Rust-GPU will ever ship with rustc

I'm not so sure about that. Rustc relies on LLVM, and as it happens, they started to upstream SPIR-V as a backend for LLVM 15. So, it may be that rustc at some point supports SPIR-V as a regular target and by extension rust-gpu will become stable.

source: https://www.phoronix.com/news/LLVM-15-SPIR-V-Backend

[x-52]

This is a very promising idea!

The real challenge here is with the build process. Since we're dealing with nightly anyway, we might as well take advantage of metabuild, another unstable feature.

Here's the gist:

Bevy would export a function called metabuild that's effectively just a build.rs script. Users can then add a few more lines to their Cargo.toml, and cargo will run our metabuild function as if it were part of the user's build.rs.

Why use metabuild? Because you can already use rust-gpu with Bevy by writing your own build.rs.

The metabuild function, in turn, will simply find all files that end in, say, .shader.rs and compile them with spirv-builder (part of rust-gpu). While we're at, we might as well put that SPIR-V in assets/ so people can use it from their code.

In case you were wondering, here's what a Bevy app's Cargo.toml would look like if it were using rust-gpu:

cargo-features = ["metabuild"] # Enable metabuild

[package]
name = "mygame"
version = "0.1"
metabuild = ["bevy"] # Use Bevy's metabuild()

[dependencies]
bevy = {version = "0.8", features = ["rust-gpu"]}

[build-dependencies]
bevy = {version = "0.8", features = ["rust-gpu"]} # Bevy is now a build dependency as well

# Not strictly necessary, but it speeds up shader compilation a lot
[profile.release.build-override]
opt-level = 3
codegen-units = 16
[profile.dev.build-override]
opt-level = 3

Bevy already requires messing around with Cargo.toml and .cargo/config a fair bit (for fast compiles), so this shouldn't be too bad if it's properly documented.

I should point out that metabuild and build.rs can't be used together in the same crate, but there's nothing stopping people from calling bevy::metabuild() from their own build.rs. Also, there hasn't been any activity on the metabuild tracking issue for a full year, so I have no idea what its current status is.

[x-52]

So, it may be that rustc at some point supports SPIR-V as a regular target and by extension rust-gpu will become stable.

There is precedent for this:nvptx64-nvidia-cuda became a built-in target in rustc in 2016. It's exactly what it sounds like: CUDA on Nvidia GPUs. So if you don't care about compatibility with people like me (who don't have Nvidia GPUs), then you could write GPU code with “vanilla” rustc on stable right now.

[thedocruby]

@x-52 That does sound promising! Maybe we could set up a sort of experimental branch of bevy that requires nightly rustc, instead of adding rust-gpu support to the main codebase. Then, it would be there for those who are willing to use it!

[thedocruby]

Also, on a slightly unrelated note, I was wondering how I would go about using SPIR-V with Bevy's wgpu, after I manually use rust-gpu to compile to SPIR-V. I saw a tutorial for using GLSL with Bevy, is it a similar process?

[x-52]

@thedocruby It is very similar.

If your SPIR-V is stored in a file, then you can use it just like any other WGSL or GLSL shader.

On the other hand, If you're using the aforementioned spirv-builder, which embeds the shaders in your Rust code, you can use bevy::render::render_resource::Shader::from_spirv. Once you have a Shader, you can get a Handle<Shader> with bevy::asset::Assets::add, then turn that handle into a bevy::render::render_resource::ShaderRef with .into().

[thedocruby]

Also, It may turn out that implementing rust-gpu through wgpu is not feasible or ill-advised for one reason or another. In that case, to implement support in bevy may require a separate graphics pipeline altogether. However, this may be an option down the road. Take this quote by cart from #639

I agree that long term it probably makes sense to have multiple render pipelines, but short term I would much rather maintain a single pipeline that runs decently everywhere while still looking good when you need it to (ex: google's Filament).

Once the renderer, the platform impls, and the rest of the engine have stabilized a bit, then we can consider splitting pipelines in the interest of optimization and/or render features.

So, Once bevy gets to a more stable place in development, there might be enough incentive and opportunity to implement support for rust-gpu, not to mention, rust-gpu will be much more developed by then as well.

[x-52]

In that case, to implement support in bevy may require a separate graphics pipeline altogether.

rust-gpu isn't a separate graphics pipeline. It's just another shading language, just like GLSL, WGSL, Cg, or HLSL. AFAIK, it wouldn't require any real architectural changes to Bevy, either. In fact, you can already use rust-gpu with Bevy (if you set up the whole build pipeline yourself).

[mockersf]

There isn't anything to add in Bevy for rust-gpu support, everything should already work. It's mostly that no one bothered yet to show how to do it because building for it is not friendly. Again, Bevy doesn't care where the shader comes from, and it can be in wgsl, glsl, spirv, ... it should work the same.

@thedocruby you would be more than welcome to do an example or using the form you prefer on how this would currently work and what could be improved, but as for "rust-gpu support", it's already supported 👍

[cart]

As others have mentioned, we'd like to support rust-gpu shaders in Bevy. I'm happy to make whatever changes are required to make that happen so that people that prefer that workflow can use it. Its possible no changes are required at all!

We've considered rust-gpu in the past for Bevy's "recommended shader language" (ex: what we use for built in bevy crates, what we document / support, etc). However I'm personally largely against that for a number of reasons:

1. It requires a nightly Rust compiler and "additional local setup". Bevy should be "plug and play" on stable rust.
2. It requires packaging rustc with your game if you want to compile shaders on the fly in a "deployed" game. This is a scenario we want to support, as there are potentially millions of permutations of a given shader (see this article about Unity shader variants). Of course, we still want to precompile shaders when possible, but being able to compile arbitrary shaders at runtime is "very nice".

[Shfty]

Edit: Cleaned up my thoughts into an issue (#7771) and pull request (#7772).

A few points I've noted in recent rust-gpu 0.5 + bevy 0.9.1 experimentation:

rust-gpu ships a spirv-builder crate that can be invoked from build.rs for easy automation, which can be segmented out into its own empty library to give it a buildable cargo target, then hot-reloaded on compile via AssetServer.

Code sharing between Bevy's WGSL implementation and Rust shaders is - naturally - nonexistent, so if you want to leverage the lighting system or PBR in a Rust shader, you have to implement from scratch.

That's quite doable (see below), but raises the question of how viable using rust-gpu with bevy would be in the long term (i.e. if-and-when-rust-gpu-is-stable) for someone who also wants to use the PBR support code. It's reasonable for Rust not to be a first-class language given the context, but having to keep your own Rust version of bevy_pbr up to date if you want lights is quite an intimidating barrier to entry. Though I suppose that can be solved for by a third-party bevy_pbr_rust crate tracking behind the mainline WGSL implementation if it becomes that popular. Anyway, no need to bikeshed speculative futures to death :sweat_smile:

For other users interested in using rust-gpu, I've put together an example workspace for compiling Rust into SPIR-V, then rendering it in Bevy. It depends on a custom fork of 0.9.1 to prevent a loading bug with ShaderProcessor, but has a functional reimplementation of bevy_pbr in its shader crate as a proof-of-concept.

[Shfty]

Some further thoughts re. @cart's commentary on runtime shader compilation:

In theory baking rustc into a game is feasible, since it can be imported as a crate and driven programatically, though as far as i know is in a "never stable" state where you have to pick a nightly version and deal with possible API breakage.

I suppose that's less an issue if rust-gpu also requires it, but there are probably more issues to hack through w.r.t. making it work this way since all of its use case doco is cargo-focused. Needs investigating, but would doubtless end up being project-domain instead of engine-domain one way or the other due to touching nightly.

On the flipside, there's something to be said for compiling statically in cases where it's a good fit. Recent high-profile PC releases have taken a lot of flak for stutter introduced by on-demand shader compilation, some of which introduced a precompilation stage to frontload the compile time as a workaround. Arguably that's not an ideal fix, since the user has to wait on it.

I expect there are certain cases where on-site compilation is a boon (vendor or card -specific extensions and optimizations that need to be conditional over the user's hardware, for example), but I think more common-or-garden cases could dodge the problem with appropriately-designed static compilation machinery.

Ex. By enumerating the shader variants required by a given game at edit-time, using that data to precompile a single SPIR-V module with appropriately-named entrypoints, which are then looked up during game runtime based on a hash of bevy's shader defs.

I'll hack on it and see if I can come up with an example.

[Shfty]

I return bearing workflow!

As expected, trying to brute-force permutate a complex shader (i.e. one with more than 8-or-so binary compile-time parameters) is non-viable; not only does it take an extremely long time, it also crashes the SPIR-V codegen backend with an ID out-of-range error.

However, hot-recompiling requested entry points on demand has proven quite viable:

To wit, my example workspace has evolved into Bevy Rust-GPU; a suite of crates designed to support the workflow pictured above.

It remains subject to the issue linked above, as well as lack of read-only storage buffer support, but is otherwise quite usable, and able to support on-demand permutation of shaders as complex as bevy_pbr.

[Shfty]

The Bevy Rust-GPU suite has been upgraded to bevy 0.10.

[nyabinary]

Any updates on this?