Shader defined effects using naga oil virtual functions

[ncthbrt]

Describe the solution you'd like

As the title describes, naga oil has functionality to declare virtual functions. It'd be fantastic if this could be leveraged to be able to write custom age, init, update functions, etc instead of using the expression builder. This experience would be much like writing vertex/fragment shaders but on a per particle basis.

Describe why you want that solution. Is this related to a problem?

Hanabi's effect expression builder can be quite complicated and in some cases, a particle effect could be authored more efficiently using plain wgsl. It would still be possible to support both experiences by using virtual functions in the shader builder.

Describe alternatives you've considered

Using the hanabi expression api as it currently exists

Additional context Add any other context or screenshots about the feature request here.

This is a great talk about an extremely flexible and powerful particle system that takes this approach https://youtu.be/qbkb8ap7vts?si=KIMa32aVuG80uxPr

[djeedai]

Thanks @ncthbrt for opening that issue, and for the reference of that GDC talk it's fantastic, I had missed it! This feels so energizing to see all the possible things Housemarque can build with particles.

On the actual feature request now, let me elaborate a bit on the vision for Hanabi and how this fits. If you check the Expression API implementation you'll note that it's under a graph module. However this part of the code (the one under that module but which is not expression related) has never really been used much by anyone, because it's not finished. The goal of the Expression API has never been to allow users to build effects from code; instead, it was to lay the foundation of a (visual) graph editor, where tech artists could build effects with a node graph, similar to all major game engines out there. The rationale is that building a VFX without real-time visual feedback is a dead end in my opinion; no one wants to do that, no one serious about development has time for that. Now, practically though unfortunately Hanabi is blocked by a couple of things, and notably:

1. lack of decently mature UI in Bevy or other to build an editor, and time to do so;
2. limitations in the asset serialization/deserialization system (due in part to the inability to handle trait objects).

As a consequence, for now most VFX are authored via code. I say "most" because I know about at least 1 user created editor that I tried, and it's amazing, but was never released publicly, but at least confirmed my instinct that this was the right path. But authoring by code is not ideal by any measure; this is only a stop gap. I believe once those limitations overcome, the visual editing is the path forward for 99% of the users.

That being said, I appreciate that, in part because Hanabi's modifiers and expressions are not user-extensible (and I'm not sure how much they could be, it's reasonably complex), having an escape hatch down to raw shader code is necessary. So your suggestions 100% fits in that vision I think. I'm not sure exactly what naga_oil's virtual functions are though; my intuition is that we can simply add a ShaderExpression which injects raw WGSL code directly into the generate one, provided you know what you're doing (great power great responsibility). I think that would be enough to unlock a lot of "exotic" effects. As Housemarque themselves note, not having a visual editor (or as a stop gap, expression API) which non-developers can use was a major bottleneck during production, so I'm skeptical about the all-shader-code approach; the more authors write shader code by hand, the harder it is to build an editor for non-devs (and in fact Housemarque seem to have a meta-language with some constraints, which is the only sane way). Or is there anything more to naga_oil's virtual functions than what I described? (I'm not sure what they are nor why they're called "virtual")

[ncthbrt]

Thanks for the detailed reply!

To expand on virtual functions, here is a snippet copied from the documentation:

virtual functions can be declared with the virtual keyword:

    virtual fn point_light(world_position: vec3<f32>) -> vec3<f32> { ... }

virtual functions defined in imported modules can then be overridden using the override keyword:

#import bevy_pbr::lighting as Lighting

override fn Lighting::point_light (world_position: vec3<f32>) -> vec3<f32> {
    let original = Lighting::point_light(world_position);
    let quantized = vec3<u32>(original * 3.0);
    return vec3<f32>(quantized) / 3.0;
}

overrides must either be declared in the top-level shader, or the module containing the override must be imported as an additional_import in a Composer::add_composable_module or Composer::make_naga_module call. using #import to import a module with overrides will not work due to tree-shaking.

My thinking here was that generated/manual code could be in a separate module and override the particular function (such as lifetime, update, or init)

[ncthbrt]

Also we had a good discussion in rendering on the bevy discord about how it might be possible to introspect on wgsl structs to allow for reflection in rust land which could be good for tweaking effect settings without having to drop into rust code.