Add GDExtension support for custom Shaders/ShaderMaterials

paddy-exe commented 2 years ago

Describe the project you are working on

I have been researching Real-Time Fluid Simulation Technologies to use for Game Development. There are quite a lot of different approaches. Some types that are well known:

Smoothed particle hydrodynamics (SPH) [Particle-based]
- doesn't generate a surface but gets particle positions and density
- Rendering via marching cubes
- Expensive on GPU
Voxel-based Volumes
- Draw every point of data in a 3D cube and trace the path of light through the volume to determine its color
- quite slow
- rendered via Raytracing
Grid-based Simulations based on Navier Stokes equations
- can run in real-time with reasonable grid sizes
- 2D and 3D adaptable
- able to run on standard PC

I would like to create a GDExtension to use the grid-based approach of fluid sims. The advantage is that the 2D sims can be "faked" to be 3D with Raymarching by using a screen-aligned quad. Since this approach needs an algorithm to solve the equations, I would like to create a custom Shader type and ShaderMaterial to optimize the rendering of said technique.

As an example on how this could benefit Visual Effects in Godot, here's a video of the Fluid Ninja Live plugin, made for Unreal Engine by Andras Ketzer, with whom I was also in touch to understand the techniques. It's loosely explained in this pdf in 1.3. This approach would make it possible to create various kinds of fluids such as different types of water (waterfall, whirlpool, lake), mud, fog etc. in a local area. This video by Ubisoft La Forge explains the technical approach a little deeper but also great visualisation of the implementation

Describe the problem or limitation you are having in your project

It is currently not possible to create a custom shader and ShaderMaterial without customising Godot itself. I heard from Calinou at least in RocketChat that it's currently not possible to do this via GDExtension. I also doubt very much that it can be done with an external module since the types are hardcoded inside the rendering code.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Having a the ability to create custom ShaderMaterials would make it possible to strip boilerplate code for the linear equation solver and other essential functions that are used for the simulations. Furthermore it would make it possible to optimise the rendering code in c++ via Compute Shaders (if supported in GDExtension -> otherwise topic for a different proposal)

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

There could be a dummy class like CustomShader and CustomShaderMaterial to signal the Godot core that a new Shader type is part of the library placed in the project's folder structure. If so, the a new entry in the shader creation dialog will be added. I am not sure how an integration in the rendering side of things would like

If this enhancement will not be used often, can it be worked around with a few lines of script?

It definitely can't since the rendering code of the Shader and ShaderMaterial creation would need to be reworked/adapted. As mentioned above, I am also not sure how this could be even done with a Custom Module.

Is there a reason why this should be core and not an add-on in the asset library?

N/A

clayjohn commented 2 years ago

I don't understand why you would need a custom shader type for this. Can't you just use a Shader material with a Shader resource?

paddy-exe commented 2 years ago

@clayjohn I probably can but that would mean a LOT of boilerplate code every time I want to create this type of effect. That is my main concern. My second reason is that I would like to offer several ways to calculate the Navier Stokes equations via different algorithms.

QbieShay commented 2 years ago

Do you think it could be enough to provide a node for visual shaders instead? Or would you rather have it in text form? (See https://github.com/arkology/ShaderV for examples on how to make visual nodes)

@clayjohn I wasn't able to follow the progress on this side, but how is 4.x coming along with custom rendering pipeline? Do you think it's possible to keep the regular shaders and integrate with an additional pass of compute + rendering that would render the fluids?

QbieShay commented 2 years ago

I wonder if we could use the fog volume material for this?

paddy-exe commented 2 years ago

@QbieShay I was also going to support Visual Shaders but for Text based shaders, I would really like to see a subshader system (I believe this was already proposed) implemented to not bloat the actual shader file. It would be a lot of boilerplate code that could be optimised by having a custom shader material. I am not sure if Godot could use this when several local shaders have a lot of the same code.

About the Fog Volume: I am not sure how all the internals of the Fog Volume works but I believe it uses a different technique

godotengine / godot-proposals