mrdoob / three.js

JavaScript 3D Library.
https://threejs.org/
MIT License
101.86k stars 35.31k forks source link

WebGPURenderer: Make output of post-processing passes configurable. #28754

Closed Mugen87 closed 2 months ago

Mugen87 commented 2 months ago

Description

Next to #28749, this is the second of two issues I have encountered when implementing the latest post-processing passes.

Modules like AnamorphicNode or AfterImageNode use a separate render pass to write their result to a TextureNode (pointing to a renderTarget.texture) that represents the input for the next pass. I did not adapt that pattern for DepthOfFieldNode since a subsequent pass in the chain does not always require texture input.

In the previous composer, color space conversion and tone mapping was implemented as a separate output pass. This is not required with the new renderer anymore since the node system is able to add the color space and tone mapping code to the final output node. This is comparable with uber shader generation since the system avoids a render pass by combining shader code.

The same should be possible for FX passes as well which perform just per-pixel operations. In fact, it is already possible in certain cases. E.g. you can chain a simple color adjustment like vibrance() after dof() without the need of an intermediate texture.

Ideally, a pass (indirectly) tells its predecessor what type of input it needs. Consider this:

postProcessing.outputNode = scenePassColor.pass1().pass2();

If pass2 requires a texture, pass1 returns an instance of TextureNode in its setup() method. If no texture input is required, it returns its main TSL function ( e.g. anamorph(), afterImg() , sobel() ) which returns a vec4 representing the current pixel value.

Depending on how a pass should produce its result, different code should run in updateBefore(). If the result should be saved to a texture with a render pass, the code can look like in AnamorphicNode or AfterImageNode. If no texture is required, updateBefore() can be simply update uniforms like in DepthOfFieldNode.

The last pass in a chain should always be configured with vec4 since for outputting the result on screen via QuadMesh we don't need a separate render pass for producing a texture (in other words the fragmentNode of QuadMesh.material does not require a TextureNode).

Solution

The hard part (the combination of shader code) is already in place thanks to the beautiful TSL and node material.

The remaining bit is some sort of automatic pass chain configuration. Each FX pass could get a new property that defines the input type (e.g. pass.inputType = 'vec4' | 'texture'). A new analyze() method checks the flag and ensures each pass produces produces the correct input for its successor.

@sunag I'm not sure yet where in the node system something like analyze() should be defined though. Do you have something in mind?

Alternatives

Always rendering effects to intermediate texture nodes as input for the next pass. This will negatively affect performance though.

Additional context

No response

Mugen87 commented 2 months ago

I believe when #28749 and #28754 are solved, we have the foundation of a future-proof FX system in WebGPURenderer. Of course there is still room for optimizations like pointed out in https://github.com/mrdoob/three.js/issues/28749#issuecomment-2194353749 but it should be a proper basis for own FX passes and third-party effects.