Implement temporal antialiasing (TAA) in the Vulkan renderer

[Calinou]

Related to https://github.com/godotengine/godot-proposals/issues/2779, https://github.com/godotengine/godot-proposals/issues/3855 and https://github.com/godotengine/godot-proposals/issues/4280.

Describe the project you are working on

The Godot editor :slightly_smiling_face:

Describe the problem or limitation you are having in your project

Right now, we have MSAA and FXAA available in both 3.x and master. These algorithms serve their intended purposes, but each of them has their own limitations:

• MSAA looks good, but it is very expensive on master due to the added complexity of shaders. This is especially noticeable when using VoxelGI or SDFGI. It also doesn't smooth out aliasing that originates from fragment shaders such as specular aliasing. (Alpha-tested surfaces are now well-handled thanks to alpha antialiasing and alpha-to-coverage support.)
• FXAA is cheap and looks decent on 1440p and higher. Unfortunately, the added blurriness makes it hard to use on 1080p and below. Post-processing sharpening algorithms make it possible to recover some of that lost sharpness, but they also add more aliasing on their own, so it's a tradeoff.

It's possible to use both MSAA and FXAA at the same time, but this has a significant cost and it won't improve the image's sharpness after it has been reduced by FXAA.

It's great that MSAA is still supported in master (thanks to the clustered forward renderer), but we have to admit that it's difficult to use in production with today's shader complexity. Therefore, this makes MSAA mostly suited to games that use few visual effects, which generally implies a stylized art direction.

TAA is also the antialiasing algorithm that usually works the best with FSR.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Temporal antialiasing uses information from previously rendered frames to reduce aliasing on the current frame.

Unlike MSAA, TAA works on aliasing generated by shaders such as specular aliasing. With the current screen-space roughness limiter not being very effective, this will provide a good alternative when specular aliasing is an issue. This is the main draw for using TAA, but TAA also often performs better than MSAA.

TAA can also be used as an upscaling method when rendering 3D at a lower resolution. This is not a goal for an initial implementation, but it could be looked at later.

The downsides of TAA are:

• Ghosting artifacts may be visible in fast-moving scenes. This is less noticeable at higher target framerates, but it can be an issue for competitive games. This is why having MSAA as an alternative is a good thing :slightly_smiling_face:
• Shaders are more complex since they need to emit motion shaders. In general, the rendering code will require a lot of changes to support TAA.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

• Make the engine emit motion vectors for each vertex (and modify the fragment shaders to account for this).
  • Doing this correctly for things such as soft bodies may be difficult. See https://bartwronski.com/2014/03/15/temporal-supersampling-and-antialiasing/.
• Integrate some kind of temporal aliasing algorithm.
  • Jitter vertex positions slightly every frame to make temporal antialiasing work when the camera doesn't move.
• Modify built-in shaders to jitter noise patterns every frame when TAA is enabled. This will allow shadow rendering to look smoother, therefore resolving https://github.com/godotengine/godot/issues/53534.
  • Edit: Separate proposal opened: https://github.com/godotengine/godot-proposals/issues/4179
• Expose project settings to configure TAA.
  • In particular, exposing a setting to allow choosing a tradeoff between better antialiasing (and more ghosting) and less ghosting (but worse antialiasing): https://alextardif.com/TAA.html
• Expose motion vectors so they can be used in custom shaders.
  • This will also allow motion vectors to be used for a motion blur implementation.

In any case, TAA isn't quite a "plug and play" solution. There's a lot of work involved in making it work well enough for production use, which is why this proposal is targeting 4.1 or later.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as TAA needs to integrate with low-level rendering functionality.

Is there a reason why this should be core and not an add-on in the asset library?

See above.

[Zireael07]

Motion vectors please please <3 IIRC there's an open request for motion blur?

[jcostello]

@Calinou Is this in the scope of 4.0?

[Calinou]

@Calinou Is this in the scope of 4.0?

reduz said he'd like to work on implementing TAA for 4.0, but he is currently dealing with moving to Spain. Therefore, I can't guarantee he'll be able to implement this by the time 4.0 is released.

[clayjohn]

Is this in the scope of 4.0?

It is if someone has the time to implement it! But as Calinou points out above, if we are relying on reduz doing it then it will likely be after 4.0.

[LaserYGD]

This would be great, nowadays TAA is a must for games using PBR. Specular aliasing is just too much.

[Calinou]

@LaserYGD Please don't bump issues without contributing significant new information. Use the :+1: reaction button on the first post instead.

For reference, TAA is currently being worked on as we speak, but it's not ready to be open as a pull request yet. A complete TAA implementation is a lot of work – you need to account for MultiMeshes, skinned meshes, shaders that move vertices around, soft bodies, …

TAA is aimed at being finished for 4.0, but I can't give any guarantees on whether TAA will actually be available in 4.0.