godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.12k stars 69 forks source link

Add TonyMcMapface as a tonemapping mode (or allow using custom LUTs for tonemapping) #7263

Open Calinou opened 1 year ago

Calinou commented 1 year ago

Describe the project you are working on

The Godot editor :slightly_smiling_face:

Describe the problem or limitation you are having in your project

Godot's current tonemappers have various limitations, with none of them "ticking all the boxes" so to speak:

Note that the above descriptions assume you're using a whitepoint of 6.0, which is common in scenes with bright highlights.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

TonyMcMapface describes itself as:

Tony is a display transform intended for real-time applications such as games. It is intentionally boring, does not increase contrast or saturation, and stays close to the input stimulus where compression isn't necessary.

It ships as a LUT along with a bit of shader code for sampling. There's no tweakables; if you need a different look, do it before and/or after the display transform.

Please note that the shader output is linear, so if "gamma space" sRGB is desired, the sRGB OETF must be used afterwards.

Bevy, which has recently adopted TonyMcMapface as its default tonemapper, has a slideshow example here comparing TonyMcMapface to Reinhard tonemapping:

https://bevyengine.org/news/bevy-0-11/#new-default-tonemapping-method

Reinhard TonyMcMapface
Reinhard TonyMcMapface

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

As TonyMcMapface was generated using expensive offline shader derivation, the tonemapper comes as a DDS texture which is a 3D look-up-table. There are also HDR/EXR formats that could be created if needed (see https://github.com/h3r2tic/tony-mc-mapface/issues/4#issuecomment-1440984013). This means the LUT needs to be embedded in the Godot binary, both in the editor and export templates.

This tonemapper could be used by default in new projects using the approach proposed in https://github.com/godotengine/godot-proposals/issues/4834. Existing projects wouldn't be affected.

The main challenge of this approach is that the DDS file is 432 KB (306 KB when zipped), which is non-negligible as it'll increase Godot's binary size by this much. However, given tonemapping is not available when using the Compatibility rendering method, we could strip this data from web exports which only support the Compatibility rendering method. This would prevent increasing web exports' base size (and therefore loading times).

This binary size concern means that it may be better to allow using a custom LUT as a tonemapper, separately from the color correction step. As a result, we won't need to bundle the texture in all projects (even those that don't use TonyMcMapface tonemapping) and users will be able to use their own tonemappers by providing a 3D LUT. The downside of this approach is that we won't be able to default to TonyMcMapface in new projects, and discoverability will suffer to an extent.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as this can't be implemented by a script or shader code without modifying the engine. See below.

Is there a reason why this should be core and not an add-on in the asset library?

You can see my existing findings with trying to use the existing TonyMcMapface 3D LUT as a color correction effect in https://github.com/h3r2tic/tony-mc-mapface/issues/4.

Unfortuntaely, it doesn't work as expected as color correction in Godot is performed after tonemapping. This is generally desired behavior, but not when you're trying to use color correction as a tonemapper :slightly_smiling_face:

If there was a way to change the existing Tonemapping → Color correction → Adjustments order, then this could be implemented by an add-on with no script or shader code needed. While this approach is the easiest to implement, it would also prevent using color correction at the same time as a custom tonemapper. This can be limiting when trying to achieve a certain scene mood, as TonyMcMapface itself does not provide any adjustments by design.

allenwp commented 1 week ago

If HDR LUT is implemented for tone mapping and HDR color grading, this is an example of how the workflow could be used.

image

To further optimize things, linear to gamma space conversion could possibly happen in this LUT, but I believe that would make it problematic for other effects like the glow effect.

Edit: My mistake. The TonyMcMapface tonemapper is actually best described as an SDR color correction LUT applied on a basic Reinhard tonemapping. So my initial thoughts I wrote above about an HDR tonemapping LUT do not apply to this proposal.

jirisvd commented 1 week ago

The TonyMcMapface tonemapper is actually best described as an SDR color correction LUT applied on a basic Reinhard tonemapping.

It seems that if Tony McMapface isn't going to be built-in, the only way to support it would be to allow users to write custom tonemapping shaders, just like they can write e.g. custom sky shaders. Then they could copy and paste the appropriate Tony McMapface sampling shader code (translated to .gdshader) and provide a copy of the 3d LUT texture as a uniform themselves.

I guess it would be good to decide which solution (hard coding support for Tony McMapface into the engine or implementing custom tonemapping shader support) is preferred for Godot, because both options have their pros and cons.

Just to summarize what was already mentioned:

Adding Tony McMapface directly into Godot:

Adding support for custom tonemapping shaders:

jirisvd commented 1 week ago

Having thought about this for a bit, directly adding Tony McMapface to the engine seems to me like the better option. The problem outlined in the proposal revolves around the limitations of existing tonemappers and a lack of a balanced neutral option that would look good out of the box without the need for extensive tweaking.

Supporting Tony McMapface natively would be a targeted solution to this problem that's very easy to use and well worth the ~300 KB of binary size considering how common the problem of "I can't get the colors in my scene to look right" is.

jirisvd commented 5 days ago

Opened a PR: godotengine/godot#97095