Rendering is gamma-incorrect

[HybridDog]

SuperTux version: 4437651fa (a few unrelated changes to afb12af17) System information: Lubuntu 18.04

Expected behavior

Mixing two physically dark grey colours, e.g. dark grey lanterns, should lead to a bright grey colour and not full-bright white.

Actual behavior

Here is an image to show the symptoms; it has to be viewed in 1:1 resolution (without scaling): lantern_test_image On the top there's a 2x2 chessboard pattern of white and black pixels; since pixels are small, to a human this should look like 50% physical brightness (ignoring that black pixels are physically not completely dark on LCDs). I also added a few white rectangles.

• Left: A lantern with colour values 0.5. Since the colourspace is sRGB, it is darker than the grey from the chessboard pattern.
• Centre: Two lanterns with colour values 0.5. Since SuperTux does not heed gamma, they mix to white instead of bright grey.
• Right: A lantern with colour values 0.75. This roughly corresponds to 52% physical light, thus looks roughly as bright as the grey from the chessboard pattern.

Steps to reproduce actual behavior

There are a lot situations where the symptoms can be observed. For example, if you use a black-white chessboard pattern as tile texture, with gamma-incorrect scaling the texture looks brighter if it is shown without blurring in 1:1 resolution than if it is upscaled.

Additional debugging information

• More information about gamma:
  • http://www.ericbrasseur.org/gamma.html?i=3
  • https://bottosson.github.io/posts/colorwrong/
• It could be difficult to implement gamma-correct rendering in SuperTux in a backwards compatibility way. For example, adding an image with 50% transparency would be different in linear light and it is unclear if the image is almost completely black and used for darkening (e.g. fading) or the image shows, for example, a tree.
• In addition to gamma correctness, colour multiplication may be incorrect. The R,G and B sRGB/bt.709 primary colours are not monochromatic, so component-wise multiplication of linearized sRGB colour vectors may give inaccurate results

[Semphriss]

What are the upsides of gamma-correctness?

[HybridDog]

I think it's easier to do realistic lighting with gamma correction. For example, if a green and magenta lantern stand next to each other with a distance, without gamma correction the colour between the lanterns could be mixed incorrectly (see for example the gradients at https://bottosson.github.io/posts/colorwrong/#comparisons). Here's an explanation of the impact of gamma correction on 3D games: https://learnopengl.com/Advanced-Lighting/Gamma-Correction

[Grumbel]

The lightmap is technically already linear, as that just gets lights additive blended to it. Merging it with the game scene is however done via a plain multiplication without taking gamma into account. That should be relatively easy to fix for OpenGL3.3 with a shader (and maybe switching to a float framebuffer to avoid clipping), but no idea how to fix that for SDL2 or OpenGL2. Though changing that for GL3.3 should still be compatible, as the lightmap is still the same, would just lead to different visual results.

No idea how much of a difference this would make visually, but worth a try.

[Semphriss]

Isn't it better to have something easy to understand than something realistic? If someone really has an interest in mixing lights, it's much simpler to have a simple "they add values" rather than having them apply a formula, or go by trial and error.

Realistic colors is much more relevant in games trying to achieve realism (mainly 3d games, as shown in the article), but SuperTux being a cartoony 2D game, I don't really see the importance of realism over practicality and ease of use...

[HybridDog]

I think gamma-correction itself is easy to understand. Ideally the only changes needed would be the gamma removal when decoding the textures and colour values, and adding the gamma back at the end of the frame rendering as post-processing step (see these old testing changes for example). However it is more complicated when backwards compatibility is needed, e.g. because of alpha blending, SDL2 support, etc..

Without gamma correction, the users needs to apply a formula or go by trial and error because of the non-linearity. If a and b are linear colour values, adding them is simply a+b whereas if they are non-linear, adding them leads to approximately (a^p + b^p)^(1/p) for some value p.

In my opinion realistic light mixing is important because mixing happens very often in SuperTux in the lightmap. In the above example, if a player carries a dark grey lantern to another dark grey lantern the mixed light is white instead of bright grey.

[Semphriss]

Without gamma correction, the users needs to apply a formula or go by trial and error because of the non-linearity. If a and b are linear colour values, adding them is simply a+b whereas if they are non-linear, adding them leads to approximately (a^p + b^p)^(1/p) for some value p.

I'm not sure I understand...

What I meant was like, say a level designer has a situation like this:

"I have a 0.3-0.3-0.3 lamp and a 0.6-0.3-0.7 lamp, what color can I make the magic block so it can be activated by the combination of both lamps? And what if I have a 0.1-0.6-0.2 lamp and neither pair including that third lamp should activate the magic block?"

With the current system:

To be activated by the two former lamps, the block can be at most(0.3 + 0.6)-(0.3 + 0.3)-(0.3 + 0.7), therefore 0.9-0.6-1.0. Not more than 0.9 red and not more than 0.6 green and anything blue.

With the third lamp, the block must be above:

• [first + third] (0.3 + 0.1)-(0.3 + 0.6)-(0.3 + 0.2), therefore 0.4-0.9-0.5 and
• [second + third] (0.6 + 0.1)-(0.3 + 0.6)-(0.7 + 0.2), therefore 0.7-0.9-0.9.

That means the color must be above 0.4 red or above 0.9 green or above 0.5 blue, AND, it must be above 0.7 red or above 0.9 green or above 0.9 blue.

This means anything between [0.7-0.9]-[0.0-0.6]-[0.0-1.0] will work (The red channel serves as constraint for both the first-third lamp combination, and the second-third one.), or [0.0-0.9]-[0.0-0.6]-1.0 (The blue channel serves as constraint), or probably other combinations.

With proposed gamma correction:

Basically, the same thing, but for every a + b, we replace it with (a^p + b^p)^(1/p).

That already is a significant added complexity - the former system could be understood by a child, whereas the current system will discourage even an adult - but add to this the fact that this function is an approximation. Trial and error will be unavoidable, and testing every lamp combination will be much more time-consuming than doing 3 additions.

---

If what I described above is correct - what need for realism overrides the up in complexity and loss in ease of use that it represents? It's almost been a running gag of developers copy-pasting the same 12-line reply to people who accuse SuperTux of not being realistic enough (as SuperTux is about a cartoony penguin superhero stomping snowballs with googly eyes) so I won't do that - but I believe the key point in SuperTux is that practicability and ease of use always have priority over realism. For example, a flying platform isn't realistic at all, but it provides more possibilities of gameplay for more fun.

I really want to urge to not to give too much important to realism in SuperTux. Some realism can look beautiful, but it shouldn't come to the sacrifice of ease of use. It doesn't really matter if a sunset in a level isn't exactly as orange as in reality - if that's the price to pay to have it simple enough so that anyone can understand how it works, I'm ready to pay the price.

[Grumbel]

The gamma correction is not applied to the lightmap itself, but when merging the lightmap with the rest of the scene. Meaning 0.3 + 0.6 will still give you 0.9 on the lightmap and all those magic blocks will behave as usual. It's still just plain addition.

What gamma correction might fix is simply how the lightmap looks when rendered, i.e. it might give a nicer gradient in the brightness falloff or when merging multiple lightsources.

[Semphriss]

Ah - that's the part I missed.

For my personal comprehension: in the top comment, it is said that two lanterns with 0.5 brightness should mix to bright grey (let's say 0.75) instead of 1; does that mean the lightmap will be set to 1, but will be rendered as 0.75? What would happen if a third lamp at 0.5 brightness is added, will all three still mix to 0.75? Will they mix to a brighter gray? What values would the lightmap hold?

[HybridDog]

The lamp combination complexity is only increased if the user only knows the non-linear colours. I already changed the colour selection GUI in the editor so that it shows the linear colours in percent (https://github.com/SuperTux/supertux/commit/27ae58ca049789d29a51aa96ddd89309981822e5). However, for backwards compatibility the colours are still saved as non-linear values in the level files.

For linear light mixing, gamma would be removed from the lightmap textures (lamp shine texture) and colours (lamp colour), then the lightmap is rendered (colours are mixed), and after that gamma would be added back. So two lanterns with 0.5 non-linear/perceptual brightness would have ca. 0.218 linear/physical brightness in the lightmap, then be mixed to ca. 0.435 linear/physical brightness, and after the gamma is added back the resulting non-linear brightness is ca. 0.685. With three lamps, the resulting non-linear brightness is ca. 0.824.

If two lamps with physical brightness 0.5 are mixed, the resulting brightness (physical and here also non-linear) is 1, and if three lamps with physical brightness 0.5 are mixed, the resulting physical brightness is 1.5.

[Semphriss]

Put it simply, the color channels will no longer be capped at 1?

[HybridDog]

I don't know if the lightmap colours are capped at 1 in SuperTux. Colour values above 1 (high dynamic range) is another topic, where tonemapping or at least gamut clipping can be applied I think.