Compute and store light direction information when baking lightmaps with the CPU lightmapper

[Calinou]

• Related to https://github.com/godotengine/godot-proposals/issues/2354 and https://github.com/godotengine/godot/issues/63016.

Describe the project you are working on

The Godot editor :slightly_smiling_face:

Describe the problem or limitation you are having in your project

When using BakedLightmap (LightmapGI in master) with lights that have their bake mode set to All, directional information is lost. This has two negative consequences:

• Materials' normal maps are no longer effective for baked lighting. This makes lighting look much more flat overall.
• On materials, specular lobes from lights are no longer displayed for baked lighting.

This is not as much of a problem with the default Indirect bake mode, since light direction is mostly visible for direct lighting (although it's not completely irrelevant for indirect lighting either).

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Both of these issues can be alleviated by storing the baked lights' directional information in a separate texture.

Here's an example of what it looks like in Xonotic, which uses fully baked lighting (both direct and indirect light):

https://user-images.githubusercontent.com/180032/127933548-65b24103-a45b-4b28-9f81-4a63ae79eb54.mp4

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Store directional information in the form of normal maps associated to each lightmap. (I don't remember if it has to be tangent-space or object-space, but it can surely be stored with red and green channels only to reduce the storage space and VRAM required.)

Then, in the default material shader, use this directional information to influence the pixel's normal direction.

For specular lobes, we will also have to figure out a way to determine the specular reflection's origin and color. I'm not sure exactly what's the math behind it exactly, but I know that the aforementioned Xonotic can display specular lobes without requiring light entities to be present in the compiled map. It just manages to do guesswork somehow :slightly_smiling_face:

There should be a property in BakedLightmap to disable baking directional light information for people who don't need it. This should speed up baking slightly and reduce the storage requirements, which is important when targeting mobile/web platforms.

This needs to be implemented separately for 3.x's CPU lightmapper and master's GPU lightmapper, although I think it's more important to have in 3.x first given the presence of low end-oriented renderers. Also, we already have some kind of directional lightmapping support in master, but it's broken for lights that use the All bake mode: https://github.com/godotengine/godot/issues/49936

Note: LightmapGI in master has a Directional property, but it only affects indirect lighting and doesn't work correctly when a light's bake mode is set to All. It seems to be more about enabling rough reflections using spherial harmonics rather than making normal maps and specular lobes effective with baked lighting.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as the lightmapper and default scene shader need to be modified.

Is there a reason why this should be core and not an add-on in the asset library?

This is core lightmapping functionality which can't be supplied by an add-on.

[clayjohn]

How would this handle multiple light sources? It sounds like you would need a normalmap for every surface and for every light source. If so, things would get very expensive very quickly.

My understanding is that effects like this are using handled by baking higher-order spherical harmonics. Even then, I can see it becoming very expensive if you desire a high level of detail.

[Calinou]

How would this handle multiple light sources? It sounds like you would need a normalmap for every surface and for every light source. If so, things would get very expensive very quickly.

This would store only one direction per texel, which makes it an approximation. Despite this, it can still provide good visuals as you can see in the video. If you design your lighting around these limitations, the difference between baked and real-time normal/specular may be hard to notice to the untrained eye.

Xonotic (DarkPlaces) doesn't rely on spherical harmonics or anything fancy like that. It's able to do this using OpenGL 2.1 only :slightly_smiling_face:

For reference, here's the lightmap + deluxemap (= light direction map) used in the Xonotic map featured in the video:

 

The final deluxemap is then added (or blended?) to the surface's normal maps, which allows it to be baked at the same resolution as the lightmap itself.

[clayjohn]

So this feature would be limited to a single light source per pixel?

[Calinou]

So this feature would be limited to a single light source per pixel?

It would average the light direction from all the nearby light sources (plus the directional light, if present at that pixel). It might sound very limiting, but it's not that bad in practice. It will almost always look better than not having any directional information available, because that results in flat-looking surfaces which look unappealing.

[atirut-w]

Not directly related but holy shit the Xonotic video blew my mind. I never thought completely baked lighting could look this good.

[Calinou]

I've found some more information about various lightmap directional storage methods here: https://geom.io/bakery/wiki/index.php?title=Manual#Directional_mode

The one I've suggested in this proposal seems to be akin to dominant direction.

Spherical harmonics are supported in master but not in 3.x. Besides, spherical harmonics are likely too expensive for a mobile/low-end oriented renderer, so we may still want to implement a cheaper method as a fallback for the OpenGL renderer in Godot 4.x. Dominant direction seems to be a good fit here, given it doesn't require too many additional texture files that would increase the export size significantly (unlike radiosity normal mapping).

[Calinou]

I experimented with supporting normal maps in the 3.x CPU lightmapper for fully baked lights. Here's an example where I took the normal map direction into account for direct light baking ("Baked Normal Map"). It largely relies on the lightmap having high texel density, so it's not really viable in production (on top of lacking specular information).

The dominant direction algorithm requires a different approach (storing a secondary texture with light direction only), so the code must be redone from scratch.