Irradiance light grid

[RobertBeckebans]

This will bring back prebaked irradiance data similar to the Quake 3 lightgrid.

However this time it will feature Spherical Harmonic based irradiance for each grid point so models will receive nice global illumination from all directions. It will use the same data structure as DDGI by Majercik and McGuire so we can update this in realtime with Vulkan Raytracing at a later point.

https://morgan3d.github.io/articles/2019-04-01-ddgi/overview.html

[BielBdeLuna]

great!

[BielBdeLuna]

for this wouldn't it be useful to have a Voxelized version of the world?

[RobertBeckebans]

Probably but I don't want to mess with any kind of voxelization because I want to keep RBDoom as simple as possible. The probe density will be based on the stuff that was already there in q3map1/2.

[BielBdeLuna]

Awesome work!

I've baked the light grid with "bakeLightGrids" and you can activate/deactivate it's usage with "r_useLightgrids", you can see the light grids with "r_showlightgrids"

a couple of questions:

can you explain how come it doesn't take ages taking all those probes? :)

can you explain how is the light interpolated between all those probes?

I see that some of the env probes in the light-grid fall outside the play area (like outside the map) couldn't a intermediate process decide if an light grid env probe is inside the play area or not in order to discard those that are outside the play area?

I see that the light grids are separated by the portal areas ( I think ) could those independent light grids be further divided more like how voxels get divided ?

can the light grids env probes be taken baked behind the game process while the game is playing?

if so how many light probes in the light grids can be processed in a game frame? maybe the light grids env probes around lights that get activated and deactivated (by scripts not by material sequences) could be reprocessed so when a level turns on/off lights a couple of frames latter the light grids env probes around those lights could change to adapt to the new light situation?

I see there is a discrepancy between how much the light grid lights up the scene and how much the env probes specular contribution lights up the scene. I think a r_forceAmbient of value 1 should be correct for the light grids env probe contribution, but then the env probes specular contribution is too high, maybe there could be two separate values for a light contribution for the light grids and a specular contribution for the env probes?

[BielBdeLuna]

Some bugs I've encountered with the light grid in this video. If someone doesn't know, the green lines in the video indicate the portals in the map. also in the video some times I skip specular/interaction/bump in order to see the light grid effect on it own, and sometimes I skip the light grid or PBR completely.

the most important of the bugs: the bounding box of a light grid inside the bounding box of another light grid at this time in the video.

another: strange light grid probes that are completely black at this time.

and also another regarding the limits of every light grid with the next at this part of the video.

and in the end, a env probe related problem, also related form the case of a env probe bounding box inside another env probe bounding box in this moment in the video.

I hope it helps, I can provide the test map for further testing.

[RobertBeckebans]

Baked environment probes and lightgrid data for all BFG single player maps (2.5 GB download): https://drive.google.com/file/d/1iS26doWJDQHLBm5wCHLyz0QbF4FEoUDp/view?usp=sharing

[RobertBeckebans]

It looks like I iterated only over the first 32x32 pixels of the captured environment probes so the black probes might be fixed with the current master if you rebake your scene.

[BielBdeLuna]

this looks much better, the black probes are still there though. In the first map, the smaller one, there doesn't seem to be strange probes with seemingly wrong colours, and the overall light grid seems to be more grounded to the extreme sky colours and the extreme colours of the scene textures in general.

I'm finding strange colours in the sponza map though, the light grid seems to sometimes section part of the sponza model like if there where a portal there (when there isn't) so the baked light grid seems to have some parts that are completely black, it has improved from the first time I tried Sponza with the light grids but there is still something strange going on still.

^ ^ ^ that is with the last commit

the lightgrid probes have a range isn't it? wouldn't it be useful to have a half resolution light grid with double the range, so more macro situations could be captured, like those arches in the sponza, or even spaces between buildings, so a wider are could be affected by a lower spatial resolution light grid, an so that could be used for more spacious GI effects, in the distance maybe? I know this would mean beaking even more probes, but maybe four probes could be mixed together? and put that half resolution light grid at the middle point of the higher resolution probes, so none of the half resolution probes would be situated in any of the same positions of the higher resolution ones. would it be any useful?

[BielBdeLuna]

I wonder if the AAS information could be used to concentrate the light grid probes around where there the gameplay happens. Maybe we could have those zones within the portal bounding box but outside the AAS information generate less probes. maybe we could generate a AI bounding box ( per portal bounding box ) and have the light grid have a higher resolution within those AI bounding boxes.

here is an example of some probes that get calculated but their contribution to the game play is most probably null:

those are Cyberdemons. so because those rooms are so tall, most of the visible probes that are there are too separated to the geometry, as well as the height those Cyberdemons can actually reach, so those high probes contribute nothing to the lightning when in play, maybe we could mitigate their creation with lower density of the grid in those zones? or even have nothing out there so there is no time wasted generating them? Just flying enemies might have a chance to be influenced by those probes, but then, those enemies might be so distant to the player, that the fact that their lightning is not using a light grid might not be noticeable.

Maybe we could have a sky ambient probe generated from the sky box ( I know currently the sky boxes aren't HDR but so is their current LDR contribution ), and use them for the "zones outside the light grid reach", generated maybe when the level starts, and if there is no sky box available then we could assume we are underground and so this "sky ambient probe" should be all black.

maybe also we could automatize so the zones without AAS data, like stuff outside the windows, zones unreachable to the player have a lower density light grid ?

[BielBdeLuna]

how is the bakelightgrid -bounce[num] number have to be entered? I enter -bounce[2], -bounce 2, and -bounce( 2 ) but it doesn't seem to work it always is 1, maybe it's not yet implemented?

[RobertBeckebans]

Hehe it's bounce2. A valid example is

bakeLightGrids limit4000 bounce2 grid(32 32 64)

[BielBdeLuna]

the env probes also convolute an ambient image, is this ambient convoluted image used anymore? maybe it's used when there isn't any light-grid? if not, maybe we could skip it's creation and then use a "light grid" approach for the env probes ( with maybe something closer to a grid(256 256 256) ) so the specular reflection has more than one source per area.

[BielBdeLuna]

I made a map for testing a lot of light-grid probes and here is the link to test it: https://github.com/BielBdeLuna/OTE_testmaps/tree/vilcabamba

I made a bake with: bakeLightGrids limit6000 bounce3 grid( 16 16 24 ) In order to allow such amount of probes in the light grid I needed two things:

• no wasted probes in several portaled spaces.
• break the big rooms for smaller grids.

hence the amount of portals, even some apparently unnecessary. Those unnecessary where meant to keep all the rooms contained as much as I could as rectangular parallelepipeds that didn't cross each other. the render time has been a lot! like half an hour or so. the textures for the map's probes are 290MB.

[BielBdeLuna]

The room with the largest amount of env-probes is the one that kills the joy... and your RAM. A good idea would be to try to create the rooms that contain the largest amount of probes first, so in the first minutes of the bake you know if you get a SIGKILL for lack of RAM and not in the 30th minute :)

[BielBdeLuna]

here is a trouble I usually find around portals, as if the portal was closed for the probes of one side and open for the ones in the other side.

[BielBdeLuna]

I think the implementation of bounces > 1 is physically incorrect:

two statements:

• light is additive.
• light bounces on walls when it travels at light speed and doesn't necessarily stay in the walls (not taking into account translucency here).

if I understand correctly that's how we are capturing this whole matter ( I will reduce the problem to a single light probe, and grossly oversimplify it ):

• we render the light with no bounces with the normal rendering procedures (the direct light)
• we capture the first bounce of the light by taking a exr cubemap in the position of the probe
• we re-render the light with no bounces with the normal rendering procedures + the contribution of the existing probe(the first bounce)
• we capture the second bounce of the light by taking a cubemap in the position of the probe
• we sum the result of the existing probe to the incoming probe and we store it as the existing probe
• we re-render the light with no bounces with the normal rendering procedures + the contribution of the existing probe(the "second bounce" )
• we capture the third bounce of the light by taking a cubemap in the position of the probe
• we sum the result of the existing probe to the incoming probe and we store it as the existing probe etc...

If this has been correct... then we are adding the old bounces as well as the direct light contributions many times, when in reality their contribution should be only once. If you had infinite light bounces the level would end up infinitely brightened, isn't it?

So we should only re-render the scene without lights and only with the most recent bounce diffuse effect on, so by about the third bounce the light contribution should be next to nothing ( unless we are using very powerful lights where then more bounces the more bountiful those would be with those powerful lights ).

Maybe every step of the bounce should be stored within a different cubemap and only be summed once once all bounces have been calculated. I know this will multiply the amount of space occupied in disk.

It would make the GI less beautiful but more physically correct, isn't it?

a possible approximation in order to waste less disk space could consist on using a fourth channel that would made of the difference between the data contained in the cubemap minus the last contribution (luminosity wise, as we would be using only a single channel for that), so eventually multiplying that fourth channel to the other three would get the resulting luminosity of the last bounce, but it would lack the correct chrominance of the last bounce which would be based on the contained sum of all the contained bounces as well as the direct light ( just as we have it now), for it so it would be just an approximation and not the real deal. edit with many bounces using this method the chrominance of the bounced light would end distorted.

What do you think? are my assumptions correct?

[RobertBeckebans]

Well this is actually a big research topic where you can find more about it here: http://www.jcgt.org/published/0010/02/01/

Right now there is light added with every bounce. Usually with DDGI you have enough energy with 5 bounces. With every bounce you would usually loose energy.

RBDoom does this at the moment for bounce >= 2

• 1. bounce ignores irradiance and assumes it is black. Only direct light interactions like in D3 and emissive surfaces are light input for the irradiance.
• With the iteration after that it will use 95% of the previous irradiance in the ambient pass. This is basically the same as with the official DDGI implementations.

I think the caveat is that it is pumping new direct light energy into the sum with every iteration. I tested this against blender and r_forceAmbient 0.5 and bounce 2 gives results which are really close to Cycles. Everything above bounce 3 looks too bright.

Right now I recommend to ignore the issue and just stick to r_forceAmbient 0.5 and bounce 2. Everything else will yield to a lot more experimentation and light rebaking time.

[BielBdeLuna]

ok yes, so as it is right now only with Bounce 1 seems to be the only physics correct bounce, isn't it?

for bounce 2 in specific, maybe we could captured it with r_skipInteractions to 1, no emissive surfaces, and the irradiance of the first bounce ON. So we would only capture the actual irradiance of bounce 1 from the probes again and add the result to the env-probes with bounce 1.

The third and on are still troublesome.

About that 95% magic number, do you know where this magic number comes from?

also, about r_forceAmbient being 0.5, isn't it slicing the irradiance effect in half? So shouldn't it be 100%? and since the effect now describes all the ambient light aspect, the normal D3 ambient light as well as the irradiance, not just a forced added ambient term, wouldn't it make more sense to have it renamed to just r_ambient?

edit the way to generate the light probes in the light grid is per stages from up to down isn't it? a way to use less memory could be to generate a different atlas per stage (so it would be the same process but we would momentarily do more files) and once an area is completed then a new process that would join all the generated atlases of the areas as single one (just like we have now ) and repoint every of the repositioned squares to the new position in the atlas. This whole process would enable using less memory for the same amount of probes, or the possibility to work on more areas, or even also more denser areas.

edit could we skip the convolution stage and simply generate 16x16 environment probes? maybe if we are making tiny env-probes we could work with more than one env-probe at the same time?

[BielBdeLuna]

I did three pull requests (because I already have my OTE fork, so github doesn't allow me to fork RBDoom3BFG further, so I had to do the changes manually)

• the first creates a flag for skipping direct lighting: https://github.com/RobertBeckebans/RBDOOM-3-BFG/pull/577

• the second adds the flag as a skip for interactions: https://github.com/RobertBeckebans/RBDOOM-3-BFG/pull/578

• the third skips the direct lighting in bounce 2: https://github.com/RobertBeckebans/RBDOOM-3-BFG/pull/579 <-- this step is not right! as a result it seems to skip the first bounce! and it still captures the sky in the second bounce when it shouldn't!

I know it compiles but I haven't it tested it yet. it works!

edit In the last pull, within the bakeLightGrids function instead of having this:

ref.rdflags = RDF_IRRADIANCE;
if( bounce == 0 )
{
    ref.rdflags |= RDF_NOAMBIENT;
}
else if( bounce  == 1 )
{
    ref.rdflags |= RDF_NODIRECT;
}

maybe we should have that just an else without an if, so it would be applied to all the next bounces, it wouldn't make bounces > 2 physically correct, but it would make them less incorrect than with all the direct lighting.

EXAMPLES

1 bounce @ r_forceAmbient 0.5


2 bounces @ r_forceAmbient 0.5


2 bounces @ r_forceAmbient 0.5 ( old system )


2 bounces @ r_forceAmbient 1.0


2 bounces @ r_forceAmbient 1.0 ( old system )


The new system is darker but I think it gains in latitude.

[BielBdeLuna]

I'm not entirely sold on this yet, it seems it just ignored the first bounce entirely and we are just seeing the effect of the second bounce.

here is a video of it in action, it seems like we are replacing the first bounce with a more diffused second bounce but then we're not summing the results.

I uploaded the map with the light-grid and the processed point-lights and env-probes up in here

in the first room it seems that the high powered sun ( it's _color is 3.768 3.768 3.945 so it's about 3 times brighter ) doesn't reflect off the red stone as in the first bounce, and we are only seeing the diffused effect of the second bounce, that's the reason there is so much red spread all over the first room, but not much around where directly the sun light hits.

[BielBdeLuna]

maybe for every step, using the appropriate mask for direct or indirect lights, we could get all the new irradiance data within it's own EXR as area0_lightgrid_bounceN.exr and then use it as the basis of the irradiance data of the next step (if any) so for the first bounce and store it as something like area0_lightgrid_bounce1.exr and for the second area0_lightgrid_bounce2.exr, and so on. In the end of the process we create area0_lightgrid_amb.exr as a sum of all bounceN EXRs, and then we get rid of all those.

[BielBdeLuna]

Also the sky is an emissive surface that I don't know how to skip it for the bounces > 1, maybe we should have a idRenderBackend::DrawEmissiveInteractions method in idRenderBackend

[BielBdeLuna]

I've found this quake 2 vertex light improvement video on YT maybe this could be used for the furthest lighting instead of the irradiance system, or for fallback system in slow machines?

https://www.youtube.com/watch?v=Oh38Fpsc1mU in the video, there is a method of vertex lighting presented for quake2

here is my comment on the video: I wonder if this could be used in RDoom3BFG as a fallback for the light grid, there is a light grid that covers every portal area, a 3D grid of points that use light probes for ambient light, but maybe it could be used in the following way:

• every point in the grid has a radius that is the distance to the next point, this makes it a sphere.
• this radius lowers the power of the whole effect per every sphere in the grid:
• for every sphere, the most significant lights get selected up to a finite num of lights (maybe 16) <-- I realize that is THE difficult problem to solve
• for every light we get a direction and the power of it
• then we proceed to light every vertex within the bounding box of every point sphere in the grid with a direct lighting of every one of the 16 possible lights (or less) and taking into account the power of the radius of the sphere.

what do you think?