ericwa / ericw-tools

Quake/Hexen 2 Map compiling tools - branch of http://disenchant.net/utils
http://ericwa.github.io/ericw-tools
GNU General Public License v2.0
309 stars 56 forks source link

[idea] speed up lightgrid calculation #425

Open BraXi opened 1 month ago

BraXi commented 1 month ago

With current approach lightgrid points are calculated for entire mins/maxs of a worldmodel which often results in lightgrid points being computed for large out of player reach areas, this also does take noticeably more time than calculating lightmaps for large outdoor areas.

My idea is to introduce _lightgrid textured brushes that would cover playable areas, grid points within such brushes would be properly calculated, leaving outside grid points fullbright or dark and/or not computed at all to save on compilation times.

This should be entirely optional and enabled only by a switch or use of _lightgrid texture in a map.

ericwa commented 1 month ago

A few ideas:

  1. a special texture name like lightgrid_lowdensity (or a func_group key) that would calculate lighting at the center of the brush, and any lightgrid sample points falling inside that brush would use the already calculated light value from the center, rather than being calculated per grid point.

    Note that we don't have a way of storing these at a lower density in the output BSPX lump, so the size of the output lump wouldn't get any smaller, but it would be faster to compute lighting. (having the LIGHTGRID_OCTREE lump able to switch to a lower density for large outdoor areas was on my feature wishlist but it didn't happen, so if we ever implemented that it'd need to be a new lump).

  2. We do have a concept of a lightgrid point being "occluded" - if it falls inside the void (CONTENTS_SOLID), we don't calculate lighting for it, and the octree lump can avoid writing out areas that are completely occluded. So we could expose this to the mapper, with either brushes to mark areas as "occluded" that normally wouldn't be, or like you were proposing an opt-in version where everything is occluded except the areas marked with _lightgrid textures.

    My only worry with this is, I don't know what renderers will do if a model goes into this area. I guess it would be "undefined lighting" so the engine could do whatever. Do we really want to expose this as a mapper feature since it sort of works against the initial goal of having lightgrid? Are there large areas where you definitely won't ever need to sample a lighting value?

  3. it's possible the lighting calculation for the lightgrid could be optimized further, it was sort of done in a rush.

any thoughts @Paril ?

BraXi commented 1 month ago

For clarification of why I proposed that, here's the light output of a rather not too big but outdoor level where lightmap pass takes roughtly 30 seconds with extra and decompledlm, but about 460 seconds to compute lightgrid, I've already shown that to @Paril earlier.


---- light / ericw-tools 2.0.0-alpha6 ----
LoadBSPFile: '../../build/main/maps/e1m1.bsp'
BSP is version Quake II Qbism BSP QBSP:38
using gamedir: 'Z:\pragma\build\main'
WARNING: failed to find basedir 'Z:\pragma\build\baseq2'
img::LoadPCXPalette: Failed to load 'pics/colormap.pcx'.
INFO: using built-in palette.
---- img::load_textures ----
img::AddTextureName: WARNING: can't find meta data for industrial/metal@rust1
img::AddTextureName: WARNING: can't find meta data for wall/conc@wall5
img::AddTextureName: WARNING: can't find meta data for wall/conc@wall2
img::AddTextureName: WARNING: can't find meta data for wall/brick@wall3
img::AddTextureName: WARNING: can't find meta data for wall/stone@wall1
img::AddTextureName: WARNING: can't find meta data for wall/brick@wall2
img::AddTextureName: WARNING: can't find meta data for industrial/metal@whiterust
img::AddTextureName: WARNING: can't find meta data for floor/wood@planks4
img::AddTextureName: WARNING: can't find meta data for wall/brick@wall1
img::AddTextureName: WARNING: can't find meta data for street/brick@paving9
img::AddTextureName: WARNING: can't find meta data for street/brick@paving10
img::AddTextureName: WARNING: can't find meta data for street/brick@paving2
img::AddTextureName: WARNING: can't find meta data for street/conc@paving2
img::AddTextureName: WARNING: can't find meta data for ground/ground@dirt3
img::AddTextureName: WARNING: can't find meta data for industrial/conc@conc2
img::AddTextureName: WARNING: can't find meta data for wall/wood@blue1
img::AddTextureName: WARNING: can't find meta data for ground/grass@grass3
img::AddTextureName: WARNING: can't find meta data for wall/conc@plaster1
img::AddTextureName: WARNING: can't find meta data for floor/wood@planks1
img::AddTextureName: WARNING: can't find meta data for dev/floor_grey_c
img::AddTextureName: WARNING: can't find meta data for roof/brick@rooftiles1
img::AddTextureName: WARNING: can't find meta data for street/asphalt@road4
img::AddTextureName: WARNING: can't find meta data for floor/marble@tiles2
img::AddTextureName: WARNING: can't find meta data for industrial/wood@chipboard1
img::AddTextureName: WARNING: can't find meta data for floor/wood@planks3
img::AddTextureName: WARNING: can't find meta data for floor/wood@planks2
img::AddTextureName: WARNING: can't find meta data for industrial/metal@rust2
img::AddTextureName: WARNING: can't find meta data for floor/conc@tiles1
img::AddTextureName: WARNING: can't find meta data for floor/wood@panels2
img::AddTextureName: WARNING: can't find meta data for wall/brick@maybewall
img::AddTextureName: WARNING: can't find meta data for industrial/metal@rust4
img::AddTextureName: WARNING: can't find meta data for wall/conc@wall3
img::AddTextureName: WARNING: can't find meta data for wall/wood@blue2
img::AddTextureName: WARNING: can't find meta data for street/brick@paving7
img::AddTextureName: WARNING: can't find meta data for street/brick@paving5
img::AddTextureName: WARNING: can't find meta data for street/conc@paving3
img::AddTextureName: WARNING: can't find meta data for industrial/metal@yellowrust
Loading extended texinfo flags from ../../build/main/maps/e1m1.texinfo.json...
---- LoadEntities ----
101 entities read, 25 are lights.

--- Options Summary ---
    "threads" was set to "8" (from command line)
    "range" was set to "1.000000" (from game target)
    "dirt" was set to "1" (from command line)
    "minlight_dirt" was set to "1" (from command line)
    "bounce" was set to "1" (from command line)
    "bouncestyled" was set to "1" (from game target)
    "bouncescale" was set to "0.850000" (from game target)
    "bouncecolorscale" was set to "0.500000" (from game target)
    "surflightscale" was set to "0.650000" (from game target)
    "surflightskyscale" was set to "0.650000" (from game target)
    "sunlight" was set to "35.000000" (from map)
    "sunlight_color" was set to "109.65 104.549995 114.75" (from map)
    "sunlight2" was set to "60.000000" (from map)
    "sunlight_dirt" was set to "1.000000" (from command line)
    "sunlight2_dirt" was set to "1.000000" (from command line)
    "sunlight_mangle" was set to "0.35355335 0.61237246 -0.70710677" (from map)
    "sunlight_penumbra" was set to "4.000000" (from map)
    "surflight_radiosity" was set to "1" (from game target)
    "extra" was set to "2" (from command line)
    "world_units_per_luxel" was set to "8.000000" (from command line)
    "lightgrid" was set to "1" (from command line)
    "lightgrid_dist" was set to "24 24 24" (from command line)

running with 8 thread(s)
running with lower priority
Embree_TraceInit: Embree version: 3.12.1
Embree_TraceInit:
        202 sky faces
        14145 solid faces
        56 filtered faces
        0 shadow-casting skip faces
SetupLights: 25 initial lights
---- MakeSurfaceLights ----
SetupLights: 25 after surface lights
SetupLights: 25 after jittering
Final count: 25 lights, 129 suns in use.
---- SetupDirt ----
       48 dirtmap vectors
---- LightWorld ----
---- CalculateVertexNormals ----
[100%] time elapsed: 0.041s
---- MakeFaceCache ----
[100%] time elapsed: 0.132s
---- CreateLightmapSurfaces ----
[100%] time elapsed: 0.450s
---- MakeRadiositySurfaceLights ----
[100%] time elapsed: 0.000s
0 surface light points in use.
---- Direct Lighting ----
[100%] time elapsed: 8.281s
---- MakeBounceLights ----
[100%] time elapsed: 0.009s
---- Indirect Lighting (pass 0) ----
[100%] time elapsed: 20.895s
---- Post-Processing ----
[100%] time elapsed: 0.002s
---- SaveLightmapSurfaces ----
[100%] time elapsed: 0.123s
Lighting Completed.

lightdatasize: 25313928
DECOUPLED_LM BSPX lump written
---- LightGrid ----
[100%] time elapsed: 457.486s
     24 24 24 lightgrid_dist
     160 214 43 grid_size
     -2048 -2304 -128 grid_mins
     1792 2816 896 grid_maxs
     1 num_styles
octree stored 1444300 grid nodes + 28020 occluded = 1472320 total, full stored 1472320 (octree is 98.09688 percent)
octree nodes size: 12540 bytes (285 * 44)
octree leafs 1920 overhead 46080 bytes
      6792965 bytes LIGHTGRID_OCTREE
---- ClearLightmapSurfaces ----
[100%] time elapsed: 0.416s
0 switchable light styles (32 max)
Writing ../../build/main/maps/e1m1.bsp as Quake II Qbism BSP QBSP:38
492.798s seconds elapsed
0 empty lightmaps```