Goldsrc support completion

[Slartibarty]

Support for goldsrc is 90% of the way there, but some features are missing to allow for easier migration from ZHLT or VHLT. I'm creating this as a draft PR to allow for discussion and whatnot.

[Slartibarty]

Working on getting surface light emission looking more how I'd expect, I noticed that ZHLT and VHLT correct surface light colour values to linear from SRGB, which isn't performed by these tools right now.

To make this easier to do, I've swapped most places that refer to surface light colours over to 0-1 qvec3f rather than 0-255 qvec3b (seems bounce lighting already uses 0-1 qvec3d). Converting from SRGB to linear is the correct thing to do when sampling the average colour from a texture, or reading light colours, but is quite tricky due to no gamma correction being performed at lightmap output time by default (gamma 1.0). Half-Life expects the final lightmap to be gamma corrected by a factor of 1.8 (or 2.0 in the original tools). Which means that when gamma correcting the lights which aren't already converted to linear, they get converted from SRGB > gamma 1.8, which is not correct at all.

Basically, the problem I'm facing now is that colour isn't handled correctly by the tools, and if it is handled properly, then the default output will look wrong due to no correction if you don't specify a gamma > 1.0. Right now the reverse is true, where output looks wrong when you specify a gamma > 1.0.

Paril and I talked briefly about the gamma situation in the Quake mapping discord, right now the broken behaviour has become standard, basically Quake engines expect the lightmap in SRGB space, but historically the tools have always output in linear space (except in Half-Life). I'm going to propose a kind of "-nolegacy" switch which ensures the lightmapping pipeline always converts to linear, and outputs SRGB. This can be toggled on if a developer wishes to author a map with this pipeline in mind, or all the time for Half-Life maps, I'll include this in a commit soon.

I have a couple of questions about the code that determines the intensity and colour of surface / bounce lights:

Right now it looks like this, where texture_color is texture_color * light_val:

// Calculate intensity
float intensity = qv::max(texture_color);

// Normalize color...
if (intensity > 1.0f)
    texture_color *= 1.0f / intensity;

This appears to match the ColorNormalize function in qrad3. I don't fully understand why it's calculated this way, wouldn't it be more correct if it was:

// Calculate intensity...
float intensity =
    texture_color[0] +
    texture_color[1] +
    texture_color[2];

// Normalize color...
qv::normalizeInPlace(texture_color);

The second solution produces slightly different (but still correct) results when the pipeline is gamma correct, if the pipeline isn't correct, then it's totally broken. I don't know if this is technically the right thing to do, or if I'm just wrong.

For some eye candy, here's a screencap of the output from VHLT, and the output from the tools after my changes, pretty much identical:

[Paril]

How does the colornormalize change affect colored lighting? Can you maybe provide a small test case for it?

[Slartibarty]

How does the colornormalize change affect colored lighting? Can you maybe provide a small test case for it?

Sure, here's a link to a test map and a .rad file to go with it: testmap.zip qbsp: -hlbsp light: -emissivequality high -extra -range 0.5 -maxlight 0 -bouncecolorscale 0

Just pushed the code so far to the branch, which is needed to test this properly. The code that calculates the intensity by adding all channels together doesn't seem like the right solution, as it makes bounce lighting way too intense, although dividing the result by 3 might be the missing piece. I took a couple screenshots of a before and after of just the normalization method being changed, old is 1 / intensity, new is qv::normalize.

The new method more closely matches other compile tools, which gives me a hunch it's probably more correct.

[Slartibarty]

Did a little bit more testing, the "new" method for calculating intensity and normalizing color is definitely not right. Calculating intensity by adding all components then dividing by 3 produces an intensity value almost identical to just picking the largest colour component. Normalizing the colour with qv::normalize just produces output that doesn't seem right in actual maps.

At the very least, converting everything to linear before doing anything produces colours in the final lightmap that more closely match the input colours, though I need to make this optional.

[ericwa]

Question about the "overbright" checkbox:

In Q1/Q2, the original engine's rendering is with byte 127 = 100% brightness and 255 = 200%. There's a limitation in the original GLQuake where 128 through 255 were clipped to 100%. Engines like Quakespasm have a gl_overbright 0 option to view maps with the "broken" GLQuake style rendering (with values >127 clipped to 100%): https://www.quaddicted.com/webarchive/www.quaddicted.com/software-vs-glquake/software-vs-glquake-overbright-lighting/

So in the Q1 context, "overbright off" means clipping 128..255 to all be 100% brightness, and "overbright on" would be 255 mapping to 200%. It looks like the "overbright off" option here instead does 128 = 50% and 255 = 100%? Is that a convention used in HL engines?

[Slartibarty]

Half-Life is a bit unusual with how it processes the lightmap, the final output is a little bit darker than typical quake engines, right now by default it also clamps the lightmap brightness in a really poor way, where colours become washed out and completely white after a certain threshold. Valve were certainly away of overbrights, as there is a command called gl_overbright, but it was broken when multitexturing support was added to the game. It doesn't perform the broken clamping when overbrights are enabled, which is only possible with some hackery (hopefully this will become the default again in the future).

The overbright checkbox can probably be reverted, I added it to more easily see the exact final output of the lightmap without the overbright multiplier added on top, though it didn't really help in the way I wanted.

[SirYodaJedi]

Regarding the overbrights discussion, ZHLT/VHLT's default overbright compensation limiter thingy kicks in at 188 brightness (modifiable with -limiter # HLRAD param). I'm not quite sure how it works, but it's important for avoiding the color banding, since the shader-based reimplementation of gl_overbright (gl_use_shaders) can't reliably be assumed to be enabled (see https://github.com/ValveSoftware/halflife/issues/3424).

[SirYodaJedi]

Also, might be better to treat any texture name starting with AAA as a trigger texture, allowing stuff like hlbasics.wad's multiple trigger textures to not waste allocblock. There isn't any actual need for trigger textures to be named AAATRIGGER or TRIGGER besides compiler optimizations.

[Enokilis]

There appears to be multiple factors contributing to unintuitive behavior of the lightmap. Aside from the aforementioned gamma inconsistency, the lightstyle feature inherited from Quake does not seem to modulate to 100% by default (at least this is the case in Xash3D, which I assume is a proper imitation). It's not a significant deviation, so it would mostly manifest as some added unnecessary banding. I don't think there is any way to get around this, for my cvar manipulations have been in vain.

I definitely think the VHLT overbright correction limiter's default state should be reverted. A lot of mappers will potentially return to Half-Life with the recent update where shaders are enabled by default. It would only harm non-HL maps a bit, but it would harm new HL maps a lot more not to.

[SirYodaJedi]

Random shower thought: since the engine discards all water faces (those textured with !) except the top face when a brush is part of an entity other than worldspawn, it might be mildly beneficial to treat those disregarded faces as nodrawn when compiling (not sure how beneficial, though, given water isn't lightmapped).

where shaders are enabled by default

Only in the base game, and server owners can disallow them via a cvar (bandaid fix for avoiding wallhack shaders). Lighting looks a lot worse when unclamped without overbright than when clamped when overbright.

[jonathanlinat]

Hey @Slartibarty 👋🏻 What's missing to get this merged? Is there anything specific we can do to push this forward?

[Slartibarty]

Hey @Slartibarty 👋🏻 What's missing to get this merged? Is there anything specific we can do to push this forward?

Still missing support for HL light entities, like light_spot, and there needs to be a heuristic to convert HL light values to what the tools currently expect (intensity of 300 is different in ZHLT and these tools).

I've been occupied with other stuff since starting this PR, and I dug myself into a hole by trying to refactor the tools use of colour at the same time, which was a mistake and should be saved for another PR. I'll probably re-start this PR with just the changes that actually need to be here, rather than trying to do two things at once.

Initially I started working on this since there was a passing mention that peeps at Valve might be interested in using the tools for the HL 25th anniversary update, but they ended up using the VHLT toolset, which hit my motivation to work on this PR. I'll see if I can get it into a better state for others to potentially poke at this week.

[jonathanlinat]

Hey @Slartibarty 👋🏻 What's missing to get this merged? Is there anything specific we can do to push this forward?

Still missing support for HL light entities, like light_spot, and there needs to be a heuristic to convert HL light values to what the tools currently expect (intensity of 300 is different in ZHLT and these tools).

I've been occupied with other stuff since starting this PR, and I dug myself into a hole by trying to refactor the tools use of colour at the same time, which was a mistake and should be saved for another PR. I'll probably re-start this PR with just the changes that actually need to be here, rather than trying to do two things at once.

Initially I started working on this since there was a passing mention that peeps at Valve might be interested in using the tools for the HL 25th anniversary update, but they ended up using the VHLT toolset, which hit my motivation to work on this PR. I'll see if I can get it into a better state for others to potentially poke at this week.

Hey. No pressure at all! Thank you for your commitment on this.

I am so sorry you felt unmotivated. I still believe your work is valuable and will benefit.

I am personally looking to create a Goldsrc Level Design Starter Kit; including ericw-tools instead of ZHLT (or VHLT) will probably be a game changer for the community (because of all the advanced light features the tool offers).