Hexen: add true color support

[JNechaevsky]

Almost a carbon copy of Doom and Heretic code with few exceptions. At the moment of writing this PR, all rendering initials and nominals should be formed, and despite of massive changes this is an easy part. Hard part is:

• Extra color panes to support 13-27 palette indexes. And what about formulas?
• Foggy maps should use pregenerated colormap table + there is something unclear with brightest levels, see my comment at p_setup.c.
• Fade in / fade out effects for finale screen. We can't operate with PLAYPAL values, but we can fade screen with I_BlendDark in f_finale.c.
• I'm sure there can potentially be couple of underwater mines in ACS scripts.

[JNechaevsky]

Looks like we're on our own with Hexen palettes. I can't properly read Slade's approach for color blending, and we need just two magical RGB+alpha variables, not whole blending formula.

It's not that hard to replicate GZDoom approach, like by using substruction of extra palette idexes's zero color from original zero index one. But... Such approach is not really looks like vanilla, let's have a look:


And what I have achieved. Left part of the screenshot is paletted render, right is true color. There is barely notable dark line, so click to enlarge.


Not sure if I can get 100% identical to vanilla colors, but at least I can do my best to replicate them as close as possible.

[kitchen-ace]

Looks like we're on our own with Hexen palettes.

I don't know if it helps but doomwiki describes the additional Hexen palettes:

https://doomwiki.org/wiki/PLAYPAL#Hexen

[JNechaevsky]

🥲... It will help a lot, thank you! I was using slightly different technique by picking tinted color from 14-27 palette's 0 (black) color and then spent hours by fine-tuning values and comparing to paletted render. According to the article, Wraithverge and Bloodscourge using different colors for every palette which is not good to keep things clear here, as we need exactly one pane for such colors, alpha value will do the rest, but ice and notably poison must get another revision.

[kitchen-ace]

It will help a lot, thank you! I was using slightly different technique by picking tinted color from 14-27 palette's 0 (black) color and then spent hours by fine-tuning values and comparing to paletted render.

Agh, sorry I couldn't reply earlier!

[JNechaevsky]

Small experiment with emulating FOGMAP in true color. It's slightly different from original table, but at least gives more or less decent colors.

Before:


And after:


[fabiangreffrath]

Looks good, much better than expected!

[JNechaevsky]

Thank you! In current implementation it looks like this: ...while vanilla table is:

I.e. we have more smooth linear fading. Fortunately, Raven did foggy maps with keeping light levels in mind, so we still have a more or less proper view, except we can see slightly farther. I think it is possible replicate vanilla table more accurately, but it needs some patience for playing with for verctors in colormap generating. Value 147 is not a magic number, it's fading color or original FOGMAP:


Remaining things to do:

• Fading in/out on finale screens. Not sure it is possible to replicate this trick on true color, but we still can use I_BlendDark for such purposes, something like in this implementation. I just have to figure out how to do it properly, since it doesn't seems to be very straight forward.
• Implement "alt" blending effect. Turns out, it's slightly more opaque than standard emulated tintmap blending, probably for about 150 or 200%? I need to experiment and verify.

P.S. It's a miracle, but vanilla bug with missing first palette in various effects is a godsend here. Should it doesn't happen, we'll Doom rad suit palette (index 13) for lesser poison effect in Hexen! 😱

[JNechaevsky]

"Alt" TINTTAB was easy. Paletted:


And True color:


Before trying to make a final step with finale screen, I need to revisit patch drawing functions, as things gets more complicated. Turns out, we have not 2, but 3 different translucency levels for Raven games. Let's see original code...

🤯	Function	Code	What does it do
1	V_DrawShadowedPatch	*dest2 = tinttable[((*dest2) << 8)];	translucent shadow
2	V_DrawTLPatch	*dest = tinttable[*dest + ((*source++) << 8)];	translucent sprite, less opaque
3	V_DrawAltTLPatch	*dest = tinttable[((*dest) << 8) + *source++];	translucent sprite, more opaque

[fabiangreffrath]

• Fading in/out on finale screens. Not sure it is possible to replicate this trick on true color, but we still can use I_BlendDark for such purposes

Yes, you should be able to incrementally fade in/out the finale screens if you draw the fullscreen patch as usual and then brush over each pixel with I_BlendDark() as in your menu shading code. You just need to adjust the second argument passed to I_BlendDark() each time FadePic() is called.

[JNechaevsky]

I've done exactly this way. 🙂 After few hours of experiments, I finally got less diff + possible framerate safe approach. Framerate safe means: we have to rely on finale tics, not to framerate tics, just in case one day framerate becomes uncapped in any states. The result is nearly 1:1 to vanilla, except colors are more smooth while fading. Durations seems to be also almost same, but probably differs just for few frames.

Ah yes, such approach is also safer than vanilla one, since we not touching palette at all.

Here it is, in full quality. Video compression probably will destroy some colors a little bit, so I highly recommend to download video or check it out in the game. https://drive.google.com/file/d/1V2OXl4wHTfnmqVhMx7ZBZOvf5M_m0Kui/view?usp=sharing

[fabiangreffrath]

Looks great, congratulations to this achievement and thank you very much for your work on this!

[JNechaevsky]

It is my highest pleasure to do this! Wish I say "we won!", but I can't, there are two nitpicks remaining:

1) I'm not very satisfied with I_BlendDark calling on finale screens. It is working fine, but such blending is CPU-expensive, especially when covering whole screen area. We are capped at 35 fps there, but it's probably worth to don't be that lavish, i.e. call blending function only while fading is happened. I need to experiment more. 2) When true color is compiled in but disabled as config option, brightest white color becomes screwed up. My first though it was a problem of HSV coloring (not happening in Crispy), but can be notified on finale screens: screenshot 1 and screenshot 2, testing sprite I'm using, replaced Fighter's gloves: white.zip

After farther investigation, I have found that color gets broken on normal maps (screenshot, VISIT01), but not on foggy ones (screenshot, VISIT08). Something wrong must be here, but if colormaps aren't generated on level loading, bright color levels will be screwed up even more (screenshot).

[JNechaevsky]

Looks likes it's not a true color code issue at all, it's a mistake made by Raven Software. Have a look at Hexen's PLAYPAL. See? Last one color (255, bottom-right) is white.


Now have a look at original COLORMAP. Last one color (most-right) is BLACK!

And now to FOGMAP, where last one color is proper white:

That's must be the cause. The only possible way must be to forcefully swap black 255 color with white color somehow. If no check for modified COLORMAP will be performed, it's another "one step back" for true color's compatibility with custom colormaps. But it's already in "few steps back" condition...

[JNechaevsky]

Completely mindless, but do the trick. Even no need to call V_GetPaletteIndex(playpal, 255, 255, 255).

    for (c = 0; c <= NUMCOLORMAPS; c++)
    {
        for (i = 0; i < 256; i++)
        {
            if (i == 255)
            {
            r = 255; g = 255; b = 255;  // [JN] THANKS, COME AGAIN !!!
            }
            else
            {
            r = gamma2table[crispy->gamma][playpal[3 * colormap[c * 256 + i] + 0]] & ~3;
            g = gamma2table[crispy->gamma][playpal[3 * colormap[c * 256 + i] + 1]] & ~3;
            b = gamma2table[crispy->gamma][playpal[3 * colormap[c * 256 + i] + 2]] & ~3;
            }

            colormaps[j++] = 0xff000000 | (r << 16) | (g << 8) | b;
        }
    }

[fabiangreffrath]

If you want to be a bit "more correct", instead of taking palette value i as base color, take the palette color that colormaps[i] from the COLORMAP lump points to as base color.

[fabiangreffrath]

TODOs left for Doom and Heretic:

• factor out HSV table generation into separate function only called once (Doom, Heretic)
• generate truecolor colormap from palette values passed through first original colormap (Doom, Heretic)
• generate pal_color array from original palette, wirhour going through colormap (Doom, Heretic)
• replace all remaining truecolor-only occurences of colormaps[] with pal_color[] (Doom, Heretic, Hexen)

[fabiangreffrath]

Possible optimization:

• If we create the pal_color[] array before the colormaps[] array, we could use the values in the former for the evaluation of the values in the latter. But, well, this would probably optimize away a few microseconds if at all. As the code currently is, it contains a lot of redundancies, but it is already complicated enough and in this case, readability and maintainability probably beats performance.

[JNechaevsky]

I have to do a couple of conclusions. Nothing special really, just a few thoughts for the record.

Common

• Most of the code comes from Doom and Heretic. Second one made things a lot easier: automap, RAW screens and translucency functions, which, finally, has been verified and corrected.
• Current implementation provides reasonable compatibility with custom COLORMAP lump, but not with any custom per-map color maps, they always will use iwad's FOGMAP, matter no what kind of name is proved in MAPINFO, i.e.:

  map 8 "DARKMERE"
  warptrans 8
  next 9
  cluster 2
  sky1 SKYFOG2 20
  sky2 SKYFOG 60
  doublesky
  fadetable fogmap // <-- ☺ I can be even pamgof in true color, ha-ha! ☻
  cdtrack 14

  The technical problem is because of despite fadetable is obivuosly a char type, it's handed as int here. In theory, it can be handled as char, but then few extra ifndef/else will be needed, and we will get nothing more than possibly yucky colormap in neat true colors.

Shoudn't be a real problem, as @kitchen-ace suggested that there no real mods which is using custom colormaps, and I know only one mod at all - Caldera, appeared many years ago in Chocolate tracker. And good thing nowadays we have a different port for such purposes, which is allowing to use RBG values for sector lighting and hardware fog instead of that colormap's ancient technology... Ancient technology which we have to hold dear and handle with care to provide backwards compatibility and vanilla look&feel in general.

f_finale.c

• We are all good, truecolor blending function is invoking only while fading effect is happening, as well as whole patch drawing function. When fading stops, then buffered drawing is used, which is a great pleasure for CPU, made by Raven Software.
• To follow less-diff way, I've made unused functions empty, but not rejected their calls. Otherwise the code will turn into ifndef/else/endif spaghettis.

i_video.c

• Four extra panes was as small price as possible. This makes fading effects (not initial one) of cleric and mage BFG a little-little bit different from vanilla, but to make them use vanilla values, then we'll need four additional panes, as such effect are using slightly different RGB colors. This is definitely too much, and I tried to fine-tune those fading to represent vanilla effects via opacity value. Not sure that difference will be noticed at all.
• Manual handling of every custom palette in I_SetPalette(). Ideally, it should be shortened to similar formulas you have whore for Doom to make code a bit smaller, but you know me, I'm bad not only with mathematics, but in arithmetics as well. But at least we have almost identical colors. Kind thanks to @kitchen-ace for providing a link to DoomWiki. My initial attempt to recreate those palettes took 8 hours in a row, by comparing frame-by-frame, then tuning and fine-tuning, and surprisingly, I've got very close results. And then spent next two days on headache painkillers, but hey, no one forced me to perform this in such maniacal "sit and stare until it's done" way, I just was too excited to make it, and make it right. 😏