NormalMapEffect/PBREffect support for normal texture encoding options

[walbourn]

Currently the NormalMapEffect assumes the normal-map is always a BiasX2 decode, which means you should always use a UNORM or positive-only format (R11G11B10_FLOAT).

Pixel-shader variants should be provided for other encodings:

• signed (i.e. no bias needed)
• reconstructing from BC5_UNORM / BC5_SNORM (aka 3Dc)
• reconstructing from BC3 (aka DXT5nm)

Understanding BCn Texture Compression Formats

Real-Time Normal Map DXT Compression

[walbourn]

The DXT5nm solution is really only needed for feature level 9.x. 3Dc (BC5) is better quality.

[walbourn]

There are eight possible shader scenarios here:

• Signed
• Unsigned: BiasX2
• 2-Channel: z = sqrt( 1. - dot( xy, xy ) );
• Unsigned 2-Channel: BiasX2 then z = sqrt( 1. - dot( xy, xy ) );
• Signed (green channel inverted)
• Unsigned (green channel inverted)
• 2-Channel (green channel inverted)
• Unsigned 2-Channel (green channel inverted)

The main value of the signed variants is that 0.5 can be exactly represented which unsigned formats can't do.

The green channel inverted scenarios are because there's both 'OpenGL' and 'DirectX' conventions where Y is either up or down depending on the pipeline, handedness, etc. Right now I'm assuming the green never needs inverted.

[walbourn]

Note that glTF2 uses the BiasX2 for normal maps (since they are always stored as PNG). As such, I think signed scenarios are not all that interesting.

[walbourn]

Since these effects only support tangent-space normal mapping, a reasonable solution is to just always reconstruct the z which works with either RGB or BC5.

[walbourn]

I've added reconstruction for the Z channel to the shaders in this commit to support both scenarios.

[walbourn]

To keep the shader permutations down, I'm going to leave the inverted green channel as a content issue. I added a -inverty switch to texconv to help with this scenario.

[walbourn]

To sum up:

• DirectX Tool Kit built-in normal map/PBR effects only support unsigned/bias x2 scenarios (i.e. _UNORM formats)
• The built-in effects are intended for tangent-space mapping so they always reconstruct the Z channel (which both improves quality and supports normal texture compression with the _BC5_UNORM format)
• The green channel is used "as is", so if it needs inverted the texture content should be modified

All of this is captured in the wiki