Improve color space handling with supercompression & transcoding

[lexaknyazev]

Here's a full description of how color space information traverses KTX pipeline with supercompression from the source image to the GPU texture.

Source PNG

As per the PNG spec, each PNG image has a color space defined by zero or more color space information chunks (sRGB, gAMA, cHRM, and iCCP):

• No color-info chunks:
  • an implementation should assume sRGB (or 2.2 gamma) transfer function with sRGB color primaries.
• sRGB chunk is present:
  • use the sRGB transfer function, the sRGB primaries, and (optionally) the intent specified in the chunk, ignore gAMA and cHRM chunks.
• iCCP chunk is present:
  • use the ICC profile provided in the chunk, ignore gAMA and cHRM chunks.
• gAMA and/or cHRM chunks are present without sRGB or iCCP:
  • use the provided gamma value and color primaries.
• sRGB and iCCP chunks shall not be present simultaneously.

KTX software follows these rules and infers the source image color space with additional limitations:

• Custom ICC profiles are not supported.
• Custom color primaries are not supported.
• Custom gamma values except 1.0 and 2.2 are not supported.
• 2.2 gamma transfer function is treated as sRGB.

When the source image actually contains linear values (e.g., it's a normal map) but the PNG file doesn't signal that by using the appropriate combination of chunks, users can override KTX software behavior by providing --linear parameter to the command-line tool.

The resulting KTX2 file always has the color space information in its DFD section.

Basis Supercompression

Basis compressor has an artifact-reducing optimization, enabled by settingm_perceptual in the compressor parameters, which handles the original pixel values as perceptual colors, i.e. sRGB. By default they are treated as linear which is suitable for arbitrary vectors as well as linear colors. The KTX software sets this parameter when transcoding textures with the sRGB bit set in the DFD section. It does not expose the parameter as a user setting.

Runtime Texture Transcoding

At runtime, a GPU texture object of appropriate format needs to be created to receive transcoded block-compressed data. Its format may or may not imply hardware sRGB sampling and filtering, so depending on the platform, applications may need to plan for some fallback behavior.

When the GPU texture is created by the KTX library, it may need to know that hardware sRGB is not available and the conversion will happen in shaders.

Here is how platform support for sRGB filtering of block-compressed formats looks like.

ASTC

On platforms that support ASTC, each combination of block dimensions exists in both linear and sRGB options.

BC1, BC2, BC3

• OpenGL Desktop extensions:
  • EXT_texture_compression_dxt1 (BC1 only, linear).
  • EXT_texture_compression_s3tc (BC1/2/3, linear).
  • EXT_texture_sRGB (BC1/2/3, sRGB).
  • When the latter extension is not available, an application has to decode sRGB in a fragment shader.
• OpenGL ES extensions:
  • EXT_texture_compression_dxt1 (BC1 only, linear).
  • EXT_texture_compression_s3tc (BC1/2/3, linear).
  • ANGLE_texture_compression_dxt (BC1/2/3, linear).
  • EXT_texture_compression_s3tc_srgb (BC1/2/3, sRGB).
  • When the latter extension is not available, it may be possible to override linear sampling by using EXT_texture_format_sRGB_override extension. Otherwise, an application has to decode sRGB in a fragment shader.
• WebGL extensions:
  • WEBGL_compressed_texture_s3tc (BC1/2/3, linear).
  • WEBGL_compressed_texture_s3tc_srgb (BC1/2/3, sRGB).
  • When the latter extension is not available, an application has to decode sRGB in a fragment shader.

Vulkan, Metal, and Direct3D have native support for BC1/2/3. Direct3D and Metal support only alpha-"punchthrough" flavor of BC1.

BC7

• OpenGL ES and WebGL:
  • EXT_texture_compression_bptc extension (linear & sRGB).
• OpenGL Desktop:
  • ARB_texture_compression_bptc extension (linear & sRGB).

Vulkan, Metal, and Direct3D have native support for both variants of BC7.

ETC RGB/RGBA

Originally, this format supported only linear sampling and filtering of RGB without alpha. Given that ETC1 RGB format is a strict subset of ETC2 RGB format, and the latter supports hardware sRGB sampling and filtering, it became possible to use hardware sRGB sampling and filtering with existing ETC1 data on capable hardware.

• OpenGL Desktop:
  • ARB_ES3_compatibility (ETC2, linear & sRGB).
• OpenGL ES:
  • OES_compressed_ETC1_RGB8_texture (ETC1, linear only).
  • Open GL ES 3.0 core spec (all ETC2 formats).
  • When the OpenGL ES version is older than 3.0, an application has to decode sRGB in a fragment shader.
• WebGL:
  • WEBGL_compressed_texture_etc1 (ETC1, linear only).
  • WEBGL_compressed_texture_etc (ETC2, linear & sRGB).

PVRTC1/2

• No support in OpenGL Desktop.
• OpenGL ES extensions:
  • IMG_texture_compression_pvrtc (PVRTC1, linear).
  • IMG_texture_compression_pvrtc2 (PVRTC2, linear).
  • EXT_pvrtc_sRGB (PVRTC1/2, sRGB).
  • When the latter extension is not available, an application has to decode sRGB in a fragment shader.
• WebGL extensions:
  • WEBGL_compressed_texture_pvrtc (PVRTC1, linear).
  • An application always has to decode sRGB in a fragment shader.
• Vulkan extensions:
  • VK_IMG_format_pvrtc (PVRTC1/2, linear & sRGB).
• Metal supports only PVRTC1 (linear & sRGB).
• No support in Direct3D.

BC4/5, EAC R11/RG11

These formats support only linear sampling and filtering.

[MarkCallow]

There is nothing to be done in the current KTX-Software as far as I can see. This useful material needs to go in a document somewhere. Where would be appropriate?

I've updated the section about the "colors" optimization in the above. Please check it.