search

KhronosGroup / KTX-Specification

KTX file format source
Other
70 stars 12 forks source link

DFD transfer function vs vkFormat #214

Closed MarkCallow closed 2 weeks ago

MarkCallow commented 3 weeks ago

@lexaknyazev I'd very much appreciate your input regarding the following.

For the correspondence of the DFD's transfer function with a KTX file's vkFormat field the spec says:

I am modifying ktx create to be compliant with the second bullet and I have some questions:

  1. Should VK_FORMAT_ASTC_*_SRGB_BLOCK be considered a variant of VK_FORMAT_ASTC_*_SFLOAT_BLOCK? It is obviously a variant of VK_FORMAT_ASTC_*_UNORM_BLOCK.
  2. Should we be allowing non-linear transfer functions with unnormalized formats such as [SU]INT. If yes should we allow sRGB if there is no sRGB variant?
  3. Same question for the SFLOAT formats.
  4. Are there any formats for which we should forbid non-linear transfer functions, e.g depth and stencil formats.

I think the answer to Q2 is no to non-linear transfer functions because [SU]INT are not normally used for color data.

I think the answer to Q3 is yes to non-linear transfer functions but am unclear on whether to allow sRGB. If sRGB should be allowed then should we not also allow it for VK_FORMAT_ASTC_*_SFLOAT_BLOCK?

Not spec-related but relevant to this discussion so I'll ask it here: should we add decode/encode of other non-linear formats to ktx create? If so, which? Currently it has ITU (ITU BT.601, BT.709 and BT.2020), ITU BT.2100 PQ, sRGB and gamma decoders and encoders. Note that the command's --convert-oetf option only lets you choose linear or srgb. The ITU and ITU BT.2100 PQ decoders are only used internally to process input files. Should --convert-oetf allow more choice of conversions for use with non-sRGB formats? If so, which?

lexaknyazev commented 3 weeks ago
  1. Should VK_FORMAT_ASTC_*_SRGB_BLOCK be considered a variant of VK_FORMAT_ASTC_*_SFLOAT_BLOCK? It is obviously a variant of VK_FORMAT_ASTC_*_UNORM_BLOCK.

No. ASTC HDR is a superset of ASTC LDR so swapping the format enum is not possible (unlike in the UNORM case).

  1. Should we be allowing non-linear transfer functions with unnormalized formats such as [SU]INT. If yes should we allow sRGB if there is no sRGB variant?

I'm leaning towards saying that transfer function is irrelevant for integer formats and suggesting KHR_DF_TRANSFER_UNSPECIFIED as the unambiguous default (linear is not semantically applicable in my opinion). There's no way to do any reasonable color management with pure integer values.

  1. Same question for the SFLOAT formats.

Assuming that UFLOAT formats are also included in this question.

Applying non-linear transfer functions to these is generally redundant but may make sense for small floats to further redistribute numeric precision. That said, many transfer functions are undefined for values outside of the [0; 1] range, which makes them unusable in practice. I think it's still fine to allow them (including sRGB for block-compressed HDR formats) with a warning.

  1. Are there any formats for which we should forbid non-linear transfer functions, e.g depth and stencil formats.

As with Q2, I think that transfer function is irrelevant for depth and/or stencil data.

The ITU and ITU BT.2100 PQ decoders are only used internally to process input files.

For which inputs is the ITU decoder used?

Should --convert-oetf allow more choice of conversions for use with non-sRGB formats? If so, which?

It's inevitable in the log run but does not seem very important as of today.

MarkCallow commented 3 weeks ago

@lexaknyazev Thanks for confirmation on ASTC HDR. I'll add a note to the spec. Agree w.r.t. integer, depth and stencil formats. I'll look over the spec. again and see if it already says this. Maybe I missed it. If it does I'll add a reference to it from the section I quoted above. If not I'll update the quoted section's language.

Re [SU]FLOAT formats I thought they might be used in some HDR/wide gamut formats and so transfer functions might be of interest but looking at the relevant TV standards just now that does not seem to be the case. Nevertheless I agree its fine to allow them.

MarkCallow commented 3 weeks ago

For which inputs is the ITU decoder used?

ITU is used when reading NetPbm files to decode and then re-encode to sRGB. I don't know where the BT.2100 PQ decode and encode are used. I was surprised to find them. I'll have to peruse the code.

lexaknyazev commented 3 weeks ago

BT.2100 PQ should be used with PNG as described in https://www.w3.org/TR/png-hdr-pq/.

However the latest revision of that document says that this workflow is now deprecated, see https://w3c.github.io/png-hdr-pq/.

MarkCallow commented 3 weeks ago

@lexaknyazev, Re. ASTC_*_SFLOAT, even though ASTC_*_SRGB is not considered a variant, it doesn't seem reasonable to allow sRGB data and transfer function for them as the decoding, and probably the encoding, will not take into account the non-linear nature of the data.

lexaknyazev commented 3 weeks ago

it doesn't seem reasonable to allow sRGB data and transfer function for them

The rationale is similar to allowing sRGB transfer function for unorm formats like RGB565. Even if it's not natively handled by GPU texture samplers (same applies to PQ, HLG, etc...), there is nothing semantically wrong with such combinations.

MarkCallow commented 2 weeks ago

Even if it's not natively handled by GPU texture samplers

I was referring to the ASTC encoder and the ASTC decoder/sampler. The libktx encoder is going to need some work to support a potential sRGB transfer function in ASTC_*_SFLOAT formats. Currently it forces LDR when the incoming transfer function is sRGB. Fortunately the work can be delayed until HDR support is added to libktx.