Clarify the use of color volume metadata

palemieux commented 1 year ago

specify that color volume metadata defines the nominal color volume of the image content
specify how mastering display metadata can be used to set color volume metadata
provide an example of how color volume metadata is used when tone mapping
specify that color volume metadata should be used when rendering a temporal sequence of images

Closes #97

LBorgSMPTE commented 1 year ago

What if both ST2086 max luminance and (HDR10) MaxCLL are available? Which one to use? (Use of MaxCLL over max luminance might be compatible with HDR10 TV sets. Pls confirm)

swick commented 1 year ago

We're also using mastering display metadata in the wayland color management protocol to let clients define the content color volume. One issue that we're not sure how to handle is differences in the white point of the color space and the mastering display white point. It doesn't seem to be specified anywhere if the mastering display volume is supposed to be chromatically adjusted to the color space or not.

cc @ppaalanen

ppaalanen commented 1 year ago

Yes indeed, thanks for the CC.

I've always been assuming that no color gamut (or tone) mapping is happening between the content color encoding and mastering display, but now that I think of it, that too is just an assumption I have made. What justifies this assumption, or am I wrong?

The assumption seems to be required for critical viewing and mastering to be meaningful. But then, if color gamut mapping is not done (apart from clipping, perhaps), why should white point chromatic adaptation be done? And if it is done, then how is it done?

After all, doing chromatic adaptation seems to be the rule whenever white points differ, but is this an exception?

These questions apply when one wants to fill in a color volume description based on a mastering display description.

svgeesus commented 1 year ago

It doesn't seem to be specified anywhere if the mastering display volume is supposed to be chromatically adjusted to the color space or not.

It seems clear that if the mastering color volume uses a different white point to the color space, then it should be chromatically adapted so that the corresponding color volume in the color space is known. Within reason, white will always appear white but the appearance of other colors will change, and that is precisely what chromatic adaptation accomplishes (prediction of corresponding colors).

ppaalanen commented 1 year ago

It doesn't seem to be specified anywhere if the mastering display volume is supposed to be chromatically adjusted to the color space or not.

It seems clear that if the mastering color volume uses a different white point to the color space, then it should be chromatically adapted so that the corresponding color volume in the color space is known. Within reason, white will always appear white but the appearance of other colors will change, and that is precisely what chromatic adaptation accomplishes (prediction of corresponding colors).

I think you are assuming that the viewer is adapted to the mastering display's white point. Can we assume that? What does the viewer in a mastering environment adapt to? Is it not mostly the monitor contents rather than monitor physical build (monitor white point) or the surround? What would give the monitor white away to the viewer?

Counter-example: night light; nothing gives the monitor white away, so the viewer adapts to what content depicts as white, so white content looks white even if it is yellow/reddish compared to monitor white.

OTOH, would one not intend to show content colorimetry as-is on a mastering display? Meaning no chromatic adaptation, no gamut mapping, no tone mapping?

Or maybe no-one is foolish enough to have a mastering display with a different white point than the content encoding, so that this question never comes up in the first place?

Would be nice to have some inside information from the industry here, how do they really do things.

My feeling is that mastering is not equivalent to end user viewing. End user viewing wants to get the best impression out of content with whatever equipment they happen to have, while mastering is about accurately inspecting the content as it is and tuning the content rather than its presentation to look intended.

palemieux commented 1 year ago

(Use of MaxCLL over max luminance might be compatible with HDR10 TV sets. Pls confirm)

MaxCLL is supposed to match the content, so it is probably a safer value.

LBorgSMPTE commented 1 year ago

1,000 nits for minimum?

On 5/30/23, 11:08 AM, "Pierre-Anthony Lemieux" @.***> wrote:

EXTERNAL: Use caution when clicking on links or opening attachments.

@palemieux commented on this pull request.

In hdr_html_canvas_element.md:

-obtained from metadata contained in a source image, and omitted otherwise. +colorVolumeMetadata specifies the nominal color volume occupied by +the image content in the CIE 1931 XYZ color space. The boundaries of the color +volume are defined by: + + the xy coordinates, as defined in ISO + 11664-3, of three color primaries: + redPrimaryX, redPrimaryY, greenPrimaryX, greenPrimaryY, + bluePrimaryX, and bluePrimaryY; + the xy coordinates of a white point: whitePointX and whitePointY; and +* a minimum and maximum luminance in cd/m²: minimumLuminance and maximumLuminance. + +If omitted, chromaticity is equal to the chromaticity of the color space of +the Canvas. + +If omitted, minimumLuminance is equal to 0. I think 1,000 is probably a safer value.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

LBorgSMPTE commented 1 year ago

Unlikely to be exceeded, yes, but also extreme and useless.

1,000 is more useful, and already the default for HDR10, i.e. PQ, and HLG

Lars

On 5/30/23, 6:55 PM, "Pierre-Anthony Lemieux" @.***> wrote:

EXTERNAL: Use caution when clicking on links or opening attachments.

@palemieux commented on this pull request.

In hdr_html_canvas_element.md:

-obtained from metadata contained in a source image, and omitted otherwise. +colorVolumeMetadata specifies the nominal color volume occupied by +the image content in the CIE 1931 XYZ color space. The boundaries of the color +volume are defined by: + + the xy coordinates, as defined in ISO + 11664-3, of three color primaries: + redPrimaryX, redPrimaryY, greenPrimaryX, greenPrimaryY, + bluePrimaryX, and bluePrimaryY; + the xy coordinates of a white point: whitePointX and whitePointY; and +* a minimum and maximum luminance in cd/m²: minimumLuminance and maximumLuminance. + +If omitted, chromaticity is equal to the chromaticity of the color space of +the Canvas. + +If omitted, minimumLuminance is equal to 0. In the balance, I think @svgeesus' proposal might be the least terrible: set the default maximumLuminance to 10,000, which is unlikely to be exceeded.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

ppaalanen commented 1 year ago

When this spec uses the unit cd/m², are you clear which viewing environment those values are relative to?

cd/m² is theoretically an absolute luminance, but any given absolute luminance value is appropriate as-is only in a specific viewing environment if the goal is a universally consistent perception of that luminance.

ppaalanen commented 1 year ago

1,000 is more useful, and already the default for HDR10, i.e. PQ, and HLG

When I want to make the same statement, which standard or report can I refer to? ITU, SMPTE, ...

palemieux commented 1 year ago

When this spec uses the unit cd/m², are you clear which viewing environment those values are relative to?

This is in the context of the viewing environment specified in BT.2100.

palemieux commented 1 year ago

One issue that we're not sure how to handle is differences in the white point of the color space and the mastering display white point.

The mastering display white point (as used in SMPTE ST 2086 et al.) and in this strawman are merely intended to characterize the volume (in CIE xy space) spanned by the pixels within the image -- for the purpose of generating stable and optimal tone mapping. These white points are not related to any scene or reference viewing environment illuminant.

This strawman specifically does not use the term "mastering display" to avoid confusion.

Makes sense?

palemieux commented 1 year ago

1,000 is more useful, and already the default for HDR10, i.e. PQ, and HLG

Does HDR10 constrain the mastering display and/or the image pixel luminance?

ppaalanen commented 1 year ago

The mastering display white point (as used in SMPTE ST 2086 et al.) and in this strawman are merely intended to characterize the volume (in CIE xy space) spanned by the pixels within the image -- for the purpose of generating stable and optimal tone mapping. These white points are not related to any scene or reference viewing environment illuminant.

This strawman specifically does not use the term "mastering display" to avoid confusion.

Makes sense?

Yes, that's the fundamental description of what those parameters are. We got up to that point in the Wayland protocol design too. It is kind of enough for an interface specification.

The open question is that we do not know what to do with those numbers. How do you compute a volume in, say, the signal encoding space from the mastering parameters? How are they used to drive color gamut mapping? We haven't found good, or any, references for that yet. I don't know how to handle those parameters in a compositor, which makes the interface design... kind of blind. I was hoping you would have contacts to find out, because I believe you will have the same questions. Maybe it's not a topic for this document under review but for another.

Our Wayland discussions so far are stuck with the question at https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/18 .

ppaalanen commented 1 year ago

Does HDR10 constrain the mastering display and/or the image pixel luminance?

All I know about HDR10 is in https://gitlab.freedesktop.org/pq/color-and-hdr/-/blob/main/doc/hdr10.md#hdr10-media-profile . It does seem to suggest delivering all that metadata.

LBorgSMPTE commented 1 year ago

This function is misnamed.: 1/2.4 is for 709 displays, not sRGB.
The matrix doesn’t clip, so Pow 1/2.4 will fail for negative values. Seems this needs a clip after the matrix.

Lars

On 6/1/23, 4:21 AM, "Chris Lilley" @.***> wrote:

EXTERNAL: Use caution when clicking on links or opening attachments.

@svgeesus commented on this pull request.

In hdr_html_canvas_element.mdhttps://github.com/w3c/ColorWeb-CG/pull/98#discussion_r1213235696:

+function rec2100PQtoSRGB(r, g, b) {

let rt = 10000 * pqEOTF(r) / 203;
let gt = 10000 * pqEOTF(g) / 203;
let bt = 10000 * pqEOTF(b) / 203;
[rt, gt, bt] = matrixXYZtoRec709(matrixBT2020toXYZ(rt, gt, bt));
const rp = Math.pow(rt, 1/2.4);
const gp = Math.pow(gt, 1/2.4);
const bp = Math.pow(bt, 1/2.4);

return [rp, gp, bp];

}

Good point, this is indeed a colorimetric conversion.

In CSS Color 4, all the predefined RGB spaces are defined over the extended range. We don't support 709 but do support sRGB over the extended rangehttps://drafts.csswg.org/css-color-4/#predefined-sRGB, as an example.

If had clipping is expected, then the stage at which it occurs should be specified and the choice of hard clip (with associated lightness and hue changes) justified.

— Reply to this email directly, view it on GitHubhttps://github.com/w3c/ColorWeb-CG/pull/98#discussion_r1213235696, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASRHAL7HZ6H4SUAKVWSKDMDXJCQMZANCNFSM6AAAAAAYEKRXLU. You are receiving this because you commented.Message ID: @.***>

LBorgSMPTE commented 1 year ago

I suggest calling it ColorPrimaries SMPTE ST 2086, which this is based on, calls it Display Primaries.

On 6/1/23, 4:22 AM, "Chris Lilley" @.***> wrote:

@svgeesus commented on this pull request.

In hdr_html_canvas_element.mdhttps://github.com/w3c/ColorWeb-CG/pull/98#discussion_r1213237116:

}



 ```idl

-  dictionary ColorVolume {

+  dictionary Chromaticity {

ColorPrimaries would indeed be a better term here. Or perhaps Chromaticities, the plural.
—
Reply to this email directly, view it on GitHub<https://github.com/w3c/ColorWeb-CG/pull/98#discussion_r1213237116>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ASRHAL7QMOPPJVYEQJSERIDXJCQQZANCNFSM6AAAAAAYEKRXLU>.
You are receiving this because you commented.Message ID: ***@***.***>

palemieux commented 1 year ago

The open question is that we do not know what to do with those numbers.

@ppaalanen The numbers can be used to drive the rendering of the image to the ultimate display and ensure that the rendering algorithm is stable over a sequence of images -- since the rendering algorithm might depend on the contents of the image, i.e., mapping to a 400 nits monitor an image with pixels that range from 0 to 300 nits would ideally be different than mapping an image with pixels that range from 0 to 10,000 nits. Same for the color gamut.

The demo at https://www.sandflow.com/public/tone-mapping illustrates the use of minLuminance and maxLuminance to drive the tone-mapping algorithm.

w3c / ColorWeb-CG

Clarify the use of color volume metadata #98