Add srgb-linear, display-p3-linear, and rec2020-linear color spaces

[ccameron-chromium]

Add the srgb-linear color space, since it has been requests by several potential users.

While updating that enum, fix the name "rec-2020" to be "rec2020". This name is consistent between CSS Color Module Level 4 and the gamut media queries API.

Add a note that there is a naming inconsistency between "sRGB" (color level 4) and "srgb" (media queries), and that srgb-linear is absent from color level 4 (but should be added).

[kdashg]

This is two changes, so it'd be nice to have two commits.

I need more description of what "srgb-linear" is, and why it's wanted.

[ccameron-chromium]

I'd prefer this be a single commit -- both changes are to the same enum, utting this into two commits (that will conflict) will decrease velocity, and two commits will not improve clarity.

I added a note that srgb-linear is not defined by CSS color level 4 and that it needs to be. Linearized sRGB is a standard color space for physically-based rendering. See:

• kCGColorSpaceExtendedLinearSRGB in CoreGraphics
• DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709 in DirectX
• EGL_GL_COLORSPACE_SCRGB_LINEAR_EXT in OpenGL

[kdashg]

It's better to have two commits because I'll happily merge the rec2020 naming one, but we need to talk more about srgb-linear.

[ccameron-chromium]

Sorry, I forgot in my earlier comment to mention the Vulkan version of the color space: VK_COLOR_SPACE_EXTENDED_SRGB_LINEAR_EXT

What are the questions about srgb-linear?

[kdashg]

We need more info on usecases, and why it's needed for them. We have had descriptions of the value e.g. srgb-encoded 8-bit provides, so it would be good to likewise have a compelling description here. We also need to evaluate how we'd implement it.

[ccameron-chromium]

We need more info on usecases, and why it's needed for them. We have had descriptions of the value e.g. srgb-encoded 8-bit provides, so it would be good to likewise have a compelling description here.

I don't see a section describing the use-cases for each color space in the spec -- could you give an example of what you have in mind?

The primary use case for srgb-linear is with float16 textures, for physically-based rendering.

Light transport equations work on linear (not sRGB-encoded) light values, and so physically-based rendering prefers to be done in linear space. There exist extensions for adding a color encoding function that will allow for shader math to be done in linear space (with conversion back to sRGB-encoding done before sampling and after blending). Those extensions only work for 8-bit texture formats.

When using a floating-point values, such sRGB-encoding extensions are no longer needed, because floating-point has sufficient precision (even at 16 bits).

Floating-point values allow the user to specify values outside of the range of [0, 1]. This allows for wide color gamut content (because all colors in all gamuts are expressible). This also allows for high dynamic range -- e.g the value (2,2,2) is exactly twice as bright as the value (1,1,1).

We also need to evaluate how we'd implement it.

Any implementation that supports srgb, display-p3, and rec2020 color spaces should be able to trivially add support for srgb-linear. The only difference between srgb and srgb-linear is that the transfer function is the identity (and any implementation that supports rec2020 as defined in https://www.w3.org/TR/css-color-4/ has to support custom transfer functions, because rec2020 has a gamma=2.4 transfer function).

[kdashg]

It's sort of confusing (for me?) to talk about "linear" because the term is overloaded. Shader math and blending are always always done entirely "linearly", at least in that e.g. 1.0-0.5 is 0.5. (pixel/gl formats/encodings are generally orthogonal to colorspace/transfer function) Gamma=2.2ish offers perceptual linearity, but not physical linearity.

So 'srgb-linear' is srgb chromaticities, with no (the identity) transfer function? (0.5 = half as physically bright, probably ~70% as perceptually bright?)

From an API standpoint it feels weird to me to asymmetrically add an srgb-without-gamma option without e.g. rec2020-without-gamma. In a previous proposal, this is where the transfer functions ("linearity") were split out from the chromaticities.

Is the idea "hey if we're using (signed) floats anyways, primaries are sorta irrelevant, so just pick any one set"?

[ccameron-chromium]

It's sort of confusing (for me?) to talk about "linear" because the term is overloaded. Shader math and blending are always always done entirely "linearly", at least in that e.g. 1.0-0.5 is 0.5. (pixel/gl formats/encodings are generally orthogonal to colorspace/transfer function) Gamma=2.2ish offers perceptual linearity, but not physical linearity.

So 'srgb-linear' is srgb chromaticities, with no (the identity) transfer function? (0.5 = half as physically bright, probably ~70% as perceptually bright?)

Yes, exactly.

From an API standpoint it feels weird to me to asymmetrically add an srgb-without-gamma option without e.g. rec2020-without-gamma. In a previous proposal, this is where the transfer functions ("linearity") were split out from the chromaticities.

Is the idea "hey if we're using (signed) floats anyways, primaries are sorta irrelevant, so just pick any one set"?

This is a good point. The idea was half "why bother giving so many options" and half that srgb-linear is the only format that is promotable to an overlay on both macOS and Windows (at least at this exact moment).

So I think you're right, we should add srgb-linear, display-p3-linear, and rec2020-linear. I also got this feedback from another direction recently.

WRT the idea of separating out the transfer function from the primaries as parameters, I'd prefer to limit the spec to named color spaces, and leave the door open to adding other parametric color spaces in the future.

I'll update the patch to include display-p3-linear and rec2020-linear.

[ccameron-chromium]

Were there any further thoughts on this? I've updated the spec to include the three relevant color spaces.

[kenrussell]

Merging now.

[kdashg]

I have not had a chance to review the changes.

On Thu, Jan 21, 2021, 10:54 PM Ken Russell notifications@github.com wrote:

Merged #42 https://github.com/WICG/canvas-color-space/pull/42 into master.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/WICG/canvas-color-space/pull/42#event-4237089006, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALHJDLM4XJ5PWWVY3RU2WLS3EOLDANCNFSM4WCYYCFA .

[kenrussell]

Apologies @jdashg , I didn't mean to step on your toes. There's a presentation on HDR and WCG Canvas at the ColorWeb CG next week and I wanted to get the latest spec changes in before that. Since you had reviewed actively before I thought that you were done reviewing. Do you want me to revert this?

[kdashg]

This change doesn't yet represent the consensus of the interested implementations here. I could have marked this approved but I have not yet done so. Procedurally, yes, this should be reverted.

I will review when I return (from pto) to office tomorrow. You have avenues for messaging me directly if it is urgent.

As a reminder, this should not be used as the basis of a presentation relating to standards without appropriate "not yet standards track" disclaimers.

On Thu, Jan 21, 2021, 11:12 PM Ken Russell notifications@github.com wrote:

Apologies @jdashg https://github.com/jdashg , I didn't mean to step on your toes. There's a presentation on HDR and WCG Canvas at the ColorWeb CG next week and I wanted to get the latest spec changes in before that. Since you had reviewed actively before I thought that you were done reviewing. Do you want me to revert this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/WICG/canvas-color-space/pull/42#issuecomment-765186174, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALHJDPJFXRCUTPWWNTCVADS3EQODANCNFSM4WCYYCFA .

[kenrussell]

I apologize again - have reverted the commit in #43.

The presentation was scheduled two weeks ago, during the last ColorWeb community group call. It's understood by all parties that this repository is work in progress, and will be presented as such.

@ccameron-chromium could I ask you to open a new pull request containing the last set of changes? Sorry for causing trouble.

[kdashg]

What I was expecting was something more like what we had before, where there's orthogonal colorspace and linearity, though my linearity enums were poorly worded. (also "perceptually linear" colorspaces would still need their respective different gammas, which is a little weird) Having foo and foo-linear for all foo? It's survivable but it doesn't feel clean. (not a hill I'd die on)

I would prefer a better name than "linear" here because it's a confusing term, even if it's commonly the term of art for one portion of devs. I like disambiguating via "perceptual" vs "physical" linearity.

I think that the compelling use-case for us here is blend-as-physical+output without extra processing. (NB: Physical blending will look bad in 8bpchannel or lower, but I wouldn't go out of our way to forbid this) That said, unfortunately IIRC float-blending is not really well supported on all hardware we run WebGL on, or even WebGL 2. If physically-linear blending isn't a requirement for an app it's (or it would be) already possible to output physically-linear-calculated values even via perceptually-linear output by doing tf compensation/inversion in the app's shader output. (for both float and unorm8 encodings)

I think there is may be entanglement here with the thoughts about "encodings" that I had in the past, that I want to consider. I need to think more about how this ties into GL's srgb formats, and blending/sampling there.

[kainino0x]

FYI, CSS WG has resolved to add srgb-linear: https://github.com/w3c/csswg-drafts/issues/6087#issuecomment-965594483

[sciecode]

I will apologize for perhaps intruding, but I have also stumbled upon this question when discussing future color management changes in https://github.com/mrdoob/three.js/pull/23936#issuecomment-1133193625. I believe to be able to contribute some considerations to the issue.

I think there is may be entanglement here with the thoughts about "encodings" that I had in the past,

I believe there is an unintended overload of the color-space interface in its current proposal, which I believe is causing some of the confusion on the matter. As kdashg noted it is not widely understood the differences between physically ( linear ) / perceptually "encoded" light.

Physically encoded ( i.e linear, additive color space - means the coordinate system is linear in light intensity, therefore, calculations can be done to accurately predict color mixing.

Perceptually encoded ( i.e gamma ) is not linear in light intensity, therefore, cannot be used to predict color calculations. This primarily occurs due to the fact that our eyes response to light is not linear with light intensity and vary in different regions on the spectrum.

The way we can map between physically or perceptually encoded data is defined by the color space itself, more specifically the gamut's primaries and the corresponding set of transfer functions:

• EOTF ( electro-optical transfer function ) physical > perceptual
• OETF ( optical-electro transfer function ) perceptual > physical

This is important because it means the color-space itself is gonna define which transfer functions will be used, but only the API's end users can determine if they need color data physically encoded ( expects further calculation, stored in high-dynamic range files ) or perceptually corrected ( render to canvas, store in uint8 files )

That being said, I would advise on considering the separation of color-space and "encodings" if it is expected to support physically encoded data in color-spaces other than sRGB.

[kdashg]

I don't think it's fair to simply categorize these into "accurate" vs "inaccurate", because it depends what you're hoping to do or get.

[kdashg]

The current state of my understanding is here: https://hackmd.io/0wkiLmP7RWOFjcD13M870A#Physically-Linear-Blending

[sciecode]

I don't think it's fair to simply categorize these into "accurate" vs "inaccurate", because it depends what you're hoping to do or get.

You are right, in that "correct" in this case is subject to expectations. I should say that it is common for color operations to be done in physically encoded data, in which you can expect the output color to be reflective of the way light intensity mixings are to be expected in the physical world ( optical system ).

There are certainly color calculations that can be done in perceptual encoded data in varying color spaces, but you cannot expect "physical correctness" In that sense of light intensity combinations, unless accounted for in the calculation itself.

[sciecode]

I would like to just further clarify that I can't quantify the overall expected usage of linear encoded colors in other color-spaces.

It is perfectly reasonable to establish sRGB as the only valid linear option and disregard the necessity of my suggestion. However if it is plausible for end users to request display-p3-linear, or rec2020-linear, then I find it pertinent to completely separate the current interface into the appropriated terms.

[sciecode]

In a final effort to demonstrate how the current API is inappropriate, and by reflection so is CSS Module Level 4 predefined color-space, I would like to show two simple examples. In both scenarios we'll be using the same image color data, but one encoded with sRGB-linear and the other with sRGB, because we don't even need multiple color-spaces to make this problem evident.

scenario | pre-context data | unpack transformation | inside context | pack transformation | post-context data

    1.   |   sRGB-linear    |     "srgb-linear"     |   sRGB-linear  |      "srgb"         |      sRGB   
    2.   |      sRGB        |     "srgb-linear"     |      sRGB      |   "srgb-linear"     |      sRGB

Now a quick explanation about both scenarios.

In the first scenario an user inputs physically encoded data into a context by unpacking with colorSpace: srgb-linear. Inside the context, data will be treated as sRGB-linear. And finally data will be packed by attributing canvas.colorSpace: sRGB, transforming the drawing buffer data to perceptual sRGB and correctly displaying the data as expected, by both the user and the specification intent.

However it might be surprising for some that the second scenario also produces the exact same result and yet it might not be intuitive by reading the specification. How? Let's try and dissect it.

In the second scenario, users have sRGB encoded image data and wants to input that into the context, however they want the data inside the context to also be sRGB encoded ( either because the shader algorithm used enforces it, or any other ulterior motive ). By reading the specification someone might be tempted to unpack using colorSpace: srgb, since we want the internal context in sRGB. However, that is not the case. In reality it would be expected for them to use colorSpace: srgb-linear. Why? because what is being called "colorSpace" by both canvas and css level 4 is not actually the target nor the source colorSpace of the data, it is in fact a name given for a transfer function - in other words. from-to function.

Turns out, what we are really after in this scenario is a NO-OP transformation. Because we are providing data already encoded in the target color-space/encoding. Turns out the same is true for when we are about to transform the data to be rendered to the drawing buffer. The key here is realizing that the no-op transformation was given the name srgb-linear, which is exactly my problem with the current proposal.

It is very important to realize that any packing or unpacking done to color data is always done with an initial-encoding and a target-encoding in mind. This is how these transfer functions are defined, but this is neither reflected on the property name ( color-space ) nor in the enums themselves ( srgb, srgb-linear, display-p3.. etc )

I hope to have made my opinions clear and I will refrain from making further comments. My only intentions are to help others better comprehend the topic of color management. Perhaps helping set a proper example for those who will be using this API and for those that come in contact with these concepts through this API. The Web was certainly a big influence on the ample adoption of sRGB standards in the past and I find that it will probably be case for future specifications, so I would prefer if we could set a standard that helps people grasp these concepts rather than further perpetuate the already established misconceptions.

That would be all, thank you for taking your time to read this.

[magcius]

Where are we on this? Can we get the srgb-linear, display-p3-linear and rec2020-linear color spaces into the spec? They would be useful.