search

w3ctag / design-reviews

W3C specs and API reviews
Creative Commons Zero v1.0 Universal
333 stars 56 forks source link

Adding support for High Dynamic Range (HDR) imagery to HTML Canvas #917

Open palemieux opened 1 year ago

palemieux commented 1 year ago

こんにちは TAG-さん!

The Color on the Web CG is kindly requesting a TAG review of its proposed modification to the Web Platform to support High Dynamic Range (HDR) imagery in HTML Canvas.

The complete proposal is at https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_html_canvas_element.md

Today HTML Canvas supports only 8 bit per color channel and two PredefinedColorSpace color spaces (srgb and display-p3). This is insufficient for High-Dynamic Range (HDR) imagery, which is in widespread use today. Accordingly, the following API modifications are needed to manipulate HDR images in HTML Canvas:

Please note that the proposed screenColorInfo property adds a fingerprinting surface. This feature is however intended primarily for advanced color volume mapping applications and is not believed to be generally necessary.

Contacts: @palemieux and @svgeesus (Color on the Web CG co-chairs)

Feedback, issues and questions are welcome at mailto:public-colorweb@w3.org and https://github.com/w3c/ColorWeb-CG/issues.

svgeesus commented 9 months ago

Hello TAG,

On 19 Feb 2024 in Tokyo there is a cross-SDO meeting HDR Stills Experts Day, with the International Color Consortium, W3C, and ISO TC 42/WG 23 (Photography) on the subject of HDR imagery. It would be super-useful to get any TAG input from this HDR in Canvas review ahead of that meeting

palemieux commented 9 months ago

@svgeesus Where is the meeting announced in the context of W3C? Any requirements for participation?

svgeesus commented 9 months ago

@palemieux It is on the ICC site in the Member-only area. It isn't a public meeting. I will be there, and Chris Cameron also.

palemieux commented 9 months ago

What is the process if myself and other W3C members wish to attend?

palemieux commented 9 months ago

To be clear, my concern is with the output of the meeting being described as including an opinion from the W3C and/or input provided in the meeting be presented as W3C input, without such input having been discussed/reviewed by W3C participants.

svgeesus commented 9 months ago

I think the proposed modification to the Web Platform to support High Dynamic Range (HDR) imagery in HTML Canvas has been adequately discussed that it can be presented at the meeting, no?

palemieux commented 9 months ago

Ok. Which document(s) are you planning on presenting -- beyond https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_html_canvas_element.md? It would be good to inform their respective groups.

jyasskin commented 9 months ago

This doesn't seem like the right repository for this side discussion.

LeaVerou commented 8 months ago

Hi there,

@plinss @ylafon and I looked at this today during a breakout. We had a few thoughts and questions:

  1. The part of the API around checking whether a color is in within the volume of the output device and mapping it manually by the author seemed both too close to the hardware to be future-compatible, and too low level to have good DX. We think a higher level method to check whether a color is within the range of the output device would be preferable.
  2. The Canvas APIs do not seem like the right place to specify primitives for querying the volume of the output device, which are useful in APIs across the entire web platform.
  3. While we appreciated the discussion on volume mapping, we still had questions about the specifics. We see several cases where gamut/volume/tone mapping come into play, and we’d love some more detail on how these are handled (actually, the same questions apply to wider/smaller gamuts too):
    1. HDR image painted on a non-HDR canvas
    2. HDR canvas displayed non-HDR screen
    3. Author painting an HDR CSS color on a non-HDR canvas
  4. The used color space is a property of CanvasRenderingContext2D, but HTMLCanvasElement provides methods for exporting the canvas contents as an image (toBlob(), toDataURL()). Is the color space preserved during these operations? (again this also applies to wide gamut)

And some more minor comments:

plinss commented 8 months ago

I was also wondering why not include the other color spaces supported by CSS. I understand that the rec2100- colorspaces are supersets, by if one was working in one of the smaller colorspaces the conversions to/from rec2011- could be lossy, right?

I want to be sure that use cases aside from rendering the canvas directly to the screen are supported, so operations shouldn't be constrained by the available hardware. e.g. an image editor should be able to work natively in any source image's colorspace.

svgeesus commented 8 months ago

@plinss

I was also wondering why not include the other color spaces supported by CSS. I understand that the rec2100- colorspaces are supersets, by if one was working in one of the smaller colorspaces the conversions to/from rec2011- could be lossy, right?

Canvas 2D already supports any of those CSS Color 4 SDR color spaces on input. Currently, the only Canvas 2D context color spaces available are sRGB and Display P3 and the specified color is converted to the context colorspace. Furthermore, current implementations are restricted in practice to an 8-bit per component range:

Both CanvasRenderingContext2D and OffscreenCanvasRenderingContext2D contain an output bitmap that they render to. The pixel format of this output bitmap is currently unspecified. Many implementations use a 8 bits per channel RGB or RGBA pixel format for this bitmap.

Thus, if the specified color is outside the gamut of the 2D context, it cannot be represented (current implementations use a naive clamping operation to force the color to fit) and if it is in gamut, it will still be quantized to 8 bit resolution.

The HDR canvas proposal assumes as prerequisite the Canvas Floating Point Color Values proposal which introduces the ability to use floating-point pixel formats in CanvasRenderingContext2D, OffscreenCanvasRenderingContext2D, and ImageData.

Together with extended range colorspaces (the colorspace is defined over the entire coordinate range, i.e. not just [0.0, 1.0] but also negative numbers and numbers greater than 1) this would allow out of gamut input colors to be stored without loss. I don't know offhand if this is required and if only HDR colorspaces benefit from Canvas FP values, but I would assume that SDR colorspaces can utilize this.

svgeesus commented 8 months ago

I should clarify that all the CSS Color 4 color spaces are defined over the extended range.

svgeesus commented 5 months ago

@ palemieux this issue is waiting for a response from ColorWeb CG and/or the proposers, regarding the questions from the TAG review

palemieux commented 5 months ago

While we appreciated the discussion on volume mapping, we still had questions about the specifics.

@LeaVerou To make sure we address the question correctly, can you specify what is missing in Annex A?

palemieux commented 5 months ago

The part of the API around checking whether a color is in within the volume of the output device

@LeaVerou Do you mean this part of the API.

If so, how is it too close to the hardware? Note the API does not require the returned values to match the hardware exactly, and may reflect user preferences/OS limitations/environmental conditions/etc.

That part of the API is intended for advanced applications. The vast majority of applications are expected to merely set the Canvas to an HDR colorspace and write pixels in that colorspace, letting the UA/OS/Device render the HDR pixels as appropriate based on current conditions.

palemieux commented 4 months ago

FYI. Comments above have been entered at https://github.com/w3c/ColorWeb-CG/issues?q=is%3Aopen+is%3Aissue+label%3Atag-comment.

jyasskin commented 2 months ago

A piece of this is shipping in https://groups.google.com/a/chromium.org/g/blink-dev/c/rBQIRHUEAe8/m/JvL612SvBQAJ