Minimal HDR Canvas proposal

[palemieux]

Refactor the HDR Canvas proposal to a minimal baseline focused on today's HDR imagery and practice without precluding future optimizations and extensions.

Highlights:

• add support for HLG and PQ color spaces, including explicit recommendations for:
  • tone-mapping from HLG and PQ to sRGB
  • mapping from sRGB to HLG and PQ
  • mapping between HLG and PQ
• add support 16-bit floats bitmaps

[michaeldsmith]

The ST2094-10 Annex B method mentioned in the section "rec2100-pq to srgb"

needs parameters min/avg/max values.

The example code at https://www.sandflow.com/public/tone-mapping/index.html performs image analysis to determine the min/avg/max values. This is a great way to obtain the min/avg/max parameters for a single image.

I think that it should be suggested that the good results will come from obtaining the min/avg/max parameters via image analysis.

If image analysis is not performed, then the min/avg/max values need to be provided as parameters to 2094-10 Annex B another way. Default min and max values could come from ST2086 mastering display metadata min luminance and max luminance, default max value could come from MaxCLL metadata. The average will depend on if the scene is bright, average or dark. Using a default average value between 5-15 nits could be a good starting place.

[ccameron-chromium]

This removes the features related to rendering content in an SDR-relative, application-tone-mapped canvas (in particular, the extended mode, and the ScreenDetails parts.

Splitting these into two proposals is probably the right thing to do -- there is too much misunderstanding between the two worlds that both call themselves HDR.

I would also suggest separating the float16 support out from this proposal. It already has its own proposal here and lengthy-and-stalled discussion here.

A remaining core part of this proposal (with this PR applied) is a standardization of how to render PQ and HLG content on phone/tablet/laptop/desktop operating systems.

These operating systems parameterize their displays by two parameters: HDR headroom and color gamut (this encompasses of Windows, iOS, macOS, Android, and ChromeOS, including the built-into-a-TV variants). None of the existing standards address this problem (they have no concept of a time-varying HDR headroom, which is fundamental to all operating systems' implementations of HDR).

There is also this issue about standardizing how PQ and HLG images are to be rendered as HTMLImageElement.

[palemieux]

I would also suggest separating the float16 support out from this proposal.

I do not think this is separable since HDR cannot exist in an 8-bit world, as described in the introduction.

there is too much misunderstanding between the two worlds that both call themselves HDR.

I do not see a misunderstanding.

HLG and PQ have been iterated for years and are used by all HDR TV and movie content. The web has become an important source of TV and movie consumption. It makes sense for the web platform to support them.

This does not preclude/conflict with other approaches, as far as I know.

[ccameron-chromium]

I would also suggest separating the float16 support out from this proposal.

I do not think this is separable since HDR cannot exist in an 8-bit world, as described in the introduction.

Changes to PredefinedColorSpace for HLG and PQ is valuable on its own, because PredefinedColorSpace is used by 2D canvas, ImageData, WebGL, and WebGPU.

There are some issues to be worked out for 2D canvas and ImageData with respect to float16 support, but WebGPU is already shipping support for it, and WebGL has a proposal that doesn't have any issues that I know of.

I wouldn't want us to block HLG and PQ support in WebGPU because of unrelated 2D canvas issues (which may take a long time to resolve).

I also don't think we should explicitly disallow 8-bit HDR (it's not unreasonable for HLG -- see these 8-bit JPEG HLG images).

[palemieux]

I also don't think we should explicitly disallow 8-bit HDR

Have you read the article at https://ieeexplore.ieee.org/document/7514362 ? It lays out the fact that 8-bit is not even sufficient for sRGB at low luminance.

P.S.: The images show up HDR in my Chrome browser! :)

[palemieux]

I wouldn't want us to block HLG and PQ support in WebGPU because of unrelated 2D canvas issues (which may take a long time to resolve).

I argue we should use HDR as a way to drive the resolution of these issues.

I am not disagreeing with you that there is an obstacle.

[kenrussell]

We could almost replace references to Float16Array with Float32Array and not only would no web developer notice, but also the references could be changed back to Float16Array once that has been shipped in browsers with no application impact. Since reads from Float16Array will immediately convert to 64-bit float (JavaScript's native number representation) and writes to Float16Array silently truncate from that representation, it will behave very similarly to how Float32Array does today.

However - since this might be contentious, getting this in to have a spec browsers can start implementing would be preferable in my opinion.

[svgeesus]

I also don't think we should explicitly disallow 8-bit HDR

I disagree. Don't give people a footgun, and instead, use this to drive the higher-bit-depth canvas upgrade.