search

mrdoob / three.js

JavaScript 3D Library.
https://threejs.org/
MIT License
100.41k stars 35.2k forks source link

Roadmap for supporting wide-gamut color workflows #26479

Open donmccurdy opened 11 months ago

donmccurdy commented 11 months ago

Related:

Roadmap and discussion for supporting for wide-gamut color spaces in three.js. Many displays now support Display P3 (+50% larger gamut volume than sRGB), and web browsers are beginning to extend that support to WebGL contexts. Other color wide-gamut color spaces exist (Rec. 2020, ACEScg) and may be relevant to our internal rendering pipeline, but probably will not be widely available as display color spaces in WebGL for a very long time.

Broadly the goals here are (1) to identify useful and correct workflows for users, and (2) to make any necessary changes in the three.js rendering pipeline. I suspect that (1) may be the harder part of the problem.

Decisions

Workflows

I'm skeptical than wide-gamut features can be enabled automatically; it will require significant care from users to get the benefits of these displays, and should be opt-in. I'm not sure which 3D modeling tools support the P3 gamut at all. Files in standard formats like glTF are almost exclusively using sRGB today. Users will need to load textures manually and assign texture.colorSpace correctly. We cannot infer texture color space safely without user input of some kind.

Tasks

Task numbers begin at 2.1, following up from #23614.

  1. [ ] (2.1) Support changing ColorManagement.workingColorSpace
    • Allow one or both of "acescg" or "p3-linear", in addition to current "srgb-linear"
  2. [ ] (2.2) Support THREE.Color getters/setters in new working color space(s)
  3. [ ] (2.3) Support renderer.outputColorSpace = THREE.DisplayP3ColorSpace
  4. [ ] (2.4) Model loaders must delegate management to THREE.ColorManagement, not convert to Linear-sRGB explicitly
  5. [ ] (2.5) Texture loaders should identify and tag wide-gamut textures where possible (OpenEXR, KTX2, ...?)
  6. [ ] (2.6) Use gl.unpackColorSpace to unpack "display-p3" PNG, JPEG, and WebP images into the working color space
  7. [ ] (2.7) Provide tone-mapping implementations from wide-gamut working color space(s) to "srgb" and "display-p3" output color spaces
donmccurdy commented 11 months ago

@WestLangley – continuing from our technical discussion, I've added more TODOs in the task list above. Please let me know if you feel anything major is missing or doesn't belong. :)

donmccurdy commented 10 months ago

PR for 2.1, 2.2, 2.3, and 2.6 —

We should work toward offering a tone map adapted for wide-gamut and HDR displays, and with a more neutral default appearance than ACES Filmic. Blender 4.0 will be moving to AgX as the default, and I believe that would be a good choice for us as well. Related details:

Blender 4.0 alpha builds are available with AgX support now. I've prototyped an implementation in three.js. I am not sure about how to add support for other working and display color spaces yet. The implementation can also be optimized by removing the 1D LUT.

/cc @bhouston

bhouston commented 10 months ago

@donmccurdy great work! I also strongly agree that we should aim to provide AgX as an alternative tone mapping and maybe if there is enough acceptance, we can switch it to the default.

donmccurdy commented 8 months ago

Updates:

donmccurdy commented 8 months ago

@WestLangley from your comment in https://github.com/mrdoob/three.js/pull/26666#discussion_r1356944154 –

I don't think we should "honor" the color space of a render target when rendering to it. The color space is relevant only when reading from the render target.

If we're reading from the render target in sRGB we do need to get that data in correctly, right? We can safely "ignore" the color space now because writing Linear-sRGB to the sRGB target will lead WebGL to (automatically?) apply the sRGB OETF for us, and we don't want our fragment shader doing that.

I think we may want to go one step further here. If we are writing to a uint8 render target having Display P3 color space, from working color space Linear sRGB, our fragment shader should do the gamut mapping and let WebGL do the OETF. Writing non-linear data to a f16 or f32 render target would not be supported, because WebGL doesn't support it.

WestLangley commented 8 months ago

I think what you said is correct, and I don't think it is contrary to my statement. 🤔

kitaedesigns commented 7 months ago

Did this AgX feature request get merged into the latest version of threejs? I couldn't tell.

PR for 2.1, 2.2, 2.3, and 2.6 —

We should work toward offering a tone map adapted for wide-gamut and HDR displays, and with a more neutral default appearance than ACES Filmic. Blender 4.0 will be moving to AgX as the default, and I believe that would be a good choice for us as well. Related details:

Blender 4.0 alpha builds are available with AgX support now. I've prototyped an implementation in three.js. I am not sure about how to add support for other working and display color spaces yet. The implementation can also be optimized by removing the 1D LUT.

/cc @bhouston

donmccurdy commented 7 months ago

It hasn't been — I don't think anyone is working on it currently. Additional discussion here:

kitaedesigns commented 6 months ago

Gotcha thanks Don, I'm not sure your Glitch link is working anymore. With Blender 4.0 any assets users create in Blender and show in three.js look quite different. Your LUT approach although perhaps not totally accurate seems quite flexible and I think many 3D designers like myself are very used to LUTs due to their popularity in game engines like Unreal and Unity, would love to test with that LUT file, I assume it's a .CUBE or .png?

I am currently using pmndrs PostProcessing library and @vanruesc mentioned it would be more efficient to have natively in three.js first which I understand, but maybe I can just use your LUT in his library and see how close I can match the colors between Blender 4 and Three.js.

donmccurdy commented 6 months ago

The LUT is part of the prototyping process here – it isn't really something we want in the final implementation, and doesn't represent the complete tone mapper. A better realtime implementation (like Filament's) would replace the LUT with a fitted function.

I think AgX is something we can and should support independently of pursuing support for wide-gamut color, so let's continue that discussion in a new thread: