godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.17k stars 98 forks source link

Implement color profile management (ICC) #903

Open fire opened 4 years ago

fire commented 4 years ago

Describe the project you are working on: 3d multiplayer game project

Describe the problem or limitation you are having in your project: Rewrote https://github.com/godotengine/godot/issues/26826 as a proposal. Thanks to @toasteater.

Expected: Godot is aware of ICC profiles, and render correctly on wide-gamut displays. Actual: Godot is not aware of ICC profiles. Colors are distorted and over-saturated.

Many systems (including Windows and X11 platforms) do not have end-to-end color management, and it's up to applications to convert images to device gamuts for display.

Currently, Godot converts all output to sRGB and pushes that directly to the device. The colors are then interpreted as coordinates in the device color gamut, which is usually not exactly sRGB either, but significantly different in case of wide gamut displays. This causes major color distortions, both in the editor and in the exported games.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

Expected: Same color rendered. Actual: The color rendered by Godot is much more saturate.

Unfortunately, the effect is much less observable on a normal sRGB-ish display. On my device GIMP renders the sRGB ff0000 as d53b1a, if that puts it into any perspective.

red (Left: Godot, Right: GIMP w/ ICC profile. Screenshot interpreted as sRGB so the difference can be seen with ordinary displays. The actual saturation difference seen is much greater than what this image suggests.)

While arbitrary gamut conversion can be nice, the most currently needed feature is conversion from sRGB to display gamut, in the editor. One possible way is to allow users to specify an ICC profile in preference, which will be used to generate a 3D LUT texture on load. This resulting texture will then be used to transform the entire editor interface as the last step of rendering. This should suffice for the current use case.

Possible further improvements outside the scope of this feature proposal may include:

It's possible to take LittleCMS (MIT license) on as a dependency for this feature, which I understand won't be a very lightly made decision. ICC profiles are rather complex, so writing one's own parser can be quite slow and error prone. I'm also not experienced with Godot's code base, so I'm seeking feedback before coding anything.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

No. A workaround is possible, but significant work is required:

It's possible to use a custom postprocessing shader and a LUT to convert sRGB (or Linear with sRGB primaries) to whichever color gamut desired. However, custom postprocessing is not currently very wieldy, and it does not apply to UI elements (e.g. color choosers) on the editor side. It's also requires users to somehow generate their own LUTs from ICC profiles.

The Color Correction option in Environment is not fit for this purpose, as it can't be used for 2D, and also conflicts with scene grading within the sRGB gamut.

See https://github.com/godotengine/godot/pull/26869 for a draft in 3.2.

Is there a reason why this should be core and not an add-on in the asset library?:

Changing the rendering pipeline is part of the core.

ScepticDope commented 4 years ago

Some possible future improvements beyond the scope of this proposal, as previously mentioned here: https://github.com/godotengine/godot/issues/27956

I'd very much like to import some HDR .png files for my game. And be able to toggle HDR on and off on viewports that then start using the correct import version of the PNG.

Also at some point would love a HDR Video Content video format support for the VideoPlayer.

@fire I read your post here: https://github.com/godotengine/godot-proposals/issues/1004#issuecomment-639721193 The Frostbite engine forces monitors to switch to HDR mode automatically. Even on Windows 8.1 it automatically switched my monitor to HDR, only other application I know of that does that is MadVR. Sadly they are both closed source, besides some very limited information, I have no clue how to go about this specific problem involving HDR.

For Windows it seems to involve sending specific HDR meta data to monitors and exclusive fullscreen somehow and well, that is all I got so far. But that would mean Godot would need to implement that feature first, some references have been made to this issue: https://github.com/godotengine/godot/issues/14542 But there seems to be no actual mention of it specifically there.

Godot's current available OpenGL 3D graphics API's possibly don't actually have a exclusive fullscreen mode. So it might be an idea to wait with more HDR features until the Vulkan build is ready, as Vulkan does support it. I am yet to find if this is a cross platform issue or this exclusive fullscreen is only something specific to Windows to bypass its compositor: https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VK_EXT_full_screen_exclusive

*Something I stumbled upon is that Nvidia and AMD drivers might also create some differences around exclusive fullscreen.

I haven't found a lot of specific info on the HDR meta data you'd need to send over your display signal to the monitor. But I did find this in the Vulkan Specs: https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#_hdr_metadata

After all this I wondered if a quick and dirty alternative to all this low-level stuff, for Windows 10, could be to simply force the Windows HDR mode to toggle on? I found that someone managed to do exactly that using the Nvidia driver!: https://github.com/bradgearon/hdr-switch

fire commented 3 years ago

@ScepticDope Are you around for a chat? I see your explanation but I don't get it in sufficient detail.

ScepticDope commented 3 years ago

@fire Sure, I'll get on Discord soon I think we talked on there before.

mysterycoconut commented 3 years ago

Currently, Godot converts all output to sRGB and pushes that directly to the device. The colors are then interpreted as coordinates in the device color gamut, which is usually not exactly sRGB either, but significantly different in case of wide gamut displays. … While arbitrary gamut conversion can be nice, the most currently needed feature is conversion from sRGB to display gamut, in the editor.

On the Mac, I solve this by adding the following to the window creation code in platform/osx/os_osx.mm:

[window_object setColorSpace: [NSColorSpace sRGBColorSpace]];

That is enough to signal the OS the window is not in the device's color space, and to have the Window Manager use a shader based color match. Without that line, most Godot games look over-saturated on the "newer" (2015 on) P3 displays.

MJacred commented 2 years ago

@fire: Are you still working on this or anyone else? I saw that https://github.com/fire/godot/tree/littlecms and https://github.com/fire/godot/commit/5e9855dd73a03f580011bdf87075bfb2aaee899a have been archived…

fire commented 2 years ago

It is salvagable, I am not directly working on this.

ScepticDope commented 1 year ago

Speculation, but with HDR maybe being a cool new feature on the new Steam Decks or part of Proton, @Plagman? HDR on Linux might get a little push. And thus it might start to be more of interest to Godot developers as well?: https://twitter.com/Plagman2/status/1610200412854046720

Also found this nice up to date overview on HDR on Linux's current state of affairs: https://wiki.archlinux.org/title/HDR_video_playback

fire commented 1 year ago

The steam deck oled launched a few days ago. This means the Linux operating systems should have a way to test oleds and be related to gaming.

Calinou commented 1 year ago

The steam deck oled launched a few days ago. This means the Linux operating systems should have a way to test oleds and be related to gaming.

Wayland/Gamescope indeed have basic HDR support (and Plasma 6 will support it officially), but NVIDIA support is still missing. Only AMD and Intel GPUs currently support HDR on Linux.

allenwp commented 2 months ago

I don't have a monitor to test with, but is it possible to change the Godot output mode to allow more OS/driver interpretation? (e.g. Windowed mode or something?) Specifically so that the visual output from Godot is treated as a legacy sRGB source by the OS/drivers and correctly converted to the display colour space automatically?

I suspect that existing OS/drivers must have this sort of functionality built in to support legacy apps that don't support interpretation of the user's ICC. And the reason Godot currently doesn't benefit from this legacy support is because Godot's output mode(s) bypass it?

Or am I wrong and all legacy sRGB apps on most operating systems, such as Windows and Mac, will just render entirely wrong on a wide gamut display just like Godot currently does?

alvinhochun commented 2 months ago

I don't have a monitor to test with, but is it possible to change the Godot output mode to allow more OS/driver interpretation? (e.g. Windowed mode or something?) Specifically so that the visual output from Godot is treated as a legacy sRGB source by the OS/drivers and correctly converted to the display colour space automatically?

I suspect that existing OS/drivers must have this sort of functionality built in to support legacy apps that don't support interpretation of the user's ICC. And the reason Godot currently doesn't benefit from this legacy support is because Godot's output mode(s) bypass it?

Or am I wrong and all legacy sRGB apps on most operating systems, such as Windows and Mac, will just render entirely wrong on a wide gamut display just like Godot currently does?

Windows 11 starting from version 22H2 (or so I've heard) has an experimental feature called Auto Color Management that expanded on the HDR support added previously to apply system-wide colour calibration for SDR displays, while allowing apps to explicitly use wide gamut colours (through 16f scRGB values) if they need to. (I've heard there are bugs or maybe it still hasn't gained support for ICC profiles, but I don't know the full story.)

Earlier versions of Windows do not have system-wide colour management like this. Applications are required to handle any colour management themselves. You can calibrate a monitor to sRGB, but then you lose the ability to use the wide gamut. Otherwise, colours are output to the monitor in its native gamut.

From what I understand, macOS has had system-wide colour management since forever. I don't if it works in practice, but from the code Godot does seem to engage this (using sRGB): https://github.com/godotengine/godot/blob/b6223c0df0300ba2db17b5742c349f13c33f8884/platform/macos/display_server_macos.mm#L114

allenwp commented 2 months ago

Thanks, @alvinhochun!

If I understand correctly, this means a wide range of existing games from many different games engines exhibit this problematic behaviour on Windows.

This may also mean that correcting this behaviour for sRGB output from Godot would result in some people being disappointed that images rendered through Godot look desaturated on their monitor, at least compared to what they expect based on output from other legacy sRGB apps or older versions of Godot. Yikes!

That said, if you’re working with a monitor that has an ICC profile, that likely means you’re the type of user who wants correct colour output, so hopefully not an issue in practice.

allenwp commented 2 months ago

After looking into it further, it seems that the new Auto color management in Windows 11 has been rolled out and any user with a compatible graphics device and monitor should now be able to use this feature.

As mentioned previously, this sort of system-wide colour management has existed in Mac OS for a very long time.

This means that the proposal should be updated, as this feature for ICC profile support with sRGB output is really only relevant for Linux operating systems going forward…

alvinhochun commented 2 months ago

This may also mean that correcting this behaviour for sRGB output from Godot would result in some people being disappointed that images rendered through Godot look desaturated on their monitor, at least compared to what they expect based on output from other legacy sRGB apps or older versions of Godot. Yikes!

This has already been a problem with the HDR mode on Windows -- you can find various posts of people complaining that HDR mode makes colour "washed out".

That said, if you’re working with a monitor that has an ICC profile, that likely means you’re the type of user who wants correct colour output, so hopefully not an issue in practice.

Some laptops have pre-installed ICC profiles and monitor drivers do usually come with default ICC profiles.

After looking into it further, it seems that the new Auto color management in Windows 11 has been rolled out and any user with a compatible graphics device and monitor should now be able to use this feature.

Looks like ACM is rolling out for real on Win11 24H2.

huwpascoe commented 2 months ago

https://developer.android.com/training/wide-color-gamut This is pretty definitive

The Vulkan support for wide color gamut is provided through the VK_EXT_swapchain_colorspace extension.

Vulkan wide color gamut color spaces:

  • VK_COLOR_SPACE_EXTENDED_SRGB_LINEAR_EXT
  • VK_COLOR_SPACE_DISPLAY_P3_NONLINEAR_EXT

In order to use wide color gamut mode in OpenGL, your app must include the EGL 1.4 library with one of the following extensions:

  • EGL_EXT_gl_colorspace_display_p3
  • EGL_EXT_gl_colorspace_scrgb
  • EGL_EXT_gl_colorspace_scrgb_linear

I tried enabling VK_EXT_swapchain_colorspace in Godot in hopes of getting this issue somewhat resolved, but discovered that my aging computer has no support for the colorspace extensions so that's the end of it.