search

microsoft / microsoft-ui-xaml

Windows UI Library: the latest Windows 10 native controls and Fluent styles for your applications
MIT License
6.28k stars 675 forks source link

Feature Proposal: HDR/WCG support in WinUI #67

Open be5invis opened 5 years ago

be5invis commented 5 years ago

Proposal: HDR and WCG support in UI controls

Summary

Currently all UI elements in WinUI are not color-managed, which means that they cannot support HDR/WCG on modern displays.

Rationale

Currently HDR and WCG monitors are getting more popular, however the default UI controls (and many first-party applications, like Photo Viewer or Edge) are always assuming that the monitor is sRGB, which breaks the experience of the users that having them. Also this is a long-lasting complain about Windows that it does not manage colors as well as Mac

Functional Requirements

Usage Examples

(This FR does not introduce new controls, but changes the display of existing controls.)

Detailed Feature Design

Open Questions

ryandemopoulos commented 5 years ago

Thanks for filing. It looks like your proposal has a couple aspects to it: HDR/WCG support, and also color profile adjustments.

Regarding HDR/WCG, we've had some discussion amongst our team on this topic before--about a year ago we added HDR/WCG support to Images and Media, and at the time we considered expanding this to a wider range of elements/controls. One of the concerns that held us back was the potential for a substantial performance hit in apps (mainly memory bloat). We could mitigate this via an opt-in model, but there's probably no way around the reality that this could cause substantial memory increase in some apps. @be5invis, are there a specific set of controls/brushes that you think would be especially useful to improve, or a specific scenario you care about? Or, is this more of a general proposal?

Regarding color profile adjustments, I'm not completely familiar with how colors are transformed across the system. I'd need to familiarize myself with that more. For Media controls, UWP XAML doesn't handle the pixel drawing--we punch out a hole in the UI and allow the MediaEngine to do it's thing as a general rule. I'll follow up to see if I can put you in contact with the right person here at MSFT for that specific piece of the proposal.

be5invis commented 5 years ago

@ryandemopoulos One great example of lacking HDR/WCG support is the current photo viewer (as well as Edge but it is dead now): it assumes all monitors' color profile being sRGB and oversaturates every ordinary image on a WCG display.

be5invis commented 5 years ago

The idea:

ryandemopoulos commented 5 years ago

@be5invis Thanks for clarifying. One of the challenges in making these improvements is that it will require UWP XAML framework changes too—not just changes in WinUI. I think we should wait until more parts of the framework are made open source per our roadmap, and then revisit at that time. Assigning this to future for now.

be5invis commented 5 years ago

@ryandemopoulos Considering that there is a demo HDR video in the Settings app, UWP XAML platform should have the potential to support HDR/WCG natively. So the things needed to be done in control level is to correct the colors, instead of assuming monitors profile.

MarkIngramUK commented 5 years ago

Coming over from my duped proposal #777 …

When composing to a HDR capable swap chain, you only need to output colours in scRGB, and Windows will take care of scRGB to screen colour conversion for you, so I'm not sure this proposal needs to talk about colour conversion? There is an issue around gamma, as the output would now be linear gamma, as opposed to gamma 2.2, so WinUI / DWM may need to perform some correction there.

Also, Windows has the notion of "SDR Brightness", where the user can change the brightness of SDR content - but not HDR content (this is controlled by the monitor OSD), so in theory, an application that supported HDR UI would have a different brightness level to other applications on the same monitor.

With regards to macOS, the window compositor applies sRGB to screen colour conversion on the entire desktop, so the entire desktop is always colour managed correctly. Unfortunately, with an opt-in approach like Windows (i.e. enable FP16 on a Direct3D swap chain and output scRGB, or manually convert the colours in your app using the screen profile) you're going to get inconsistencies (and questions from the user such as, "Why do the colours in my picture look different in these two applications?") when one application is colour managed, and another isn't.

mdtauk commented 5 years ago

Is there no API to query the device's current Colour Range?

Xbox could benefit from that now the Xbox One X exists, and presumably the Next Xbox will maintain support for HDR displays.

MarkIngramUK commented 5 years ago

Is there no API to query the device's current Colour Range?

DXGI can provide that info, but no Windows Runtime / WinUI API that I'm aware of.

lhak commented 5 years ago

I think the Windows.Graphics.Display.DisplayInformation class contains methods that will give you this information (I have not personally tried them yet).

One of the first steps to enable these features would probably to change the current Color struct to a real class like in WPF. The current struct can only hold 8bit ARGB values, while the WPF class can manage sRGB and float scRGB values (not sure how the data is managed internally). Adding support for color transformations using a supplied color profile could also be added here.

MarkIngramUK commented 5 years ago

I think the Windows.Graphics.Display.DisplayInformation class contains methods that will give you this information (I have not personally tried them yet).

Ah yes, looks like Windows::Graphics::Display::AdvancedColorInfo::IsAdvancedColorKindAvailable will give you that information. You can query for HighDynamicRange support:

The display supports high dynamic range. In this case, it is safe to assume that OS composition is being done using an RGB:FP16 surface encoded as scRGB gamma. ...

MarkIngramUK commented 5 years ago

Adding support for color transformations using a supplied color profile could also be added here.

macOS / iOS has similar capabilities in NSColor / UIColor.

ryandemopoulos commented 5 years ago

Adding the needs-winui-3 label (update from my last comment about this requiring fundamental framework changes). We can take a look once WinUI 3 ships and we have the entire XAML framework lifted up and shipping independently of the OS.

be5invis commented 5 years ago

@ryandemopoulos Also, aside from WinUI applications, legacy apps using GDI should also be converted from sRGB to monitor space, unless it claimed it will do the conversion all by itself by the MANIFEST file. Please, PLEASE, PELASE, Stop assuming all monitors being sRGB.

mdtauk commented 5 years ago

Xbox would also benefit from these API changes. As would future versions of Cleartype

peter-bertok commented 2 years ago

This is an incredible step backwards for Microsoft, while the rest of the industry is leaving them in the dust.

As far back 2006 Windows introduced support for modern display technology using the scRGB (not sRGB!) color space. This was industry-leading at the time, leapfrogging Apple. Traditionally, all professionals used Apple because of the consistent end-to-end color management, but suddenly Microsoft enabled wide gamut and HDR support for ordinary applications. This color space uses linear light, 32-bit values to support 10-bit and 12-bit displays, and had a gamut that covered all realistic display technologies for now and the future.

Someone at Microsoft was forward-thinking.

In 2022 all Apple devices use the wide Display P3 gamut by default. All Apple devices correctly display HDR including Dolby Vision. All Apple Devices are color managed by default.

In 2022 it's hard to find a TV that doesn't do HDR and wide color, except perhaps in the "ultra budget" category.

In 2022 all major web browsers are color managed and wide-gamut capable. There's progress being made towards wide-gamut colors outside of image/video files as well. (e.g.: CSS background colors)

In 2022 the Win UI 3 library has gone back to the 1990s: 8-bit SDR sRGB only, no color management at all. It's broken by design, and can never be fixed without a rewrite.

I write this on a laptop with a 10-bit HDR OLED display with wide gamut (exceeding AdobeRGB) support... but using Windows 11 and Win UI 3 which reduces this to 8-bit SDR sRGB!

I'm just going to delete the Windows App SDK and go back to doing web development, Windows native GUI development is a dead-end for me and all developers that care the slightest bit about graphics.

Let that sink in, please: I'm better off using HTML and Google Chrome to develop a graphics-heavy application for Windows 11 than the native Win UI.

xylobol commented 2 years ago

Let that sink in, please: I'm better off using HTML and Google Chrome to develop a graphics-heavy application for Windows 11 than the native Win UI.

It's incredible. I've been working on some graphics software that really needs to be color-accurate for a while and I actually can't do it with WinUI. What's funny is that I've been experimenting with using .NET for this - there's a common, shared library between all ports of the software - and the Windows version is still likely to end up as Electron. I've been trying to push Windows as a platform that can be used as designers, but it's starting to get difficult justifying this over just pushing a native version exclusively for macOS while giving everyone else a web app.

ZHOUYue67 commented 1 year ago

Windows 11 22H2 has Auto Color Management (ACM). Does this affect WinUI? https://support.microsoft.com/en-us/windows/auto-color-management-in-windows-11-64a4de7f-9c93-43ec-bdf1-3b12ffa0870b https://devblogs.microsoft.com/directx/auto-color-management/

jordan-msftfan commented 1 year ago

Any news recently?

electro-logic commented 9 months ago

The Auto Color Management feature is something cool, but many monitors and hardware configurations (CPU/GPU) are not supported and there isn't an easy way to customize things such as the rendering intent, etc.. for each app based on needs.

About color management, I just released the Mscms.WPF nuget library if you want to have a look. A GPU-accelerated version is available with the commercial license and make de-facto WPF applications fully color-managed without the performance hit.

If there is interest I may develop a WinUI version of the library.

OnlyOnePro commented 8 months ago

It is still mind boggling that windows of all things don't have proper support for HDR. Everybody knows that Window's HDR feature set is miles behind the competitors. I don't really think they will implement this in WinUI before they fix their implementation natively in the OS... and right now, Microsoft seems more interested in AI and "techbro word salad nonsense" instead of delivering a solid OS.