search

terrymacdonald / DisplayMagician

DisplayMagician is an open source tool for automatically configuring your displays and sound for a game or application from a single Windows Shortcut.
https://displaymagician.littlebitbig.com
GNU General Public License v3.0
449 stars 19 forks source link

Custom Color Settings Not Restored #45

Closed Mobeeuz closed 3 years ago

Mobeeuz commented 3 years ago

I'm using a custom color setting for my Nvidia card, (32bit, 12bpc, RGB, Full Range). When I create a profile and switch to it, DM will not only choose the incorrect color settings, but my settings are not available to select so I can repair it. I have to switch to another resolution via the Nvidia control panel first, then the settings will reappear.

In this example, I went from 4k HDR to 1080p SDR and was stuck at 32Bit, 8bpc, YCbCr22, limited color (with no other options populated in the drop down menus).

terrymacdonald commented 3 years ago

Hmmmm. That one is odd. There is a potential that the new library I wrote isn't returning those settings, so they're not being stored. I'll need to take an in-depth look at the code to see where it could be failing.

I'll probably need to get you to test a new version of NVIDIAInfo again so that we can make sure that we grabbing the settings. I certainly hope so, otherwise I'll need to update the display profile format again ☚ī¸.

Thanks for the bug report! It's the only way I can really test DisplayMagician as I just don't have enough video cards to hand to test by myself.

Thanks Terry

On Tue, 5 Oct 2021, 12:11 Mobeeuz, @.***> wrote:

Assigned #45 https://github.com/terrymacdonald/DisplayMagician/issues/45 to @terrymacdonald https://github.com/terrymacdonald.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/45#event-5407150453, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LFZMOIGLTVMOBFKV5LUFIYDBANCNFSM5FKOKWIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Mobeeuz commented 3 years ago

No worries, let me know...

terrymacdonald commented 3 years ago

@Mobeeuz Can you please write some instructions and take some screenshots of the settings screens that you are set in the NVIDIA drivers and in Windows so that I can replicate the settings? I don't modify any colour settings myself, so I'm not sure where to set these things. And can you please also attach the DisplayProfiles_2.0.json file for me to see what DM has captured?

Please attach both those files by visiting this issue and attaching the files as part of your reply. The github mail forwarder doesn't allow files through unfortunately.

I'll review what you send, and it will help me figure out if the existing NVIDIA functions I use to get the settings include the colour settings necessary to catch those colour settings. I may need to update the NVIDIALibrary code I made to add those lost data points in.

Thanks Terry

terrymacdonald commented 3 years ago

Ok, another data point. I found out that DM wasn't recording the color data, only the HDR color data. So I have begun to add the color data to the NVIDIAInfo test suite so that will start tracking it. If you look at the NVIDIAInfo repository, you'll see that I've added the creating of the configuration, and have it attempted to apply the colour settings as well. But there is currently an issue I'm trying to troubleshoot, where a color configuration isn't actually being applied even though it says it is.

I'll do some more troubleshooting again tomorrow, but at this stage it could be one of two things...

  1. I read a forum post where NVIDIA said that NVIDIA specific settings are supposed to override the windows ones. At present they are applied first in NVIDIAInfo, not last. I'll try that change tomorrow and see if that works.
  2. I may need to double check the struct sizes, as I'm not sure they're quite right. They may be causing the nvapi function to error while attempting to apply the color settings.

There is also a lot of additional improvement I've realised I can make to avoid applying display changes unless they're needed, so I'll try to make those changes at the same time.

My goal is to have fixed NVIDIA color settings as part of v2.1. I may also need to roll in a fix for #44 as well which will delay this fix, as I'm trying to minimise display profile file format changes and roll them together where possible.

Anyway, that's enough for tonight.

Regards Terry

Mobeeuz commented 3 years ago

Was at work for the past few days, did you still need the above info?

terrymacdonald commented 3 years ago

No I don't think so, I think I have found the detail I needed elsewhere. I will get you to test a new version of NVIDIAInfo shortly which will have the colour settings in it. I'll drop a link in here when it's done.

terrymacdonald commented 3 years ago

@Mobeeuz I now have an updated version of NVIDIAInfo that should store and load color settings as well as NVIDIA surround and HDR settings. Can you please download it, unzip it and give it a test for me please? https://github.com/terrymacdonald/NVIDIAInfo/releases/tag/v1.0.5

I can't remember if you've used NVIDIAInfo for any testing before, so please take a look at the instructions on the NVIDIAInfo release page to see how to use it.

Basically I need you to create some different display configurations, saving the config into NVIDIAInfo cfg files:

Then swap between those saved cfg files to make sure that the correct info is applied correctly. Please copy and backup the NVIDIAInfo.log file after each run so I can see if there are any issues.

Please note that NVIDIA Setup 'Change Resolution' screen doesn't update automatically, and you have to swap to another part of NVIDIA Setup and then swap back in order for it to show the settings properly.

Once you've tested, please visit this issue in your web browser, and please attach the different NVIDIAInfo.log files to a reply to this message. I will then review them and make sure everything works! I'm hoping it all goes smoothly so I can add this update to the next release of DisplayMagician v2.1.0.

Thanks Terry

Mobeeuz commented 3 years ago

So, the same thing is happening. The Standard to custom color switch is not being applied either. I have attached the cfgs and logs for loading, saving, and switching between resolutions and settings. I do not have a surround setup, so I cannot test that. NVD.zip

terrymacdonald commented 3 years ago

@Mobeeuz That was awesome work on the logs! It really helped me nail down some bugs.

I think I've managed to crack the problem open. It looks like Windows HDR is really finicky and it's critical when the NVIDIA custom color settings are applied compared to the HDR settings. So I've had to develop a reasonably complicated series of checks to make sure that the right settings are applied at exactly the right points to make the stick. Frankly, it's been a giant pain in the butt :).

I think I've got there now. On my test system the changes are applied consistently, and work fine. The only annoyance is that you need to click on the 'Adjust desktop colour settings' menu then back to 'Change resolution' within NVIDIA Setup in order for the new settings to be shown correctly (NVIDIA Setup doesn't expect another bit of software to change the settings behind it's back!).

So... please test again with NVIDIAInfo v1.0.6, and if it doesn't work I'll have a little cry and then try to figure out a fix ;D.

https://github.com/terrymacdonald/NVIDIAInfo/releases/tag/v1.0.6

Thanks Terry

Mobeeuz commented 3 years ago

I'll upload some tissues ;) Some progress: Going from 1080p standard color to 4kHDR Custom color did work correctly, complete with color mode switch. Going from custom color to standard in any res does not switch color modes. Same issue with the NVidia CP - I don't leave it open when using the util, and tried reloading the CP multiple times, but the dropdowns are still missing items after using the util.

NVD2.zip

Postulate: Perhaps the CP is broken or reading the info incorrectly. Is there a quick util/mod you can whip up to read and display the current color settings using the same routines that NVIDIAInfo uses?

terrymacdonald commented 3 years ago

I do already. It uses the NVAPI to perform all of the work. There's nothing else I can do other than try to replicate the order the changes happen (of which I can only test).

The sequence you described is puzzling. The fact that it works going to 1080p to 4hHDR and yet doesn't work in the backwards direction.....that's odd. It's exactly the same steps for both setups: 1. turn surround on/off with NVAPI, 2. setup windows layout/settings using Windows CCD API, 3. turn off any NVAPI color settings that set to windows control, 4. Apply NVAPI HDR settings, 5 apply new NVAPI color settings that are overrides. I'm not sure how that can work in one direction and not the other?

I just had a thought. @Mobeeuz Are you able to delete the old cfg files, and to create new ones using the NVIDIAInfo v1.0.6 and then try it again? It could work with fresh cfg files

Thanks

Terry

Mobeeuz commented 3 years ago

Those are fresh configs - I just assumed I should redo everything.

terrymacdonald commented 3 years ago

Ah cool. That's awesome. I normally have to remind ppl so good work!

Right, so looking at the logs, it looks like there is an error when trying to get your color settings as it's being saved:

2021-10-07 21:18:12.1009|WARN|DisplayMagicianShared.SharedLogger|NVIDIALibrary/GetNVIDIADisplayConfig: A miscellaneous error occurred. NvAPI_Disp_ColorControl() returned error code NVAPI_ERROR.

For some reason, when we call NvAPI_Disp_ColorControl() to get the color config in order to save it, it is erroring and not updating the color data. We create a default colordata structure to pass to that function, and it is supposed to fill that structure with what the real settings are, but that error tells me that it is not doing that.

The trick is to now try to figure out WHY this could be happening.... I'll do some more experiments over the weekend on my test system to see if I can provoke it into breaking.

Can you please do some screenshots of the settings that you're putting in NVIDIA Setup? Maybe I can replicate those and see if that causes the same problem on my PC?

Thanks Terry

terrymacdonald commented 3 years ago

By the way, NVIDIAInfo thinks you have two screens connected, not one screen. Didn't you mention earlier that you only had one screen? It thinks there is a BENQ screen and a SONY screen as far as I can tell. Can you please provide a bit more info about your display setup? It could help with troubleshooting.

Mobeeuz commented 3 years ago

I do have 2 just not setup for surround (To me surround means gaming with 2 screens as 1 wide screen with bezel setup) The BenQ is just not suited for that, size wise or capability, so I just use it for info. I also change nothing on that screen for any of my cfgs.

My settings: NVSettings.zip

Mobeeuz commented 3 years ago

To further test on my end, I used Display Driver Uninstaller to completely remove all my drivers settings and reinstall. During this process, I realized I also use NVSlimmer to remove the cruft and telemetry from the drivers before I install. On the slim chance that I am not installing a backend service that is required, I'm including a screen of my NVSlimmer settings. I have also included a shot of the CP settings I am stuck with when using your util to switch back to 1080p. The settings you see in the dropdowns are all that's available.

NVSettings2.zip

(After going through the above process, the same problem is occurring by the way.)

terrymacdonald commented 3 years ago

Ohhh, that could be it. Is there any chance you could reinstall the full nvidia bloatware driver and then see if that works? It could be that one of those extra components are used by the driver for providing NVAPI color functionality. The only real way to test that is to compare a NVSlimmer install versus the full install after a reboot.

If you are able to do a full install, then reboot and recreate the NVIDIAInfo cfg files again from scratch, then it will be a guaranteed test, as we know your system had the error, so if after all that the error goes away we know its the driver changes that made the difference. If not, then its nothing to do with the driver, and it's either my code, a driver bug, or a hardware issue.

Does that all make sense?

Are you able to test with the full NVIDIA driver?

Thanks Terry

terrymacdonald commented 3 years ago

I've been doing more reading, and it looks more and more like the NVIDIA driver colour settings aren't designed to be used when the Windows HDR switch is enabled. The NVIDIA Control Panel help says "NOTE: (Windows 10) Beginning with Windows 10 Creator's Update (Redstone 2), the operating system controls these colour settings. You can either use the Windows settings, or override them through the NVIDIA Control Panel by selecting the Use NVIDIA colour settings tick box."

Additionally, the testing I've done shows that if you have custom NVIDIA colour settings enabled, then when you turn on windows HDR, the NVIDIA driver automatically turns off the custom NVIDIA colour settings. It very much looks like NVIDIA don't want custom NVIDIA colour settings enabled at the same time as Windows HDR being turned on.

I'm part of the NVIDIA developer forum, so I think I'll ask a questions there and see what the NVAPI team say. Sometimes it's best to hear it from the people who made it :)

Anyway I'll keep doing some more testing and see what I find.

Terry

terrymacdonald commented 3 years ago

And I've added a question: https://forums.developer.nvidia.com/t/how-does-win-hdr-and-nvidia-hdr-work-together-programmatically/191566

Let's see what they say.

terrymacdonald commented 3 years ago

@Mobeeuz ok... While waiting for NVIDIA, I've been playing around seeing if I could figure out how the HDR an non-HDR colour settings are supposed to be applied by the NVAPI. And I think I may have something. I've rewritten the logic so that it now acts differently when NVIDIA Surround is used, and it now uses either in HDR more (using HDR colour settings), or not (using normal colour settings). And in testing, that worked perfectly!

So now, I'd really appreciate it if you could do some more testing for me, but this time using NVIDIAInfo v1.0.8: https://github.com/terrymacdonald/NVIDIAInfo/releases/tag/v1.0.8

If you could delete the old cfg files, and create some new ones again, hopefully this time it works! 🤞

I also found a bug in the way that the windows config was being applied, so that hopefully has fixed the refresh rate application process. So I have high hopes for this one :).

I also wanted to say thank you to you for all the help you've provided so far. This has really helped me focus on fixing the issue, knowing that I have your help to check whether it works or not. I'm sure the rest of the 2000 DisplayMagician users will thank you when we're done!

Thanks Terry

Mobeeuz commented 3 years ago

So, same thing is happening. There was a small hiccup going from 4kDefault-1080pHDR (log in the first zip) - it didn't apply and spit out an already applied error. The NV3 is using 1.0.8 and my slimmed down driver. The second zip is with the full bloated driver (even GF Experience - ick) but the same thing is happening.

NVD3.zip NVD4.zip

terrymacdonald commented 3 years ago

@Mobeeuz I am so confused right now.

NVIDIAInfo v1.0.9 responded that those configs are all different based on my testing. It's really weird that it could complain 

NVIDIAInfo v1.0.9
=================

By Terry MacDonald 2021

Attempting to apply NVIDIA display config from non-surround-colour-default.cfg
NVIDIAInfo Display config successfully applied

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\4KDefault.cfg .\4KHDR.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\4KDefault.cfg and .\4KHDR.cfg are NOT equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\4KDefault.cfg .\4KDefault.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\4KDefault.cfg and .\4KDefault.cfg are equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\4KDefault.cfg .\1080pDefault.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\4KDefault.cfg and .\1080pDefault.cfg are NOT equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\4KDefault.cfg .\1080pHDR.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\4KDefault.cfg and .\1080pHDR.cfg are NOT equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\1080pHDR.cfg .\1080pDefault.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\1080pHDR.cfg and .\1080pDefault.cfg are NOT equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\1080pHDR.cfg .\1080pHDR.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\1080pHDR.cfg and .\1080pHDR.cfg are equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\1080pHDR.cfg .\4KDefault.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\1080pHDR.cfg and .\4KDefault.cfg are NOT equal.

PS H:\vscode-projects\NVIDIAInfo\NVIDIAInfo\bin\Debug> .\NVIDIAInfo.exe equal .\1080pHDR.cfg .\4KHDR.cfg

NVIDIAInfo v1.0.9
=================
By Terry MacDonald 2021

The NVIDIA display settings in .\1080pHDR.cfg and .\4KHDR.cfg are NOT equal.

I tested various combinations and they all compare accurately as far as I can tell.

I've now greatly increased the logging verbosity for NVIDIAInfo v1.0.9. Can you please do some more testing, but please don't be as exhaustive for now. This is really to make sure that I'm understanding the logic flow on your machine, and to make sure that I'm logging the right things.

NVIDIAInfo-v1.0.9.zip

Can you please send me just the log files, but please name them so I can tell which of your NVIDIAInfo configs you are going from and to (as you have before). Please recreate the .cfg files too as their format is slightly different now.

I've tweaked the colour application logic a bit more, so now it's like this:

AFAICT from testing, it's the way the NVIDIA Control panel does it, so I'm hoping that could help unlock this issue. To be honest we may not be able to work it out, and iI may need help from NVIDIA before I figure this out :(.

Let's see how it goes.

terrymacdonald commented 3 years ago

@Mobeeuz I've been hunting around in other people's code, and I think I've found the underlying issue. It think it's in the Windows CCD library I created. That library uses the Windows CCD DisplayConfig API to make changes to the windows display layout. It 'appeared' to make changes to the layout and the settings like refresh rates/colour settings during my testing, but after some research over the last couple of days I'm not so sure.

Soroush's WindowsDisplayAPI uses another windows API called DisplayContext to make changes to the settings, and it appears the ChangeScreenResolution programme does too. So I'm going to go implement that in the CCDInfo library, and then see if that makes things work.

So, in a week or so I will likely ask you to please test the colour settings for CCDInfo for some non-surround window display layouts just to confirm that works. Please note CCDInfo won't work for surround displays, but it will allow me to confirm that the colour settings works for you.

Once that's tested, then I'll port the library into NVIDIAInfo and you can test it works with surround layouts. I'm hopeful that at the end of this we'll have a fully working video library as originally promised.

The one downside of all this, is that we'll need to have people recreate their Display Profiles if they want specific refresh rates or colour settings. as the new DisplayContext stuff will require additional settings to be tracked.

Really appreciate your testing. Unfortunately as you can tell this stuff isn't simple, and a lot of it isn't well documented. It requires trial an error. But the result will be worth it!

Thanks Terry

Mobeeuz commented 3 years ago

Wasn't sure if you still wanted another test, here it is anyway.

NVD5.zip

terrymacdonald commented 3 years ago

@Mobeeuz That is why I enjoy working with you so much! You're a dream tester.

To say thanks, I worked for a couple of hours today to try and shoehorn in my first attempt at colour settings/refresh rate setting storage into CCDInfo. CCDInfo helps me test the built-in windows library functions without any specific AMD or NVIDIA logic. It's the one that required the addition of the Windows GDI library functions that are used to manipulate colour settings and refresh rate.

Specifically, I'm using the EnumDisplayDevices and ChangeDisplaySettingsEx GDI functions to get and set the new device modes. These device modes have been around since Win95, and so I'm really puzzled why we have to still use them when Win10 brought out the CCD API which was supposed to replace the GDI one... but there you go.

The GDI libraries allow us to change the following settings:

I'm hoping that will cover most of the basic settings we need.

I've created a special alpha build for you of CCDInfo which doesn't have any of the special error checking or anything in it. It's really just the quickest build I could make to get you some code to test. I'd really appreciate it if you could test and see if that fixes your issues.

CCDInfo-v1.0.9b.zip

Please note, you won't be able to get it to change your NVIDIA Surround settings for you, as CCDInfo doesnt know about that.

Instead, please perform the following tests:

TEST 1: Non-surround

TEST 2: Surround enabled

If this works, then that means the new GDI API works for our needs. I will then be able to make the WinLibrary GDI code more reliable, and I'll then be able to add it into NVIDIAInfo as the next step.

Anyway... fingers crossed.

Terry

terrymacdonald commented 3 years ago

@Mobeeuz I've made those same GDI API changes to NVIDIAInfo v1.1.0 now, so in addition to performing those steps I highlighted previous with CCDInfo, I'd like you to also try and use NVIDIAInfo v1.1.0 to swap between surround and non-surround settings. Based on my testing, it seems to apply all the right settings in the right places, but I'd really like to know if it works for you 🤞.

You can get NVIDIAInfo v1.1.0 from https://github.com/terrymacdonald/NVIDIAInfo/releases/tag/v1.1.0

Thanks Terry

Mobeeuz commented 3 years ago

Some progress... My settings are no longer locked and unchangeable, however the default to custom setting doesn't appear to change. The settings are correct, but it doesn't toggle.

NVD6.zip

terrymacdonald commented 3 years ago

OK @Mobeeuz! We're definitely getting closer.

I've now restructured the NVIDIALibrary to ensure that NVIDIA colour settings are applied appropriately at the right times. The process flow is now like so:

  1. Mosaic changes are made (either on/off)
  2. If the target display has windows colour settings, then the NVIDIA colour settings are removed
  3. Windows display layout changes are made
  4. Windows colour /refresh changes are made
  5. If the target display has NVIDIA colour settings, then the new NVIDIA colour settings are applied

I'm quite hopeful that this will be the one that works properly for you. 🤞

Can you please download NVIDIAInfo v1.1.1 from https://github.com/terrymacdonald/NVIDIAInfo/releases/tag/v1.1.1 and give it a go?

Thanks for all your hard work testing this!

Thanks Terry

terrymacdonald commented 3 years ago

@Mobeeuz There is an update...please use NVIDIAInfo v1.1.2 instead :)

https://github.com/terrymacdonald/NVIDIAInfo/releases/tag/v1.1.2

terrymacdonald commented 3 years ago

And, once you've tested the above, @Mobeeuz, I've got a new test build for DisplayMagician v2.1.0 for you to test for me please!

DisplayMagicianSetup-v2.1.0.7.zip

If this fixes your current color settings issue, then please log that fact here.

If you find another issue not related to this issue, then please log a new issue so that I can troubleshoot it separately. I need to iron out all the bugs so that I can bake a proper release.

Thanks Terry

Mobeeuz commented 3 years ago

Problems are gone, things working now as expected - excellent work!

terrymacdonald commented 3 years ago

OMG that's great! I've been having so many problems with Cloned screens I was worried this wouldn't work too. Great to hear.

Thanks again for all your testing @Mobeeuz. It really, really helped.

Thanks

Terry MacDonald

On Tue, 19 Oct 2021 at 14:02, Mobeeuz @.***> wrote:

Problems are gone, things working now as expected - excellent work!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/45#issuecomment-946282499, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LGJS3HFHKP5CFY3RE3UHS7T3ANCNFSM5FKOKWIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

terrymacdonald commented 3 years ago

@Mobeeuz I've managed to find and solve a WHOLE BOAT LOAD of issues over this last week. And I'm now at a point that I think the video library stuff is working. I'm confident enough to provide you with test version 2.1.0.35 for you to test. It should handle cloned windows, NVIDIA surround, weird layouts (even a combined surround with extra single display works!) and it works after multiple reboots.

I'd really love you to install this version, and to test the different displays. I really want you to try and break it :D.

DisplayMagicianSetup-v2.1.0.35-test.zip

One important thing to note - it will require you to create new Display Profiles as part of the upgrade. It turns out I needed a bit more information than was available in the DisplayProfiles_2.0.json format I created earlier. So this new 2.1 version creates a DisplayProfiles_2.1.json instead.

Hopefully everything works, and then I can release a new DisplayMagician version and move on to adding other features!

Thanks Terry

adam-jan commented 2 years ago

Hi!

I was never able to save/restore BPC with my NVIDIA triple/surround profiles, but with version 2.1.0.269 it works perfect! Thanks!

Best regards, Adam.