Color calibration profiles download feature (X-Rite replacement)

Rast1234 commented 1 day ago

Version

2.24.1

OS

Windows 10 22H2

Device

Legion Pro 7 16ARX8H

Is your feature request related to a problem?

My laptop comes with a color-calibrated LCD panel. Lenovo offers X-Rite Color Assistant app which should download and install profiles for a given panel, but the app is finicky and doesn't work for everyone.

Calibration profiles are zipped, X-Rite tries to download them but

zip file contents for my panel have different modification dates and this fails installation (dates should all be the same)
app reports an obscure cryptographic error most likely related to HTTPS

For more details and other user reports you can check this thread: https://forums.lenovo.com/t5/ThinkPad-P-and-W-Series-Mobile-Workstations/X-Rite-Color-Assistant-fails-to-restore-color-profiles/m-p/5159754

I guess Lenovo does not provide these zips with *.icm files on download page for each laptop model, because they are unique for each panel, so you need a serial number, not just machine type.

People found out what X-Rite does:

constructs an url with model + serial
calls API to generate temporary link
downloads zip with *.icm profiles
installs them to %WINDIR%\System32\spool\drivers\color

So it's feasible to implement in LLT as it's a perfect Vantage/SystemUpdate replacement :)

Here's a Powershell implementation of zip retrieval:

$model = $(Get-WmiObject Win32_ComputerSystem).Model
$serial = $(Get-WmiObject Win32_Bios).SerialNumber
$url = "https://supportapi.lenovo.com/ColorCalibration/GetFileBySn?sn=1S$model$serial"
$jsonResponse = Invoke-RestMethod -Uri $url
echo "Temporary link to download calibration zip: $jsonResponse"
$response = Invoke-WebRequest $jsonResponse -verbose
$fileName = $response.BaseResponse.ResponseUri.AbsolutePath.Split('/')[-1]
[IO.File]::WriteAllBytes($fileName, $response.Content)
echo "Saved to $fileName"
echo "Now copy ICM files to C:\Windows\System32\spool\drivers\color"

I didn't look for a way to actually bind profiles to a monitor though, that turned out to be pretty straightforward to do manually in native Color management tool.

How would you like the problem to be solved?

Add a feature to LLT to download and install color calibration profiles for laptop's LCD panel

What alternatives have you considered?

X-Rite Color Assistant app offered by Lenovo support page doesn't work for everyone and is really not needed as people figured out how it works

Additional information

If you don't have hardware which comes with color profiles and X-Rite, i can help with testing and/or share my serial to debug URL construction logic

Ace-Radom commented 22 hours ago

What do you mean by 'not working for everyone?' XRite function is only provided for legion 7 or higher models as I know. I'm wondering if you mean XRite sometimes doesn't work on those officially supported models or just, not for all models. But it's really cool if we can implement this. If needed I can also provide datas for chinese-specific models with XRite.

Rast1234 commented 22 hours ago

I mean, it fails on officially supported models like in my case. Looks like it happens not quite often but also i'm sure not all hardware owners go ahead and figure out they can obtain color profiles from a 3rd party app even if it's offered on support page.

I honestly have no idea what models or product lines are color-calibrated and supported by XRite with cloud color profile downloads. I just found that thread on Lenovo site and it appears there are other users who can't get XRite to work. Notably this is Thinkpad support forum, not Legion - so maybe i'm the lucky one Legion owner with bad .zip in their cloud. Still, other users figured out what the algorithm is and i compiled this knowledge together so maybe it will be useful for other Legion owners to avoid bloatware. XRite app is 127 megabytes and all it does is download zip + install icms + UI to select one of the profiles lol

Ace-Radom commented 22 hours ago

I mean, it fails on officially supported models like in my case. Looks like it happens not quite often but also i'm sure not all hardware owners go ahead and figure out they can obtain color profiles from a 3rd party app even if it's offered on support page.

Weird. I'll go over all info you provided. Tbh I'm also worrying if the XRite provided on my device, which is chinese-specific, works differently since that's the thing lenovo does all the time, but I'll give some tests.

XRite app is 127 megabytes and all it does is download zip + install icms + UI to select one of the profiles lol

Indeed xD.

Rast1234 commented 22 hours ago

Check out the thread i linked, it has all the info how XRite works: we sniffed its traffic, checked the logs, and replicated profile install manually to make app happy and get through errors. So one can pretty much figure out how it works

BartoszCichecki commented 22 hours ago

This seems like a neat addition. The only problem is that, from what I know X-Rite applies some additional transformations to the icm files before importing them.

SHA-1 Checksums for profiles directly from the downloaded ZIP do not match the ones that were restored via X-Rite version 2.53. This is the result that I got on my 16IAX7. There is an announcement posted on Legion Series Discord precisely about this and it seems that it's true.

This is the script that is ran on downloaded profiles:

chcp 65001
"C:\Program Files (x86)\X-Rite Color Assistant\ColorCalibrationManager.exe" --install-profile "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\DisplayProfiles\TPLCD_0A1F_Default.icm"  --install-profile "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\DisplayProfiles\TPLCD_0A1F_Native.icm"  --install-profile "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\DisplayProfiles\TPLCD_0A1F_Native_HDR.icm"  --install-profile "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\DisplayProfiles\TPLCD_0A1F_REC709.icm"  --install-profile "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\DisplayProfiles\TPLCD_0A1F_sRGB.icm"  --set-registry-path "HKEY_LOCAL_MACHINE\SOFTWARE\X-Rite, Inc\X-Rite Color Assistant\PanelInfo" --add-registry-key HdrSupported --registry-value SoQF8Nj0UfeE3ljhPwxtBA== --add-registry-key DefaultProfileColorSpace --registry-value bEm7JcHCuWSz8q2Fq2PPQg== --add-registry-key DefaultProfileGamma --registry-value 4tTvyPewlq86wu4fdCsGeA== --add-registry-key RequiredProfiles --registry-value zlMhatnVTz9IMMSPcGzNvJ6yS/njrWU/DkWLKFywj00= --add-registry-key ProfileFormat --registry-value EUXmXPVBjokHcAfvFZcIzQ== --add-registry-key DisplayDeviceId --registry-value MONITOR\BOE0A1F\{4d36e96e-e325-11ce-bfc1-08002be10318}\0000 > "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\Users\barto\scripts\stdout.txt" 2>&1 && echo %errorLevel% > "C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\Users\barto\scripts\exitcode.txt"

So there is also ColorCalibrationManager.exe involved in this process, which probably alter the icms.

BartoszCichecki commented 21 hours ago

Another finding is the X-Rite logs that contain definite proof of alterations:

2024.10.31 21:54:09,129.650 [25096] INFO ProfileModifier <>: Changes are required, old tag values: [0.449439,0.223865,0.010132]
2024.10.31 21:54:09,129.689 [25096] INFO ProfileModifier <>: new tag values: [0.436028,0.222479,0.013926]
2024.10.31 21:54:09,129.710 [25096] INFO ProfileModifier <>: Changes are required, old tag values: [0.379019,0.714168,0.093874]
2024.10.31 21:54:09,129.730 [25096] INFO ProfileModifier <>: new tag values: [0.385101,0.716897,0.097092]
2024.10.31 21:54:09,129.749 [25096] INFO ProfileModifier <>: Changes are required, old tag values: [0.135760,0.061967,0.721202]
2024.10.31 21:54:09,129.768 [25096] INFO ProfileModifier <>: new tag values: [0.143091,0.060624,0.714191]

So unless someone can figure out what happens really, I don't see how this can be added to LLT.

Rast1234 commented 21 hours ago

Ouch. Too bad they went this way, instead of updating icms in the cloud...

shockdude commented 20 hours ago

Ouch. Too bad they went this way, instead of updating icms in the cloud...

For what it's worth, you can install the profiles manually, and still be able to run X-Rite Color Assistant 2.53 to apply the transformations/patches to the profiles. The patching even works offline.

Rast1234 commented 20 hours ago

For what it's worth, you can install the profiles manually, and still be able to run X-Rite Color Assistant 2.53 to apply the transformations/patches to the profiles. The patching even works offline.

It's offtop i guess, but since i can't make XRite to restore profiles, how do i trigger patching offline, if i have icm files? I tried the script, tried switching between now-installed profiles - no ProfileModifier in the log. Maybe my profiles just don't need to be patched, who knows.

BartoszCichecki commented 19 hours ago

I would try looking into the log files of X-Rite to see why it's not restoring. Thread you linked is from 2 years ago, and X-Rite had a big update couple months back.

Rast1234 commented 19 hours ago

I just opened xrite binaries in ghidra and ProfileModifier, Changes are required, old tag values: strings are in XRiteColorAssistant.exe (main GUI), not ColorCalibrationManager.exe (CLI tool). I'm not reverse engineering guy, no idea where this gets called from though

BartoszCichecki commented 19 hours ago

Let's keep the conversation here about facts. Respectfully, we don't need to know about every thing that you do. Otherwise it will quickly spiral into off topic and will be difficult to follow.

shockdude commented 18 hours ago

For what it's worth, you can install the profiles manually, and still be able to run X-Rite Color Assistant 2.53 to apply the transformations/patches to the profiles. The patching even works offline.

It's offtop i guess, but since i can't make XRite to restore profiles, how do i trigger patching offline, if i have icm files? I tried the script, tried switching between now-installed profiles - no ProfileModifier in the log. Maybe my profiles just don't need to be patched, who knows.

Make sure X-Rite 2.53 is installed: https://downloads.xrite.com/Downloads/Software/XRiteColorAssistant/Lenovo/v2.53.0/XRiteColorAssistantSetup.exe Have the downloaded profiles installed in the system with Color Management Open X-Rite normally. You might get a UAC prompt, that's the part where it's about to patch the profiles. Manually select each color profile in X-Rite, then close X-Rite and finish assigning your profiles with Windows Settings or Color Management. I think the patched profiles should be named differently.

If there's no UAC prompt or the patches aren't named differently, that's weird 🤷‍♂️

Rast1234 commented 9 hours ago

Have the downloaded profiles installed in the system with Color Management Open X-Rite normally. You might get a UAC prompt, that's the part where it's about to patch the profiles. Manually select each color profile in X-Rite, then close X-Rite and finish assigning your profiles with Windows Settings or Color Management. I think the patched profiles should be named differently.

Thank you, it worked. Interesting bit: for some profiles log has No significant change in new and old values, hence no change is required. XRite really does some complex conversion there, so just downloading profiles won't be enough.

Relevant log if someone is interested

``` 2024.11.01 13:03:38,188.517 [12620] INFO MainWindow <>: MainWindow: CorrectTagsForAllProfiles started 2024.11.01 13:03:38,188.754 [12620] TRACE MainWindow <>: allProfileData size : 4 2024.11.01 13:03:38,188.966 [12620] TRACE MainWindow <>: input profile Path: C:\Windows\system32\spool\drivers\color\TPLCD_0B11_REC709.icm 2024.11.01 13:03:38,188.987 [12620] TRACE MainWindow <>: output profile Path: C:\ProgramData\X-Rite, Inc\X-Rite Color Assistant\Temp/TPLCD_0B11_REC709.icm 2024.11.01 13:03:38,189.002 [12620] TRACE ProfileDataObtainer <>: FetchProfileType start... 2024.11.01 13:03:38,190.728 [12620] INFO MainWindow <>: Profile Type : PROFILE_TYPE_MATRIX_TRC_MHC2 2024.11.01 13:03:38,193.197 [12620] TRACE ProfileModifier <>: getChromaticity numCoord : 3 2024.11.01 13:03:38,193.244 [12620] TRACE ProfileModifier <>: getChromaticity colorSpace : 1 2024.11.01 13:03:38,193.264 [12620] TRACE ProfileModifier <>: coordinates 00 0.436028 2024.11.01 13:03:38,193.278 [12620] TRACE ProfileModifier <>: coordinates 01 0.222479 2024.11.01 13:03:38,193.292 [12620] TRACE ProfileModifier <>: coordinates 02 0.0139264 2024.11.01 13:03:38,193.305 [12620] TRACE ProfileModifier <>: coordinates 10 0.385101 2024.11.01 13:03:38,193.318 [12620] TRACE ProfileModifier <>: coordinates 11 0.716897 2024.11.01 13:03:38,193.331 [12620] TRACE ProfileModifier <>: coordinates 12 0.0970922 2024.11.01 13:03:38,193.343 [12620] TRACE ProfileModifier <>: coordinates 20 0.143091 2024.11.01 13:03:38,193.356 [12620] TRACE ProfileModifier <>: coordinates 21 0.0606241 2024.11.01 13:03:38,193.369 [12620] TRACE ProfileModifier <>: coordinates 22 0.714191 2024.11.01 13:03:38,200.180 [12620] INFO ProfileModifier <>: No significant change in new and old values, hence no change is required 2024.11.01 13:03:38,208.422 [12620] INFO ProfileModifier <>: Changes are required, old tag values: [0.447913,0.222324,0.008606] 2024.11.01 13:03:38,208.447 [12620] INFO ProfileModifier <>: new tag values: [0.436028,0.222479,0.013926] 2024.11.01 13:03:38,208.464 [12620] INFO ProfileModifier <>: Changes are required, old tag values: [0.381369,0.708766,0.089784] 2024.11.01 13:03:38,208.480 [12620] INFO ProfileModifier <>: new tag values: [0.385101,0.716897,0.097092] 2024.11.01 13:03:38,208.494 [12620] INFO ProfileModifier <>: Changes are required, old tag values: [0.134951,0.068910,0.726833] 2024.11.01 13:03:38,208.510 [12620] INFO ProfileModifier <>: new tag values: [0.143091,0.060624,0.714191] 2024.11.01 13:03:38,209.924 [12620] INFO MainWindow <>: Did profile got corrected, for profile TPLCD_0B11_REC709.icm: 1 ```

BartoszCichecki / LenovoLegionToolkit