Derive correct ColorMatrix by checking CalibrationIlluminant

[CharlesRT2]

In reply to a query about why RT's camera white balance (and often the auto white balance, too) is wildly high for some cameras, ilias_giarimis diagnosed the problem:

Usually RT shows abnormal WB values for raws of non fully supported cameras (missing colormatrix data) or cameras with different than the expected colormatrix2 at D65 illumination in DNG exif. This last is what happens with this DNG sample … it reports a D50 colormatrix2 … RT supposes a D65 and the calculations fail. If we change the matrix to D65 (copy from Adobe dcp or just convert the DNG with DNG converter the values calculated are temp: 5222 tint: 1.030 https://discuss.pixls.us/t/why-very-high-white-balance-temperature-for-some-cameras/10498/2

So would it be an easy fix that RT check the two color matrices, or the two calibration illuminants in the Exif, and handle cases like the D50 one in his diagnosis?

And maybe this fix would help with issue #3894 White balance error on Sony RX100, too? I've seen the wildly high temperature on Sony raw files (sorry, do not remember which of their cameras).

[Beep6581]

@CharlesRT2 upload as many problematic files as you can find, use https://filebin.net/

[CharlesRT2]

@CharlesRT2 upload as many problematic files as you can find ...

I checked several Sony camera .arw files and no problem. I saw a very high Kelvin temperature on one of these several years ago. Maybe it was an earlier version of RT that showed me a problem.

I already posted a Leica SL example at https://pixls-discuss.s3.dualstack.us-east-1.amazonaws.com/original/3X/d/e/deece1794e20091f4e8ec518be48267af6957e94.DNG

I found that not all Leica camera models have the problem, but here is a Leica Q example. It is at https://filebin.net/ka231giw5py3n4w3/WBcalc_LeicaQ.DNG?t=xkf8xe6t for one month.

[Beep6581]

It is typical for ColorMatrix1 to correspond to StdA and for ColorMatrix2 to correspond to D65. This seems to be just a custom, as I didn't find any reference to a correct order in the DNG specification.

It would solve a growing number of problems if RawTherapee didn't blindly pick ColorMatrix2 assuming it's the daylight one, but instead checked the CalibrationIlluminant.

From personal experience, the daylight CalibrationIlluminant is not always D65, e.g. the Realme RMX1851 phone uses D55. However, the other illuminant is always StdA. RawTherapee would need to find the daylight illuminant, and apply a chromatic adaptation transform to whatever illuminant RawTherapee uses internally (D50 or D65?).

[Entropy512]

Of note, from a brand new (2-3 hours old) Google Pixel 4:

Calibration Illuminant 1        : D65
Calibration Illuminant 2        : Standard Light A

Pretty much the exact opposite of the custom you mention here.

What I'm seeing is that, at least on first inspection (could be a PEBCAK error though), the colors seem to be what I'm used to seeing with DNGs with horribly broken color matrices.

Where in the source is the handling of the two calibration illuminants, and would swapping illuminants 1/2 cause an issue in the current implementation?

I probably won't be able to look at this until November, the arrival of the Pixel is due to two last-minute realizations (which are why I STILL haven't looked at prioritization of DCP profile vs. DNG data for HDRMerge-created images or other "synthetic multiframe DNG" approaches): 1) I really don't want to lug an A7III + 24-105G around Disney World next week 2) My old phone was exhibiting signs of hardware failure (increasing frequency of failure to resume from suspend, and spontaneous reboots).

But this is definitely now near the top of my list of things to look at once I return from Vacation #2. (Usually my family does one single long two-week vacation, doing two separate one-week trips is really throwing me off...)

[CharlesRT2]

Workaround: I found for a couple of Leica cameras that you can run the DNG through Adobe DNG Converter. I didn't do technical analysis, but the converted DNGs have typical temp and tint values in RT, while the unconverted ones have temps of 9000 or more and tints of 0.60 or less. I just find it easier to work with the converted DNGs.

[Beep6581]

Pretty much the exact opposite of the custom you mention here.

Yes, common for smartphones.

Some of the effects of illuminants not in the expected order is that the white balance values in RawTherapee are way off, and that the mixing in dual-illuminant profiles is done incorrectly - if you use a dual-illuminant profile and the image would require a mix of 90% StdA and 10% D50, you get 90% D50 and 10% StdA.

would swapping illuminants 1/2 cause an issue in the current implementation?

Swapping the matrices in the DNG file using ExifTool solves the problem.

increasing frequency of failure to resume from suspend

Was that a Samsung Galaxy by any chance?

[heckflosse]

@Entropy512

Where in the source is the handling of the two calibration illuminants

https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/dcraw.cc#L6626 https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/dcraw.cc#L6749 https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/dcraw.cc#L6632 https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/dcraw.cc#L6669

[Entropy512]

Pretty much the exact opposite of the custom you mention here.

Yes, common for smartphones.

Some of the effects of illuminants not in the expected order is that the white balance values in RawTherapee are way off, and that the mixing in dual-illuminant profiles is done incorrectly - if you use a dual-illuminant profile and the image would require a mix of 90% StdA and 10% D50, you get 90% D50 and 10% StdA.

OK, thanks, good place to start on investigation.

would swapping illuminants 1/2 cause an issue in the current implementation?

Swapping the matrices in the DNG file using ExifTool solves the problem.

Interestingly, it seems to have made no difference... But I need to run a few more experiments. Unlikely to happen before November at this point.

increasing frequency of failure to resume from suspend

Was that a Samsung Galaxy by any chance?

Nope - Moto Z2 Play. Reliable for two years, then started acting up. The ramoops console in pstore had nothing after a failure... grumble... As I said though, unusual circumstances next week of really needing something smaller than what I usually use camera-wise, which is why I got the Pixel.

Ingo - thanks for the links, I'll take a look when I get back from vacation. Tonight and tomorrow are packing, I don't think I'll get another chance to poke at this.

Ugh, dcraw.cc, guess I'll have to FORC3 myself to become familiar with Dave's macros again. :)

Hmm, this is interesting: /* RT -- do not use CameraCalibration matrices for DNGs - see #4129 */

From my DNGS:

Camera Calibration 1            : 0.6719 0 0 0 1 0 0 0 0.3359
Camera Calibration 2            : 0.4531 0 0 0 1 0 0 0 0.6484

Before I leave I'll try and get some sort of sample images on raw.pixls.us - I now have a ColorChecker but whatever I take with that won't be under a proper illuminant due to time constraints. Last night's photos are ones I don't feel comfortable putting up on raw.pixls.us

[Entropy512]

Reverting https://github.com/Beep6581/RawTherapee/commit/1fdc2d7b49327c9e08ca037d04a51a7c22e159cd#diff-9967648f0eb70b821877dd6105e978ef changes the result of using the white balance colorpicker significantly, but doesn't change behavior of colors after selecting WB on the same point.

The image shown in the preview window and the image shown in the thumbnails become VASTLY different in this case, with the thumbnails gaining a very greenish tint.

OK, really out of time, I swear, I have to go to work now.

[Entropy512]

I uploaded samples to raw.pixls.us, although they don't appear to be showing up in the repo, I'm guessing it takes a while?

I think the Pixels may be oddball animals that might need their own issue... They appear to report JPEG compression but 16 bits/sample for the raw image - and it does appear that RT is seeing mosaiced data, but it almost seems like the data has already had the color matrices applied - if the input profile is set to "Camera Standard", the colors are desaturated similar to what I have seen when a camera's color matrix is completely wrong (such as the Mi Sphere's DNGs), but if the input profile is "None", the colors look properly saturated.

In the event that the samples I uploaded don't actually show up in the repo - the Pixel 3A sample that is available behaves very similarly.

[Entropy512]

OK, disregard basically everything I've said today.

https://9to5google.com/2019/10/23/google-camera-7-2/

Note all of the "added back feature we accidentally removed" notes. It seems like 7.1 is a severely broken version that should never have wound up in users' hands. I'm wondering if this is because the phones were supposed to ship yesterday and arrive at the earliest today, but instead shipped Monday and arrived yesterday.

The DNG metadata is COMPLETELY different now - the CameraCalibration matrices are no longer wildly far off from 1.0 for R/B, and there's even a HueSatMap!

Now to see if I can find where this new depth data is. :)

[heckflosse]

I am working on this issue now

[heckflosse]

@CharlesRT2

I found that not all Leica camera models have the problem, but here is a Leica Q example.

This kind of fixes the issue with Leica Q files. https://github.com/Beep6581/RawTherapee/commit/5732500e2fd0b4c7c8b8744c511435ad1eacbba3

Leica Q dng files have

Calibration Illuminant 1 : Standard Light A
Calibration Illuminant 2 : D50

Old dcraw code did not find D65 and used the first one which is Standard Light A. New code uses D50

[Beep6581]

@heckflosse what happens now if a raw file has this?

CalibrationIlluminant1: D55
CalibrationIlluminant2: Standard Light A

Do not use Standard light A if D65 does not exist

If D65 does not exist (are both illuminants checked for that string?), what is used?

[heckflosse]

@Beep6581 In your example D55 will be used