Black crush and color deviations with some ICC profiles

[haasn]

Continuation of the discussion in #534.

Some ICC profiles are experiencing black crush when used in mpv, in particular ones that use complex transfer curves rather than simple pure powers.

Proposed solutions including reworking the 3DLUT/lcms mechanism to pick a connection space more gracefully.

[UliZappe]

To get some terminology clear, there still is an intermediate space here - that space is just BT.709 in your example. There has to be some intermediate space by design, because the 3DLUT is a function from one space to another. The former space is the “intermediate space” that we generate the 3DLUT against.

OK, every color space conversion, by its very essence, requires two color spaces. But in my terminology, an intermediate color space would be a space that is neither the source nor the target space (nor, strictly speaking, the PCS that is used in all non “device link” conversions).

So clearly, if the video material is in BT.709, BT.709 is no intermediate space in the way I used the terminology. (It would be if you used it for all video material, BT.709 or not).

Are you implicitly suggesting we generate the 3DLUT against XYZ or Lab? Because both of those would be possible.

No I wasn’t, although, now you say it, this might at least work better than using BT.2020 for this purpose. But I still think the optimal solution would be a dedicated 3DLUT for a specific video color space – monitor profile combination. Just as a (well-constructed) device link profile is superior to two profiles connected via PCS.

It was working fine for me, which may be simply because I have a wide gamut profile that uses a pure power curve - both of which are ideal combinations for minimizing the error introduced by the change.

Probably – just like my test with Pro Photo RGB as the monitor profile, where everything worked fine.

[haasn]

OK, every color space conversion, by its very essence, requires two color spaces. But in my terminology, an intermediate color space would be a space that is neither the source nor the target space (nor, strictly speaking, the PCS that is used in all non “device link” conversions).

There are many different conversions going on here, each one of the spaces that is not at the very beginning or very end of the pipeline is an “intermediate” space in my books.

During typical video playback, the source video will undergo the following color spaces:

1. space=Y'CbCr prim=BT.709 trc=BT.1886
2. space=RGB prim=BT.709 trc=BT.18886
3. space=RGB prim=BT.709 trc=linear
4. space=RGB prim=BT.709 trc=sigmoid
5. space=RGB prim=BT.709 trc=linear
6. space=XYZ prim=BT.709 trc=linear
7. space=XYZ prim=BT.2020 trc=linear
8. space=RGB prim=BT.2020 trc=linear
9. space=RGB prim=BT.2020 trc=gamma2.4
10. space=RGB prim=monitor trc=monitor

I have highlighted steps 5 which is where CMS begins and step 9 which is what gets fed into the 3DLUT. We could use a different set of conversions here.

[UliZappe]

@4ad

So, in the end, I think we're looking at two problems here? One is that the intermediate BT.2020 color space conversion introduces color distorsion. We can see this by measuring the colors with various profiles.

The other problem is that either LittleCMS itself, or the way mpv uses LittleCMS can't handle complex profiles properly. We can see this by measuring black levels with simple and complex profiles.

Yep, it seems so.

black levels are good with simple profiles (but colors are off)

Which confirms that it’s two issues.

[4ad]

Repeating the experiments with BT.709 gamma 1.961 (uploaded in the previous thread), which uses a simple power function for the transfer curve.

I am reading native values. This test also measures color.

Program	black#1	black#7	black#8	black#15	white#15
QuickTime	1-1-1	8-8-8	9-9-9	17-17-17	238-238-238
mpv (recent)	1-1-0	7-8-7	8-9-8	17-17-16	239-238-240
mpv (before)	1-1-1	8-8-7	9-9-8	17-17-17	239-238-240
mpv (after)	1-1-1	8-8-7	9-9-8	17-17-17	239-238-240

Program	bright red	bright green	bright blue	medium red	medium green	medium blue
QuickTime	255-0-0	0-255-1	0-0-255	191-0-0	0-191-0	0-0-192
mpv (recent)	254-5-4	12-254-7	4-4-254	192-5-2	7-191-5	3-3-193
mpv (before)	255-1-1	0-255-1	0-1-255	192-0-0	0-191-0	0-1-193
mpv (after)	255-1-1	0-255-1	0-1-255	192-0-0	0-191-0	0-1-193

[UliZappe]

There are many different conversions going on here, each one of the spaces that is not at the very beginning or very end of the pipeline is an “intermediate” space in my books.

Ah, OK. But then I would differentiate between primaries conversions and TRC conversions. From my experience, the critical conversions are the primaries conversions.

So if we focus on these, your list becomes:

BT.709 BT.2020 Monitor

From this POV, there is one intermediate space, and this is BT.2020. This should remain BT.709, so that we get

BT.709 Monitor

i.e. no intermediate space in this sense.

I have highlighted steps 3 which is where CMS begins and step 7 which is what gets fed into the 3DLUT.

Uhm, the highlighted steps were 5 and 9?

[haasn]

Concerning the pipeline I just outlined, I have some two important observations to make:

The majority of these conversions are done using exact computations with 32 bit precision, so the margin of error is negligible both in theory and in practice.

The video gets rounded and clipped while in some spaces, due to various reasons:

1: inherently clipped to 8-10 bit precision (input video depth) 4: clipped to 16-bit precision (or more/less depending on texture depth) while upscaling, for performance reasons 9: “reduced” to ~8-bit precision while passing through the 3DLUT. 10: Clipped to 8-bit precision with dithering (monitor depth)

Of these operations, the most significant are the two 8-bit precision clips. That's most likely a big part of the reason why we are introducing so much distortion, because clipping to 8-bit BT.709 values and then again to 8-bit BT.2020 values reduces the overall bit depth below 8 bits because the “round numbers” are not aligned.

(There is another 8-bit clip in step 10 but this is less significant since it's dithered, so we retain much of the precision)

So if we want to minimize distortion, we have to make sure that space 9 and space 1 are as close as possible, to eliminate the effects of “double clipping”. This is the idea behind using a profile similar to the video source as the intermediate space. (Basically, it would be the same as in step 2)

But looking at this pipeline it seems obvious to me that we could eliminate distortion even more by going all the way back to the space used in step 1 - the input space - and generate the 3DLUT against this. That way the 3DLUT would essentially be a mapping from source Y'CbCr to monitor RGB.

(The internal calculations are performed with 64-bit floating point precision by LittleCMS, so the extra conversion inside LittleCMS should be essentially free)

[UliZappe]

But looking at this pipeline it seems obvious to me that we could eliminate distortion even more by going all the way back to the space used in step 1 - the input space - and generate the 3DLUT against this. That way the 3DLUT would essentially be a mapping from source Y'CbCr to monitor RGB.

Yes, exactly.

[haasn]

From my experience, the critical conversions are the primaries conversions.

Clearly, from the evidence in this discussion, the TRC selection matters just as much as the selection of the primaries - that's what's causing the black crush.

Uhm, the highlighted steps were 5 and 9?

Oops, I had duplicates in my numbering while editing and GitHub automatically changed them to count consistently.

[haasn]

How much does this patch improve things? (https://github.com/haasn/mpv/commit/2ed5e5a72d5c28ccdcd5ee1c132c10d2dfeaf6ec)

diff --git a/video/out/opengl/lcms.c b/video/out/opengl/lcms.c
index c956127..db93e08 100644
--- a/video/out/opengl/lcms.c
+++ b/video/out/opengl/lcms.c
@@ -197,7 +197,7 @@ bool gl_lcms_get_lut3d(struct gl_lcms *p, struct lut3d **result_lut3d)
         // because we may change the parameter in the future or make it
         // customizable, same for the primaries.
         char *cache_info = talloc_asprintf(tmp,
-                "ver=1.1, intent=%d, size=%dx%dx%d, gamma=2.4, prim=bt2020\n",
+                "ver=1.1, intent=%d, size=%dx%dx%d, gamma=1.961, prim=bt709\n",
                 p->opts.intent, s_r, s_g, s_b);

         uint8_t hash[32];
@@ -242,9 +242,9 @@ bool gl_lcms_get_lut3d(struct gl_lcms *p, struct lut3d **result_lut3d)
     if (!profile)
         goto error_exit;

-    // We always generate the 3DLUT against BT.2020, and transform into this
+    // We always generate the 3DLUT against BT.709, and transform into this
     // space inside the shader if the source differs.
-    struct mp_csp_primaries csp = mp_get_csp_primaries(MP_CSP_PRIM_BT_2020);
+    struct mp_csp_primaries csp = mp_get_csp_primaries(MP_CSP_PRIM_BT_709);

     cmsCIExyY wp = {csp.white.x, csp.white.y, 1.0};
     cmsCIExyYTRIPLE prim = {
@@ -253,9 +253,9 @@ bool gl_lcms_get_lut3d(struct gl_lcms *p, struct lut3d **result_lut3d)
         .Blue  = {csp.blue.x,  csp.blue.y,  1.0},
     };

-    // 2.4 is arbitrarily used as a gamma compression factor for the 3DLUT,
+    // 1.961 is arbitrarily used as a gamma compression factor for the 3DLUT,
     // reducing artifacts due to rounding errors on wide gamut profiles
-    cmsToneCurve *tonecurve = cmsBuildGamma(cms, 2.4);
+    cmsToneCurve *tonecurve = cmsBuildGamma(cms, 1.961);
     cmsHPROFILE vid_profile = cmsCreateRGBProfileTHR(cms, &wp, &prim,
                         (cmsToneCurve*[3]){tonecurve, tonecurve, tonecurve});
     cmsFreeToneCurve(tonecurve);
diff --git a/video/out/opengl/video.c b/video/out/opengl/video.c
index d13cd30..3a0c2c4 100644
--- a/video/out/opengl/video.c
+++ b/video/out/opengl/video.c
@@ -1768,9 +1768,9 @@ static void pass_colormanage(struct gl_video *p, enum mp_csp_prim prim_src,
     enum mp_csp_prim prim_dst = p->opts.target_prim;

     if (p->use_lut_3d) {
-        // The 3DLUT is hard-coded against BT.2020's gamut during creation, and
+        // The 3DLUT is hard-coded against BT.709's gamut during creation, and
         // we never want to adjust its output (so treat it as linear)
-        prim_dst = MP_CSP_PRIM_BT_2020;
+        prim_dst = MP_CSP_PRIM_BT_709;
         trc_dst = MP_CSP_TRC_LINEAR;
     }

@@ -1800,10 +1800,10 @@ static void pass_colormanage(struct gl_video *p, enum mp_csp_prim prim_src,
     }
     if (p->use_lut_3d) {
         gl_sc_uniform_sampler(p->sc, "lut_3d", GL_TEXTURE_3D, TEXUNIT_3DLUT);
-        // For the 3DLUT we are arbitrarily using 2.4 as input gamma to reduce
+        // For the 3DLUT we are arbitrarily using 1.961 as input gamma to reduce
         // the severity of quantization errors.
         GLSL(color.rgb = clamp(color.rgb, 0.0, 1.0);)
-        GLSL(color.rgb = pow(color.rgb, vec3(1.0/2.4));)
+        GLSL(color.rgb = pow(color.rgb, vec3(1.0/1.961));)
         GLSL(color.rgb = texture3D(lut_3d, color.rgb).rgb;)
     }
     if (need_gamma)

It's not the full idea I proposed (using the input space), but for BT.709 videos this should be similar, perhaps.

[4ad]

With haasn@2ed5e5a.

First with BT.709 gamma 1.961. I am measuring native values.

Program	black#1	black#7	black#8	black#15	white#15
QuickTime	1-1-1	8-8-8	9-9-9	17-17-17	238-238-238
mpv (recent)	1-1-0	7-8-7	8-9-8	17-17-16	239-238-240
mpv (2ed5e5a)	0-1-0	7-8-6	8-9-7	17-17-16	238-238-239

Program	bright red	bright green	bright blue	medium red	medium green	medium blue
QuickTime	255-0-0	0-255-1	0-0-255	191-0-0	0-191-0	0-0-192
mpv (recent)	254-5-4	12-254-7	4-4-254	192-5-2	7-191-5	3-3-193
mpv (2ed5e5a)	255-0-1	0-255-1	0-1-255	192-0-1	0-191-1	0-0-192

With iMac color profile.

Program	black#1	black#7	black#8	black#15	white#15
QuickTime	1-1-1	7-7-7	8-8-8	16-16-16	241-241-241
mpv (recent)	0-0-0	3-4-3	4-5-4	15-16-14	241-240-242
mpv (2ed5e5a)	0-0-0	3-4-3	5-5-4	15-15-14	240-240-241

Program	bright red	bright green	bright blue	medium red	medium green	medium blue
QuickTime	235-51-36	117-251-76	0-26-246	183-37-25	90-195-57	0-17-191
mpv (recent)	235-50-34	115-251-75	1-26-247	183-36-24	89-194-56	2-17-192
mpv (2ed5e5a)	234-51-35	117-250-76	0-26-245	183-37-25	89-194-57	0-18-191

With BT.709 gamma 1.961, your patch improves color accuracy significantly.

With iMac color profile, your patch improves color accuracy, but less so, since the recent build wasn't too bad with this profile in the first place. Still, an improvement.

Black crush is not improved by your patch; of course, this is expected, since the problem is in LittleCMS, or with LittleCMS interaction which was not changed by your patch.

[haasn]

Wasn't black crush improved when using an sRGB input curve for the 3DLUT as well?

Another thing I would like to try is generating the 3DLUT against your monitor profile (i.e. extract a gamut and TRC from the profile and use that) - if that has similar performance, we could consider just doing that.

That would have the benefit of preserving the status quo of “one 3DLUT per ICC profile”.

We can also test generating it against XYZ or Lab, but I doubt this is going to improve black crush at all.

[4ad]

Wasn't black crush improved when using an sRGB input curve for the 3DLUT as well?

In haasn@114f69a? No, that made black crush slightly worse.

[4ad]

@haasn

I rebased my DCI-P3 patch on haasn@2ed5e5a, and now vo=opengl:target-prim=DCI-P3:target-trc=bt.1886 matches QuickTime within +/-1 if I set P3 gamma 1.961 as the display profile.

Of course it's not usable because this uses gamma=1.961 which is not right for my display (in other words P3 gamma 1.961 doesn't match my display, nor the "real" Display P3.icc).

My patch is at 4ad@dbed5e0.

[haasn]

I rebased my DCI-P3 patch on haasn@2ed5e5a, and now vo=opengl:target-prim=DCI-P3:target-trc=bt.1886 matches QuickTime within +/-1 if I set P3 gamma 1.961 as the display profile.

This does not surprise me, since in the absence of a 3dlut all operations are done with 16-32 bit precision, which is more than enough to meet the 8-bit requirement you gave it. (In fact, due to mpv's good dithering, the mpv result will be virtually exact overall)

[haasn]

A possibility that is somewhat far out there but possibly worth consideration in general is to use the 3DLUT for less and do more with exact shader computations - e.g. we could implement parts of the ICC spec in mpv directly, such as the bits concerning the transfer characteristics.

(Theoretically it would be possible to implement all of ICC in pure shader code and get a virtually perfect result that way, but the implementation overhead of such a feat would be monstrous - and the potential for errors due to bugs or negligence far greater.)

[4ad]

@haasn

Hmm, I have an interesting result.

So, I added target-trc=DCI-P3. Now, I don't have access to the standard, so I implemented the complex parametric transfer function by looking inside the color profile. In other words, I read the transfer function from the profile, just like LittleCMS does!

And guess what, the result is the same, I get black crush.

In other words, with the Display P3.icc color profile, vo=opengl:icc-profile-auto and vo=opengl:target-prim=DCI-P3:target-trc=DCI-P3, produce the same output, which has the same problem.

My code is here 4ad@5711136.

[haasn]

@4ad Maybe the correct result of a transformation through that profile results in crushed blacks even in theory?

It's not impossible that QuickTime is the one that gets this particular profile “wrong” - or that they have some extra logic that goes beyond simply transforming it in order to actively mitigate black crush.

[4ad]

I don't think so. On 1-Black Clipping.mp4 I should see 17-25 flashing, but I only see 19-25 flashing. Plus, in real movies, mpv is lacking shadow detail (not noise, genuine detail) that is visible in QuickTime.

[haasn]

And here we are again at the important distinction between “looking good” and being a correct implementation of the spec.

Sure, I don't doubt that crushed blacks are bad. But if that's the way your profile was generated, and if Little CMS correctly implements the standard, then this is the result we would and should expect.

Since you have independently extracted the transfer curve from the profile, implemented, and gotten the same result - I'm inclined to believe the problem here is not LittleCMS but the profile.

[4ad]

Well, the profile contains both a parametric tone response curve, and a 3*1024 point LUT called the "tone response code". Maybe they don't match, and LittleCMS uses the parametric curve, while QuickTime uses the LUT?

[haasn]

That would be a good thing to investigate. Inconsistencies like these often arise when different programs decide to use different versions of what should be redundant data, but isn't (e.g. due to bad profile generation).

Edit: Though I think the 3x1 “tone response code” might be for per-channel adjustments independent of the parametric tone response curve.

[4ad]

Well that's why I want to get the real standard.

[UliZappe]

QuickTime Player, mpv, ColorSync and Little CMS

In the following I’m trying to more clearly locate the root of the deviations we see in the reproduction of the dark patches.

Video Players

For mpv, I’m using my “working version” in this test that has no BT.2020 intermediate space and of which we know that it reproduces the colors very close to QuickTime Player (which it again did in my following tests). So I’m concentrating on the dark patches. Specifically, I look at the 9th patch which we can assume is RGB 8-8-8 in BT.709 gamma 1.961. I use the DigitalColor Meter to measure the colors.

CMMs

To evaluate the ColorSync and Little CMS CMMs which QuickTime Player and mpv use, respectively, I calculate the conversion from 8-8-8 in BT.709 gamma 1.961 to the tested monitor space using the Calculator in the ColorSync Utility application included with OS X for ColorSync and the transicc command line utility included with Little CMS for Little CMS.

Monitor Profiles

I have tried to use a representative collection of monitor profiles. I will list native monitor RGB values, since we are interested in the (relative) differences between the color values obtained in the four different ways, not the absolute values. Using native monitor RGB values minimizes the required calculations and focuses on what we want to know. The absolute values in the following table do not matter, only the differences.

I have omitted profiles with simple tone response curves, as we know that these work well.

We have not much discussed yet that when it comes to complex transfer curves, there are still two different variants: matrix profiles with such curves, and LUT profiles.

Note that as soon as we deal with LUT monitor profiles, rendering intents become relevant (whereas for matrix monitor profiles, only the chromatic adaptation of the absolute colorimetric intent might make a difference). In the table, I use p = perceptual, r = relative colorimetric, a = absolute colorimetric.

Here’s the rationale for the monitor profiles I used:

ICCv2, Matrix, complex tone response curve

sRGB (well known ICCv2 standard profile) eciRGBv2 ICCv2 (from the European Color Initiative; valid ICCv2 standard profile with an unusual tone response curve (L*)

ICCv4, Matrix, complex tone response curve

P3 (taken from OS X El Capitan, ICCv4 standard profile) eciRGBv2 ICCv4 (valid ICCv4 and supposed to deliver identical results to the ICCv2 version above)

outside of ICCv2/ICCv4 spec, Matrix, complex tone response curve

iMac (tagged as ICCv2, using ICCv4 features, real life relevance as it’s default profile in recent iMacs)

ICCv2 LUT profile

spring-LED.LUT (individual profile created with Quato iColor Display)

ICCv4 LUT profile

LED ipLUT (individual profile created with X-Rite i1 Profiler)

Test Results for the dark patch with RGB 8-8-8 (BT.709 gamma 1.961)

Nr./RI	Monitor Profile	QuickTime Player	mpv	ColorSync	Little CMS
1pr	eciRGB v2 ICCv2	6-6-6	3-4-3	4-4-4	3-3-3
1a				4-4-6	3-3-3
2pr	eciRGB v2 ICCv4	6-6-6	3-4-3	4-4-4	3-3-3
2a				4-4-6	3-3-3
3pr	sRGB	8-8-8	5-5-4	6-6-6	4-4-4
3a				7-6-6	4-4-4
4pr	P3	8-8-8	5-5-4	6-6-6	4-4-4
4a				7-6-6	4-4-3
5pr	iMac	8-8-8	5-5-4	6-6-6	4-4-4
5a				7-6-6	4-4-3
6p	spring-LED.LUT	16-16-16	1-1-0	15-16-14	12-12-12
6r				7-0-0	0-0-0
6a				4-0-0	0-0-0
7p	LED ipLUT	2-3-4	10-11-0	0-0-1	9-10-11
7r				10-11-0	7-8-0
7a				10-11-0	7-8-0

Comments

1. For all matrix profiles (nr. 1 – 5), the calculated results for both ColorSync and Little CMS are always darker than the actual colors in QuickTime Player and mpv. Maybe that’s because my assumption is wrong that the 16 dark patches in the video have RGB values from 0 – 15, or maybe my self-created BT.709 gamma 1.961 profile (which the calculated results use, in contrast to QuickTime Player and mpv) is somehow flawed. But we should not ruminate too much about that, because for our problem it’s only helpful if QuickTime Player and mpv display the dark tones a bit brighter than calculated.
2. For all matrix profiles (nr. 1 – 5), mpv and Little CMS are always darker than QuickTime Player and ColorSync. So unfortunately, this seems simply to be a characteristic of Little CMS, and nothing that mpv can improve upon.
3. Profiles 1 and 2 (an ICCv2 and an ICCv4 profile which are otherwise identical) sport exactly the same result. So Little CMS and thereby mpv has no issues with ICCv4 profiles.
4. Little CMS has problems with the absolute colorimetric rendering intent. For instance, eciRGB (profiles 1 and 2) has a D50 white point. Displaying a D65 color source such as Bt.709 in an absolute manner means eciRGB must shift the colors to blue. ColorSync does this, but Little CMS does not. From a lengthy discussion with the creator of Little CMS I know that this is a conscious decision of his. I think he’s wrong, but he thinks Apple and myself are wrong. So we should not go into this, and I think we don’t have to, as the absolute color rendering intent should be irrelevant for watching videos.
5. If we ignore the absolute rendering intent, profiles 3 – 5 produce exactly identical results. This means that the profiles are obviously very similar, but especially, it means that Little CMS and thereby mpv has no problems to deal with the out-of-spec profile nr. 5 correctly.
6. The differences in behavior for the two LUT profiles (nr. 6 and 7) are huge and almost the opposite: in profile 6, mpv is completely off, in profile 7, QuickTime Player is completely off. Looking at the results of Little CMS, it seems that mpv uses the relative colorimetric rendering intent as the default, when the perceptual intent would provide much better results (and it wouldn’t at least hurt in any other of the 7 test cases, though this only refers to this one dark patch, of course). An alternative might be to use black point compensation for the relative colorimetric rendering intent, which in case of profile 6 changes 0-0-0 into 13-12-2. Of course, QuickTime Player does not offer any way for this kind of fine tuning. What completely amazes me is the huge difference between these two LUT profiles; it seems anything is possible here. Nr. 6 is a ICCv2 profile created with software from the (now defunct) Quato; nr. 7 is a profile created with software from X-Rite. Whether it’s ICCv2 vs. ICCv4 or the different profiling software which makes the difference is not clear from this test.

Summary

For matrix profiles with complex tone response curves, it simply seems that Little CMS does not handle dark tones too gracefully, and that there is little mpv can do. The only thing I can think of is to replace Little CMS with ColorSync for the Mac platform, which would be technically possible, but of course a huge platform specific deviation.

For LUT profiles, the possible differences in behavior are baffling. The options for fine tuning that mpv offers (rendering intent, black point compensation) become much more relevant here than I would have thought. Perceptual might be a better default rendering intent than relative colorimetric (but this would need further confirmation).

[UliZappe]

Regarding comment nr. 1 above: Assuming that, for whatever reason, it is the 8th (not the 9th) patch in the test movie that has RGB 8-8-8, the table would look as follows (different values for QuickTime Player and mpv):

Nr./RI	Monitor Profile	QuickTime Player	mpv	ColorSync	Little CMS
1pr	eciRGB v2 ICCv2	5-5-5	3-3-2	4-4-4	3-3-3
1a				4-4-6	3-3-3
2pr	eciRGB v2 ICCv4	5-5-5	3-3-2	4-4-4	3-3-3
2a				4-4-6	3-3-3
3pr	sRGB	7-7-7	4-4-3	6-6-6	4-4-4
3a				7-6-6	4-4-4
4pr	P3	7-7-7	4-4-3	6-6-6	4-4-4
4a				7-6-6	4-4-3
5pr	iMac	7-7-7	4-4-3	6-6-6	4-4-4
5a				7-6-6	4-4-3
6p	spring-LED.LUT	15-15-15	0-0-0	15-16-14	12-12-12
6r				7-0-0	0-0-0
6a				4-0-0	0-0-0
7p	LED ipLUT	0-1-0	9-10-0	0-0-1	9-10-11
7r				10-11-0	7-8-0
7a				10-11-0	7-8-0

[haasn]

Why are you listing mpv under pr and nowhere else? mpv's default intent is relative colorimetric, and you can choose your own.

Regarding perceptual vs colorimetric, I use a LUT-based profile (created by ArgyllCMS) and I see no difference whatsoever between the two intents.

[haasn]

Does this patch improve things?

diff --git a/video/out/opengl/video_shaders.c b/video/out/opengl/video_shaders.c
index a083364..1b01d87 100644
--- a/video/out/opengl/video_shaders.c
+++ b/video/out/opengl/video_shaders.c
@@ -251,8 +251,18 @@ void pass_linearize(struct gl_shader_cache *sc, enum mp_csp_trc trc)
                              pow((color.rgb + vec3(0.055))/vec3(1.055), vec3(2.4)),
                              lessThan(vec3(0.04045), color.rgb));)
         break;
-    case MP_CSP_TRC_BT_1886:
-        GLSL(color.rgb = pow(color.rgb, vec3(1.961));)
+    case MP_CSP_TRC_BT_1886: {
+        double white = 1.0;
+        double black = 0.05 / 120; // 0.05cd/m² blacks on a 120 cd/m² display
+        double gamma = 1.961;
+
+        double dif = pow(white, 1/gamma) - pow(black, 1/gamma);
+        double gain = pow(dif, gamma);
+        double lift = pow(black, 1/gamma) / dif;
+
+        GLSLF("color.rgb = %f * pow(color.rgb + %f, vec3(%f));\n",
+                gain, lift, gamma);
+        }
         break;
     case MP_CSP_TRC_GAMMA18:
         GLSL(color.rgb = pow(color.rgb, vec3(1.8));)

The idea is to implement the “full” BT.1886 spec, by simulating a device with a real black point (one which the user could possible tune directly in the further). That way we basically get our own “black point compensation” into mpv.

Note that you can get a similar result by playing with the “brightness” and “contrast” controls in mpv, i.e. making the overall image slightly darker or brighter. That method actually has the benefit of preserving image values that were technically below the minimum, i.e. “super-blacks”, if your source has any.

[haasn]

A good test-pattern for testing both black and white clipping at the same time is https://github.com/haasn/cms/raw/8286d4ed3b4a00e5a44e086e931bd2eae669921c/MP4-2c/Basic%20Settings/2-APL%20Clipping.mp4

[4ad]

@haasn

Does this patch improve things?

It certainly changes things, but doesn't improve them. With iMac color profile and measured native values.

Program	black#1	black#7	#black8	#black15	#white15
QuickTime	1-1-1	7-7-7	8-8-8	16-16-16	241-241-241
mpv (recent)	0-0-0	3-4-3	4-5-4	15-16-14	241-240-242
mpv (patch)	2-2-1	8-9-7	10-10-9	21-22-20	241-240-242
mpv (patch + black=1/5000)	1-1-1	7-7-6	8-8-7	19-20-18	241-240-241

I also changed the black level to various values between 1/700 to 1/15000, the best for black#7 and black#8 seems to be around 1/5000, but #black15 is way off.

[haasn]

Well, the idea behind this change is only to raise the black levels overall in order to prevent black crush, it's not to make the transfer curve exactly match QuickTimes.

As far as we can tell, mpv's transfer curve for this profile was fine in theory, but the black crush caused problems in practice.

This patch fixes those practical problems.

[4ad]

But with simple color profiles, QuickTime and mpv agree, and we agree that it's producing correct output.

This patch not only changes output with complex profile, it changes output with simple profiles too. So if they were right before, now with this patch it means that they must be wrong.

Also, while this patch changes black levels, I don't think the changes are correct. I compare mpv and QuickTime on sRGB-like monitor, with simple profile. I'm watching a real movie, shadow-detail seems good, and consistent between mpv and QuickTime.

Now I watch the same movie on the iMac. Shadow detail is missing (quite obviously I must add, people unaware of this bug here have complained to me about the shadows missing!). When I add your patch (with blacks set to 1/5000), something changes, however, it doesn't look like the output on the sRGB monitor, while the QuickTime output does look the same as mpv and QuickTime on the sRGB monitor.

Also if the problem is related to black-point compensation, surely that must be addressed in the final transformation from the intermediary color space (now BT.709) to the target display space, and not in the intermediary space, no?

[haasn]

This patch not only changes output with complex profile, it changes output with simple profiles too. So if they were right before, now with this patch it means that they must be wrong.

The idea is that the user would tune these parameters to suit their taste, rather than having some hard-coded default for everybody.

On that note, I think this patch is a bad idea in general because it's redundant with the existing settings for “brightness” and “contrast”.

Instead, we should use those to fix issues like black crush, since that's what they're for. I have started using them myself.

[haasn]

Also if the problem is related to black-point compensation, surely that must be addressed in the final transformation from the intermediary color space (now BT.709) to the target display space, and not in the intermediary space, no?

I'm not sure why this would be the case. The problem is that the final conversion crushes blacks, so why can't we fix it by raising blacks before passing it through the final conversion?

[4ad]

@UliZappe

The only thing I can think of is to replace Little CMS with ColorSync for the Mac platform, which would be technically possible, but of course a huge platform specific deviation.

Well it would be worth trying, at least as an experiment. Didn't you already do this for issue 534 though?

[4ad]

Btw, I think our issue is a duplicate of #1563. Even the pictures match!

The idea is that the user would tune these parameters to suit their taste, rather than having some hard-coded default for everybody.

Surely the default is that mpv should produce metrologically correct output, no? It already does for simple profiles, don't we agree that it should also do it for complex profiles? Don't we agree that the final, sRGB readings should be the same with the simple (correct) profiles and the complex profiles?

What we know now seems to indicate a bug in LittleCMS. Sure, my "manual" patch reproduces LittleCMS output, but I don't think that's evidence for correct behavior. More like, it's evidence for too simple behavior. After all, LittleCMS behaves differently with simple and complex profiles.

The problem is that the final conversion crushes blacks, so why can't we fix it by raising blacks before passing it through the final conversion?

Since we don't know exactly the deviation induced by the final stage, we can't implement an inverse "deviation" in the intermediary stage (plus it would add another source of noise).

In fact, we see this. We can "fix" black#7 and black#8, but we distort black#15 a lot. Again, this a lot is not a subjective measure, we do measure it compared to the simple profile case which we agree it works correctly.

[haasn]

Surely the default is that mpv should produce metrologically correct output, no?

I thought it does? Didn't we establish that mpv matches the theoretically computed values?!

[haasn]

Also, this is not a duplicate of #1563 because that one was mostly user error, though in part caused by mpv's lack of flexibility at the time.

Since mpv now has options that correspond to what that user wanted (but wasn't using), that issue was resolved.

[4ad]

I thought it does? Didn't we establish that mpv matches the theoretically computed values?!

No, we established that mpv produces the same output with a particular icc profile, and when using this formula I came up with. I haven't found it written anywhere, in any document, I just came up with it:

f[x_] := (0.948*x+0.052)^2.4 /; x>=0.039
f[x_] := 0.077*x /; x<0.039

Those numbers, indeed come from the profile, but the color primaries do not. The color primaries in the profile do not match the ones I used in my patch!

I found lots of values on the Internet for the color primaries, which didn't agree with each other, and lots of values for the white point. I tried all the combinations until I found one that matches the output with the ICC profile.

We don't know if the formula is good.

We don't know with certainty if the primaries are good.

We don't know with certainty the white point.

That's why I want to get a copy of the standard, which is apparently impossible to get.

This formula also doesn't do any black-point compensation, which we found important. There's also no intent setting in the formula, which surely is important too.

Even if the formula is good, we don't know that the "current practice" expects an approximate version of the curve, like it does with BT.709.

I think the patch was an interesting experiment that might, or might not help diagnose the LittleCMS problem, but I don't think it indicates anything more than that.

[4ad]

I have found this document, Apple makes everything so hard to find: https://developer.apple.com/library/mac/releasenotes/MacOSX/WhatsNewInOSX/Articles/MacOSX10_11_2.html

The Display P3 color space uses the standard DCI-P3 primaries,
a D65 white point, and the same gamma curve as the sRGB
IEC61966-2.1 color space. It is a canonicalization of the
new iMac display profiles that is useful for tagging wide
color gamut content during export.

The curve is indeed extremely similar to the sRGB curve, but curiously, unlike Apple's claim, it's not exactly the same. It is however, close enough for all practical purposes.

https://www.wolframcloud.com/objects/db8fc435-1eb2-46fc-a4c6-9092a2e28623

[UliZappe]

@haasn

Why are you listing mpv under pr and nowhere else?

The p, r and a rows in the table only refer to the calculated results (columns 5 and 6). In contrast, QuickTime Player and mpv always appear in the first row only, and are always used with their default settings (QuickTime Player does not have any options, anyway). Sorry for the confusion, I did not quickly find a better way to present all results in a simple table.

mpv's default intent is relative colorimetric, and you can choose your own.

I’m well aware of that, but I had to limit the test variations somehow, so I only tested the default behavior of QuickTime Player and mpv.

Since I think the test clearly shows that mpv closely reproduces the (relative colorimetric) results of Little CMS and does not seem to add big distortions of its own (IMHO one important test result), you can (roughly) infer mpv’s behavior with other settings from the Little CMS results. At least that’s a preliminary result; of course, we could look in more detail how mpv behaves with specific settings if this became important.

Regarding perceptual vs colorimetric, I use a LUT-based profile (created by ArgyllCMS) and I see no difference whatsoever between the two intents.

That completely depends on the specific profile, and as you can see from this tiny sample of two profiles, the differences in behavior are huge here. Only LUT profiles can have different intents (matrix profiles only have one matrix which they always use for perceptual, colorimetric and saturation alike), but that does not mean that these intents necessarily differ (although admittedly, I would expect this under normal circumstances).

[4ad]

@UliZappe

Can you file a bug with LittleCMS that describes everything that we learned so far? I'd do it but I don't think I am competent enough to describe this accurately.

Maybe the LittleCMS developers are aware of the problem, or maybe they have some insight that we lack.

[UliZappe]

A general comment on the latest discussion:

I would strongly suggest not trying to somehow „improve“ on the regular color management workflow by some „clever“ additions. Doing so is usually a recipe for disaster, i.e. it optimizes the output for some specific situations, but makes it much worse for others.

We face 2 issues that are completely separate.

1. The color deviations that were introduced with the BT.2020 intermediated color space.
2. The problems with the resolution of the dark color patches.

As for 1), this did already work correctly and can be fixed by going back to an approach without an intermediate color space. This should definitely be done and is a clean solution and straightforward in principle.

As for 2), my tests from last night strongly suggest that mpv simply behaves like Little CMS does, so for now let’s say we “know” this. If so, there’s hardly anything mpv can do. The only reliable, non-tinkering solutions would be: a) reporting this as a bug to the Little CMS author and hope that he acknowledges it as a bug and fixes it b) replacing Little CMS by another CMM, ColorSync on OS X and IDontKnowWhat on other systems.

Sadly, I don’t see any other solution here. Sure, user controls might help pragmatically, but using these is completely against the very idea of color management.

[haasn]

This should definitely be done and is a clean solution and straightforward in principle.

This is not as obvious as you make it seem, since it requires generating multiple 3dluts per profile - one for every color space, dynamically, ad-hoc, at runtime and also requires extensive changes to the way the lcms code is plugged into vo_opengl.

[haasn]

Also, I still have no idea why you think LittleCMS is bugged. What evidence are we basing this assumption on again, exactly? I lost track.

(And keep in mind that for me, “QuickTime does it differently” is not evidence)

[4ad]

What evidence are we basing this assumption on again, exactly? I lost track.

mpv should produce the same sRGB output for the same BT.709 input, regardless of what the ICC color profile is.

When using simple color profiles, this is true. The output also matches QuickTime.

When using complex color profiles, the result is different from the case of the simple profiles, although it really shouldn't. Also QuickTime indeed measures the same with both simple and complex profiles, only mpv (LittleCMS) changes.

Apart from that, since we're doing color management, the colorimetric output on different screens, should be the same, if they use the correct profile.

Now, on my Macbook screen with a simple, common, sRGB gamut, with both mpv and QuickTime I see the same thing. Also, shadows look fine, I can see them.

On my iMac screen, QuickTime looks exactly the same as mpv and QuickTime on the Macbook screen. Shadows are visible and colors look the same, color management is working.

However, on the iMac screen, I don't see the shadows on the screen. This leads me to believe that the LittleCMS+complex profile is not correct.

To summarize in a table:

Device	QuickTime	mpv
Macbook	shadows=yes	shadows=yes
iMac	shadows=yes	shadows=no

[UliZappe]

This is not as obvious as you make it seem, since it requires generating multiple 3dluts per profile - one for every color space, dynamically, ad-hoc, at runtime and also requires extensive changes to the way the lcms code is plugged into vo_opengl.

I don’t dispute that the implementation might be expensive – that’s why I wrote “in principle”. ;–)

Still, it’s straightforward in that it’s the only real solution of the problem. Everything else would be workarounds which would probably fail sooner or later or at least deliver less than optimal results. The only thing I could envision worth trying would be to use one of the established PCSs (XYZ or Lab) as the intermediate color space, but even in the best case, this would be an inferior solution, just like two distinct ICC profiles are inferior to a device link profile. And since video means applying the same conversion again and again, I think that a 3DLUT that encloses the complete conversion from the original video colors to the monitor colors is the way to go.

If we deal with something as complicated as color management and we had a process that worked, I do think it is obvious to go back to this process.

Also, I still have no idea why you think LittleCMS is bugged. What evidence are we basing this assumption on again, exactly? I lost track.

1. We know that ColorSync and LittleCMS produce different results for the same color conversion.
2. We also know that this happens with matrix profiles which (contrary to LUT profiles) should be unambiguous, i.e. there should be no “room for interpretation” for a CMM; the result should always be the same. So either ColorSync or Little CMS is wrong.
3. It is true that this alone does not say which of the two results we get is the correct one. But the fact that ColorSync produces visually differentiated results and Little CMS does not argues that ColorSync is the correct one.

[haasn]

If we deal with something as complicated as color management and we had a process that worked, I do think it is obvious to go back to this process.

It didn't

We know that ColorSync and LittleCMS produce different results for the same color conversion.

Which says nothing

We also know that this happens with matrix profiles which (contrary to LUT profiles) should be unambiguous, i.e. there should be no “room for interpretation” for a CMM; the result should always be the same. So either ColorSync or Little CMS is wrong.

This is hardly the case. There's always room for implementation details, especially with some of the definitions floating around like for “perceptual” intent which is defined as “should look good”.

It is true that this alone does not say which of the two results we get is the correct one. But the fact that ColorSync produces visually differentiated results and Little CMS does not argues that ColorSync is the correct one.

This alone is not a compelling enough reason.

[4ad]

This alone is not a compelling enough reason.

Do you agree that we should see (measure non-zero output) for bars 17-25 in 1-Black Clipping.mp4 with every color profile? This is true with QuickTime with any profile. This is true with mpv with simple profiles. This is false only with mpv with complex profiles.

This suggest to me that it's QuickTime which is correct, and not LittleCMS.

[haasn]

mpv should produce the same sRGB output for the same BT.709 input, regardless of what the ICC color profile is.

This makes no sense. If it is using an ICC color profile, it is not outputting sRGB. Unless you're talking about your “screen reader” tool's sRGB values, in which case...

When using complex color profiles, the result is different from the case of the simple profiles, although it really shouldn't. Also QuickTime indeed measures the same with both simple and complex profiles, only mpv (LittleCMS) changes.

Are you really that surprised about using Apple software for both directions gives you a round-trip, but using Apple software in one direction and LittleCMS in the other does not?

I am willing to bet that if you rewrite your screen reader tool to use LittleCMS for the screen->sRGB conversion, then mpv would suddenly be the one that matches the values perfectly while QuickTime would not.

As such, this is useless information as far as determining which of the two is more correct goes.

However, on the iMac screen, I don't see the shadows on the screen. This leads me to believe that the LittleCMS+complex profile is not correct.

I see no reason why this should be the case but “Apple's handling of complex profiles is bugged” is not. As far as I can tell, you're again making favorable comparisons here by using profiles generated by Apple tools with Apple tools to prove that it's compatible with itself.

[haasn]

Either way, feel free to believe what you want, and feel free to make as many LittleCMS bug reports as you want.

Either way we look at it, one thing is clear: mpv's handling of TRCs does not seem to be bugged, and I suppose that's the only thing that really matters here.

[4ad]

This makes no sense. If it is using an ICC color profile, it is not outputting sRGB. Unless you're talking about your “screen reader” tool's sRGB values, in which case... [...] Are you really that surprised about using Apple software for both directions gives you a round-trip, but using Apple software in one direction and LittleCMS in the other does not?

Ok, true, however we know, both from reading the ICC profile, and by confirmation from Apple documentation that Display P3.icc uses the sRGB gamma. Which means we can compare the values on screen directly.