Profiles ICC - Output Profile

[Beep6581]

Originally reported on Google Code with ID 1166

This patch improves:
a) the files *.icc *.icm "Output profile"
b) display, if you used a text editor that supports color management (Photoshop CS
...), a new  Tag "desc" for "Output gamma"

a) I changed the development files "output profile" (RT_Largexxx.icc; RT_Middle_gsRGB.icc;
RT_sRGBxxx.icm").
Now they are no longer prepared using a profile editor, but using LCMS. I think profiles
are cleaner!

They have the same useful features during processing (apply / convert) than the previous:
the same primary, same white point, same TRC. For cons, the LUT of TRC are more accurate:
4096 points instead of 1024.

I have slightly modified some Tag of iCC profiles:
* Dmnd = "RawTherapee" (in all cases)
* Dmdd = "RTLarge", "RTMiddle", "RTsRGB" (depending on profile)
* Cprt = "No Copyright RawTherapee" label adapted to the profile, such as "Prophoto
compatible"

Tag "desc" is unchanged, for example "RT_Large Linear1.0 gamma (Prophoto compatible)
The files have a size larger because of the new LUT (25K instead of 1K/8K)

b) When using the "Output gamma" ("drop-down menu", or "Free Gamma") appeared previously,
the file name that was used to convert RGB (which is in the "options" [Color Management].
But this file is modified for Tag "TRC" to fit the profile "apply" to the choices made
in "Output gamma". 

Until now the Tag 'desc' was not changed. Now appears when using an external editor
(Photoshop CS ...) a brief description that can be made ​​according to the choices:
* Output Gamma: sRGB; Output Gamma: low ; and so on.
* Output Gamma: Gamma Free

This change in the Tag 'desc', does not change the output image, but only the text
displayed in the external editor.

Reported by jdesmis on 2011-12-21 08:22:54

---

• Attachment: profICC.patch
• _Attachment: output_profil.zip_

[Beep6581]

What do you think about a new option "Same as internal" at the output profiles list
where no transformation from internal to output color space takes place ?.

Reported by iliasgiarimis on 2011-12-21 09:27:21

[Beep6581]

The match between "preview" and "output profile" is a lottery ... The best chance is
to choose "working profile"=sRGB and "Output Profile"="RT_sRGB" and have the monitor
calibrated with a gamma sRGB...But there will be differences.

Then if we choose: "working profile"=Prophoto, correspondence is the least bad with
 choice "Output Gamma"=Standard_g1.8" ...

But once you make a conversion, many parameters come into play (work ==> destination),
including "intent" (relative perceptual,...), off-color gamut, etc.. and it is impossible
to have the same thing.

The only solution is that the "preview" takes into account the output conversion, as
do such as "Photoshop CS" or "Nikon Capture NX2" :)

Reported by jdesmis on 2011-12-21 10:38:16

[Beep6581]

Hi Jacques! Great to see this patch, I will be able to test it tomorrow only evening.
Just by reading a few suggestions / questions:

1. RT_Middle can be renamed to RT_Medium

2. "The only solution is that the "preview" takes into account the output conversion,
as do such as "Photoshop CS" or "Nikon Capture NX2""

I think this may be the best solution, although possibly impacting performance - have
preview convert from the output profile and after that to the monitor profile - so
there is one more conversion step. What would be the performance penalty? Preview is
usually small (not many pixels) so this is probably not so bad.
From my point of view accuracy is more important than a small performance penalty.

3. Would it be possible to have the Output Gamma reflected in the preview?
I guess it will be so if #2 is implemented.

Reported by michaelezra000 on 2011-12-21 12:21:27

[Beep6581]

No problem to rename RT_Middle ==> RT_Medium. My English is very average, and for me
these two expressions are almost identical. But if the patch tests are correct, this
poses no problem ...

I think that there is no performance impact, to put in "preview" the converted image.
This image comes from "Output Profile" or "Output gamma" is in the end the same.

The "only problem" is that I am a very humble programmer - do not confuse the colorimetry
capabilities and those in C + + - and it seems to me personally difficult. The key
is in "iplab2rgb.cc" and how to transform the 16-bit code in 8-bit code including the
conversion and sending it to the screen the right data.

If we can do this, you will (try if you have Photoshop or CaptureNX2) that the color
conversion (gamma, apply, convert, intent ...) significantly alter the images and histograms.:)

Reported by jdesmis on 2011-12-21 12:51:16

[Beep6581]

Great that this can be done without much performance penalty for the preview, this will
be most accurate. I will look at the code. Probably help from Emil or Olli would be
useful in this area as well.

Btw, as I think about this, could it be that now in RT we have *conversion* (which
is good) to the output profile, but gamma is only *assigned* - and this causes most
of the discrepancy with the preview?

Reported by michaelezra000 on 2011-12-21 14:57:59

[Beep6581]

in fact,in "simpleprocess.cc" (patch), line 377, it was in all cases, data that take
into account the conversion "output profile" with or without "Output gamma".

If we can move this algorithm (from line 249), of course, keeping only what is useful
in 8-bit and adjusting the data type for the screen, it should work.

Perhaps to minimize the real-time calculations, should put a check box "preview" in
"Output profile"

Reported by jdesmis on 2011-12-21 15:34:39

[Beep6581]

Reading the patch, a few minor suggestions for consistency of descriptions:

 "Output Gamma : sRGB"    should be "Output Gamma: sRGB"      <- add a numerical value?
 "Output Gamma : BT709"   should be "Output Gamma: BT709"     <- add a numerical value?
 "Output Gamma: Linear10" should be "Output Gamma: Linear 1.0"
 "Output Gamma : 2.2"     should be "Output Gamma: 2.2"
 "Output Gamma : 1.8"     should be "Output Gamma: 1.8"

Reported by michaelezra000 on 2011-12-21 16:49:25

[Beep6581]

Also, could probably skip word "Output" in the Gamma description, as when RT's output
image is opened in image editor, this is just Image's Gamma Encoding and word "Output"
refers to RT's perspective only. 

Reported by michaelezra000 on 2011-12-21 16:53:29

[Beep6581]

To address the issue that 
 Condition: when Output profile = "No ICM: sRGB output", 
            working space = ProPhoto
 Problem:   the output image is rendered LIGHTER than it should be.

The problem must lie in this area
in lab2rgb16 see lines 297-301

    xyz2srgb(x_,y_,z_,R,G,B);

    image->r[i-cy][j-cx] = (int)gamma2curve[CLIP(R)];
    image->g[i-cy][j-cx] = (int)gamma2curve[CLIP(G)];
    image->b[i-cy][j-cx] = (int)gamma2curve[CLIP(B)];

improcfun.cc 
void ImProcFunctions::xyz2srgb (float x, float y, float z, float &r, float &g, float
&b) {
    r = ((sRGB_xyz[0][0]*x + sRGB_xyz[0][1]*y + sRGB_xyz[0][2]*z)) ;
    g = ((sRGB_xyz[1][0]*x + sRGB_xyz[1][1]*y + sRGB_xyz[1][2]*z)) ;
    b = ((sRGB_xyz[2][0]*x + sRGB_xyz[2][1]*y + sRGB_xyz[2][2]*z)) ;        
}

icc.matrices.h

const double sRGB_xyz[3][3] = 
{{3.1338561, -1.6168667, -0.4906146},
{-0.9787684,  1.9161415,  0.0334540},
{0.0719453, -0.2289914,  1.4052427}};

So the question is: 
is the above sRGB_xyz matrix suitable for new "RT_sRGB_gBT709" output profile?

Also, it is a bit confusing why xyz2srgb is using matrix sRGB_xyz?
Should it not be using xyz_sRGB?

What is the meaning of the name of the matrix in icc.matrices.h: TO_FROM or FROM_TO?

Reported by michaelezra000 on 2011-12-22 15:53:59

[Beep6581]

I still don't understand why it should be a "lottery" to match conversion output to
preview image.  This is what color management is all about.  The preview image is produced
from converting the internal lab representation using the monitor profile.  The output
image is transformed from Lab to any of the output spaces.  The output space should
be embedded into the output file so that they can be displayed by any color aware program/device.
That includes the gamma of the output space.  Am I missing something fundamental here?

Reported by ejm.60657 on 2011-12-22 16:14:12

[Beep6581]

I am a bit new to this details, but found this info which may be helpful: http://www.color.org/sRGB.pdf

Could you please review section "2) Measurement ‘correction’" and formulas at the very
end. Do they need to apply in this case?

Reported by michaelezra000 on 2011-12-22 16:17:02

[Beep6581]

There is one more thing to review: gamma2curve which is always used after xyz2srgb.

gamma2curve is filled in improcfun.cc line 100:
for (int i=0; i<maxindex; i++) {
    gamma2curve[i] = (CurveFactory::gamma2(i/65535.0) * 65535.0);
}

and gama2 is set in curves.h line 157:

// standard srgb gamma and its inverse
static inline double gamma2 (double x) {

       return x <= 0.00304 ? x*12.92 : 1.055*exp(log(x)/sRGBGammaCurve)-0.055;

}

sRGBGammaCurve value is set in in curves.cc
    // Wikipedia sRGB: Unlike most other RGB color spaces, the sRGB gamma cannot be
expressed as a single numerical value.
    // The overall gamma is approximately 2.2, consisting of a linear (gamma 1.0) section
near black, and a non-linear section elsewhere involving a 2.4 exponent 
    // and a gamma (slope of log output versus log input) changing from 1.0 through
about 2.3.

    const double CurveFactory::sRGBGamma = 2.2;
    const double CurveFactory::sRGBGammaCurve = 2.4;

Is all above correct?

Reported by michaelezra000 on 2011-12-22 16:41:59

[Beep6581]

There is obviously an issue with gamma assignment in case of "No ICM: sRGB output",
because when I open the resulting output image in Photoshop and *assign* sRGB profile,
then it becomes darker and looks correct.

Reported by michaelezra000 on 2011-12-22 16:52:02

[Beep6581]

Emil, what you say is true ... but in precisely the TIF there is embedded ICC profile.
Several cases can arise:
1) "working space" = "Output profile". Everything is fine in theory because the sRGB
gamma for internal calculations is the same as the output (gamma sRGB). 
But as I pointed out if the gamma calibration profile of the screen is different we
will have a different image ... 
It is clear, written in the user manual of the product (for me DispalGUI)

2) Always "Working profile" = "Output profile", but in other cases (Prophoto, Adobe,...)
gamma ICC profile is different or that in calculations. 
This leads to differences but are low because the color management partially offset.
These differences are more or less important according to several criteria including
the gamma (RT_Large_BT709) uses a gamma=2.2 s=4.5 instead of Prophoto = 1.8, 
but can be offset by "Output standard1.8 gamma =", but also the intention, and of course
the monitor ...

3 whether "working profile" different from "Output Profile", there are important differences.
This will depend on where you made the conversion ?
If this is done with RT, the conversion engine is LCMS2, if it is done in Photoshop,
the conversion engine is either "Adobe ACE" or "Microsoft MCI" 
that give very different results. Just try and compare ....

Moreover, if there is color "out of gamut"
1) it will not appear on the screen (except the happy owners of Widegamut screen)
2) the conversion of space, as the conversion engine and intent, processes these colors
so differentiated
You can compare several things:
a) histograms that display the RGB values and not TRC
b) the TRC tone curves that are embedded in the file header
With the same values RGB , you will have with Tag TRC rendering different.

Do you have Photoshop CS or better Nikon Capture NX2? The latter is close in spirit
to RT, because it is the demosaicing, and the work of post-processing in the same module.
Nikon why he expected the "ICC Input" and "ICC Output" with the visualization of result
??? ... It is obvious, and just try! .. 
The histograms (and images too) change according to the conversion, and intent.

There is no bug or malfunction in there, of course there may be bugs in the colorimetric
RT, and it is up to us to correct them, but the fact that the image changes according
to profiles output is also clear that the passage of time ...

And what will there be if one day RT has a printing module?

Reported by jdesmis on 2011-12-22 17:01:42

[Beep6581]

Michael, look at the code "iplab2rgb.cc" and "simpleprocess.cc", you will see that the
gamma and the TRC are always calculated and "accorded"

ICC files that I have provided is :
*  sRGB gamma (g = 2.4 s = 12.92), which amounts to an overall average gamma of 2.2,
but with a linear part that opens the shadows
* BT709 is different, it gives more contrast ...

Try the options menu "Output gamma" or better with "Free Gamma" ...

And why did Adobe choose for Lightroom, an internal profile called Melissa, which is
nothing but RT_Large_gsRGB. That is, a conversion matrix Prophoto RGB, and gamma  sRGB(2.2
12.92) because it provides an optimal ...

Reported by jdesmis on 2011-12-22 17:13:19

[Beep6581]

Tomorrow, I will update the patch with the changes requested

Reported by jdesmis on 2011-12-22 17:54:11

[Beep6581]

Jacques, about ACR/Lightroom here is info from author Eric Chan:
http://www.luminous-landscape.com/forum/index.php?topic=60179.msg485409#msg485409

"The internal working space of ACR and LR is Referred Input Medium Metric (RIMM), which
has ProPhoto RGB primaries with linear gamma.  Temporary excursions are made to other
color spaces and image decompositions, as needed, to suit the image processing."

One more test:
A. Working space:  ProPhoto
   Output profile: Prophoto

B. Working space:  ProPhoto
   Output Gamma:   standard_g1.8

C. Working space:  ProPhoto
   Output profile: linear_g1.0

D. Working space:  ProPhoto
   Output profile: any other value

results: A = B = C != D

Another thought based on this post: (http://www.luminous-landscape.com/forum/index.php?topic=58102.msg469653#msg469653)
"But aside from quantization error, the choice of gamma encoding (1.8, 2.2, sRGB, 1.0,
etc.) has no impact on the color reproduction capabilities.  In fact, conceptually
the first thing that happens in an ICC-based workflow is the transformation from the
RGB encoding space back to the Profile Connection Space (a.k.a., PCS, usually D50-relative
XYZ or CIE L*a*b*) before conversion/mapping to the desired output space.  So the encoding
is basically irrelevant as long as you can get safely back to the PCS."

My question is - why do we do this conversion:
Lab->XYZ->RGB->output space
instead of Lab->output space?

Reported by michaelezra000 on 2011-12-22 18:00:06

[Beep6581]

small correction in C. and D. above I meant "Output Gamma", just like in B.

Reported by michaelezra000 on 2011-12-22 18:02:06

[Beep6581]

Do not mix everything ... The demosaicing is done in linear (obvious), but then using
Lightroom "Melissa" is the name of the person who made the development. To simplify
management, there is a choice (if output can not be changed as RT) and gamma sRGB conversion
Prophoto with a linear part up to r = 12.92.

It is normal for A = B because it is the same (with hints that I introduced for copyright).
As against C is different, look at the histograms ...

To convert from RGB to Lab must necessarily pass through XYZ (except when using LCMS,
but the fact that in-house).

In RT converting RGB Output is not in the preview, which I think is unfortunate ...
but it would solve all the problems and would stop the discussions:)

Reported by jdesmis on 2011-12-22 18:28:42

[Beep6581]

When opening A,B,C,D in photoshop I confirm that A !=C; Strangely, Bridge CS5 shows
them the same...

About renaming the embedded profile descriptions for custom Output Gamma - should the
primaries be mentioned as well?

Reported by michaelezra000 on 2011-12-22 19:13:56

[Beep6581]

Here the new patch with the changes requested.
And the new profile "RT_Medium" instead of "RT_Middle" (just the name, and Tag are
changed)

Reported by jdesmis on 2011-12-23 07:16:19

---

• Attachment: profilICC2.patch
• _Attachment: RT_Medium_gsRGB.icc_

[Beep6581]

Thanks Jacques, I tested the patch.
When using Output Profiles the output files are tagged like this:

   RT_Large gamma BT709(Prophoto compatible).tif
   RT_Large gamma Linear1.0(Prophoto compatible)
   RT_Large gamma sRGB(Prophoto compatible)
   RT_Medium gamma sRGB(AdobeRGB compatible)
   RT_sRGB gamma BT709(IEC61966 equivalent)
   RT_sRGB gamma BT709(IEC61966 equivalent) <-for NO-ICM
   RT_sRGB gamma Linear1.0(IEC61966 equivalent)
   RT_sRGB gamma sRGB(IEC61966 equivalent)

Is there a difference between "compatible" and "equivalent"?

There is a problem in tagging when using Output Gamma.
In various combinations of Working space and Output Gamma Bridge CS5 shows that profile
tag is absent (try BruceRGB for example).

for stri[ns] - how is the value of ns being passed from GUI?

Reported by michaelezra000 on 2011-12-23 14:50:59

[Beep6581]

All Tag you mention are good !

I tried to visualize the Tag with "BruceRGB" and "output gamma" in two software:
* Photoshop CS5: displays "GammaTRC: BT709 G2.2 S4.5 BruceRGB"
* PhotoME (which is the reference software to read all the tag of a file (TIF, JPG,
RAW ...)), 
he can see for example in case (BruceRGB + "Output Gamma '): 160 tag for the photo
(TIF treated with RT) (objectif, white balance,  exposure time, etc..) and 28 for the
ICC profile tag, 
the tag "desc" is taken "GammaTRC: BT709 G2.2 s4.5 BruceRGB".

for another profil ICC  type "RT", to another file, always "Output Gamma"
* Photoshop CS5 provides "gamma RT_sRGB BT709 (IEC61966 equivalent)"

* PhotoME:
        "RT_sRGB gamma BT709 (IEC61966 equivalent)" (description)
        "RawTherapee" (Device Manufacture)
        "RTsRGB" (device Model)
        "No copyrightRawtherapee - IEC61966 equivalent: (copyright)"
It also contains the "good" curves for TRC, good values for white point and primary
...
These are the values that I have written in the ICC profile     

If I look at the ICC profiles with "ICC Profile Inspector", no problem is reported,
all Tag have datan(correct) 

I put "equivalent" to RT_SRGB and "compatible" for others on a suggestion DrSlony.  

The parameter "ns" comes from "simpleprocess" (patch) lines 256-264

Reported by jdesmis on 2011-12-23 16:27:10

[Beep6581]

Hi Jacques, yes the tags which do get written are good:)

I tried to reproduce your results for Output Gamma, but in many cases profile & description
does not seem to get to the output... Here is an example of output TIFF-16 http://min.us/mZErba9yv
Photo Me and Bridge CS5 do not show ICC info.

Reported by michaelezra000 on 2011-12-24 02:59:21

[Beep6581]

I am curious, when working space = "ProPhoto" in GUI is it using the 'true' ProPhoto
or RT_Large?

There is a strange behavior I observe with my new patch (issue 1152) - 
there is *less* clipping shown in preview in working space=sRGB vs working space=ProPhoto

Reported by michaelezra000 on 2011-12-24 03:29:01

[Beep6581]

Hello Michael, TIF file that you have joined (D700_F5.6-1.tif) does not contain any
ICC profile.
 As "Output gamma" make a "patch" the existing profile, it is normal that the file
contains no Tag ICC

"Working Profile" does not use the good "Prophoto", nor "RT_Large", it uses a matrix
in "Rawimagesource" (iccmatrice.h). 
All the "working space" are matrix ; the "output profile" are ICC profiles

It is normal that there are differences in the "preview" as the "working space".
If the image contains values "out of gamut" for the smallest space, these data will
be processed differently in RT, according to the "intent" used. 
In the end comes up different data. If this screen is a profile "narrow" (such as sRGB),
the more the original images will be "out of gamut", the more the differences displayed
in the "preview" between "working space" and sRGB "working space" Prophoto will be
great.

If you have no further comments questioning the validity of the new ICC profile, I
will commit to default, to morrow (for me in France)

Reported by jdesmis on 2011-12-24 06:28:34

[Beep6581]

Hi Jacques, Happy holidays to you and your family as well!

image below shows the reason of my question. The output image does not have ICC info
only in some cases, not in all cases:

About differences in working spaces sRGB vs ProPhoto: 
ProPhoto should be larger, so when working space=ProPhoto there should be *less* clipping.
Am I missing something?

Reported by michaelezra000 on 2011-12-24 13:45:23

---

• _Attachment: RT_ICM_01.jpg_
• _Attachment: sRGBvsProPhoto_01.jpg_

[Beep6581]

P.S. in my RT preferences intent = 'Relative Colorimetric', switching to Perceptual
does not change illustration.

Reported by michaelezra000 on 2011-12-24 13:48:57

[Beep6581]

Have you installed on your computer Bruce.icm (windows/system32/spool/drivers/color)
otherwise it will not work.

For color spaces, the reverse certainly Prophoto is greater, but the screen is in sRGB
so it clips. If you choose sRGB, values ​​beyond sRGB are better addressed to the display
(sRGB), but less so if your screen ("deLuxe") is WideGamut, or if you print to a printer
upscale.

Reported by jdesmis on 2011-12-24 14:16:07

[Beep6581]

bummer! I dont think I have Bruce.icm:)!!! Sorry for confusion:)

I do have wide gamut monitor - NEC LCD2690Wxi 93% AdobeRGB, but clipping indicators
are independent of the monitor/monitor profile, they show clipping in the working space
( see rtgui/cropwindow.cc line 710).

Reported by michaelezra000 on 2011-12-24 14:27:54

[Beep6581]

Regarding this issue & patch: 
all profile descriptions seem very good.

I have one more question: 
When using Free Gamma or Output Gamma there is substitution of Working space name in
the profile description:
sRGB       ->RT_sRGB
Adobe RGB  ->RT_Medium
ProPhoto   ->RT_Large

Should this substitution be carried?

Reported by michaelezra000 on 2011-12-24 15:04:16

[Beep6581]

The substitution is made in "options" [Color Management] ... 

In fact I have made(6 months ago now) the substitution in place, edit with "options"
because I did not know what profiles (ICC / ICM ) and with what names were/were not
installed in each operating system (Windows / Mac / Linux ...). This allows everyone
to change if desired profiles "RT_" by other

I will prepare Christmas dinner now, in France is in a few hours ...

Happy Christmas to you

Jacques

Reported by jdesmis on 2011-12-24 15:19:02

[Beep6581]

Happy Christmas to you!
Enjoy the dinner:)
Michael

Reported by michaelezra000 on 2011-12-24 15:21:19

[Beep6581]

commit to default

Reported by jdesmis on 2011-12-25 08:06:55

[Beep6581]

Can this be closed as "fixed"?

Reported by entertheyoni on 2012-04-23 21:54:03

[Beep6581]

yes you can :)

Reported by jdesmis on 2012-04-24 05:32:11

[Beep6581]

Thank you

Reported by entertheyoni on 2012-04-24 10:04:30

• Status changed: Fixed