JPEG output is suboptimal

[Beep6581]

Originally reported on Google Code with ID 2753

Well, in fact, it really sucks!

Take a look at the dark areas of the background on the center of the image, and compare
it with IrfanView's output. All image were saved with a slider set to "90%" for quite
good image quality. The IrfanView version has been saved first to Tiff/8 bits by RT,
then converted by IrfanView with 90% quality too.

What can we see:
1. all images out of RT has strong posterization.
2. the IrfanView version is the smallest one, even compared to the "Best compression"
version of RT's one, and for the same compression ratio.

Tagged as critical.

Output files:
-------------
http://www.rawtherapee.com/shared/test_images/BlackCat%20output/BlackCat-A.Best%20quality.jpg
http://www.rawtherapee.com/shared/test_images/BlackCat%20output/BlackCat-B.Eqilibrated.jpg
http://www.rawtherapee.com/shared/test_images/BlackCat%20output/BlackCat-C.Best%20compression.jpg
http://www.rawtherapee.com/shared/test_images/BlackCat%20output/BlackCat-IrfanViewV4.32.jpg

Test image:
-----------
http://www.rawtherapee.com/shared/test_images/BlackCat.DNG
http://www.rawtherapee.com/shared/test_images/BlackCat.DNG.pp3

Reported by natureh.510 on 2015-04-26 11:43:40

[Beep6581]

Slight correction: The IrfanView's version has stripped EXIF data, so it's a bit lighter
(enabling "Keep EXIF data" does not change anything). RT lacks an option to strip EXIF
data of the jpeg output, which may be what users wants to have the smallest files possible
for the web.

But still, RT's jpeg output is dirty.

Reported by natureh.510 on 2015-04-26 11:54:43

[Beep6581]

Added the Gimp2.8 version for comparison, with Optimize checked, 4:2:2 Horizontal subsampling
and Integer DCT.

http://www.rawtherapee.com/shared/test_images/BlackCat%20output/BlackCat-Gimp2.8-Optimize+4-2-2H.jpg

Files size: 275kb

Reported by natureh.510 on 2015-04-26 12:12:51

[Beep6581]

Now that's interesting! Converting from RT's Tiff/8 bits to jpeg with Gimp 2.8 lead
to a differently toned image (darks are different)!? Any clue on what's happening?
Testing with uncompressed checked/unchecked, with or w/o RT_sRGB output color profile
doesn't change anything.

Reported by natureh.510 on 2015-04-26 12:21:25

[Beep6581]

I cannot see anything significant past JPEG squares.

Visibility of JPEG artifacts can be dependent on brightness so I suggest that you get
exact tone matching before comparison.

>The IrfanView version has been saved first to Tiff/8 bits by RT, then converted by
IrfanView with 90% quality too.

I do not understand it. My clue is that following should be done:
1) export TIF/8 from Irfan;
2) export JPEG from Irfan;
3) import TIF into RT and export JPEG.

Or in other way: export TIF from RT and import it to Irfan.

And those two JPEGS are to be compared.

Reported by pinhuer on 2015-04-26 12:41:33

[Beep6581]

>Added the Gimp2.8 version for comparison, with Optimize checked, 4:2:2 Horizontal subsampling
and Integer DCT.

This one looks better than images from OP.

Reported by pinhuer on 2015-04-26 13:49:38

[Beep6581]

I see the compression artifacts in the form of posterization and blockiness but I don't
see any problem. They are the same regardless of software used. They are visible only
in the darkest tones, and the IrfanView image has fewer of the darkest tones so they
are less visible there.

Using your PP3 I changed resizing to 1920x1080 and saved a copy to a 16-bit TIFF and
another copy to JPG using "Balanced" subsampling, which is 4:2:2 which is also called
2x1. I converted the TIFF to JPG using the same compression settings in GIMP-2.9 (just
now's git checkout, b42e764) using floating-point DCT and with enabled "optimized"
and "progressive".
You can get a nightly build of GIMP-dev (2.9) for Windows from http://nightly.darkrefraction.com/gimp/

I also converted using Image Magick-6.9.0.3:
convert BlackCat_RT.tif -quality 92 -sampling-factor 2x1 BlackCat_IM_92_422.jpg

Result: identical artifacts in all.
http://filebin.net/to3odukzb0

Looks like a false alarm to me. But when talking about black cats, one can't be too
careful.

Reported by entertheyoni on 2015-04-26 14:58:45

[Beep6581]

Hmm .. it's a difficult sample which needs investigation from experts .. Hombre/Drslony
can you send it to libjpeg guys ?

Or maybe find a quantization table which is better adapted for this kind of data ..
(we need jpeg compression experts for this ..) 

I tried exporting with pure gamma 2.22 and the posterization is A LOT less. Even AdobeRGB
(which has a pure 2.2 gamma) gives less posterization despite it's larger color space.
This tells me that there is a need for more codes at the darks .. (pure gamma steals
codes from the mids and gives them to the darks ..). 

A workaround solution I am thinking about is to use a tone curve (TRC) adapted to HumanVisualSystem
instead of using the standard sRGB ..
https://www.smpte.org/sites/default/files/23-1615-TS7-2-IProc02-Miller.pdf 
https://www.smpte.org/sites/default/files/2014-05-06-EOTF-Miller-1-2-handout.pdf  the
papers gives the calculation ready .. (I think)
If we could replace the 4096 points LUT-TRC that exists in RTXXX.icc's with a PQ LUT
we've done it ..

Reported by iliasgiarimis on 2015-04-26 20:42:12

[Beep6581]

"I tried exporting with pure gamma 2.22 and the posterization is A LOT less."
How?

Perhaps your monitor's profile is to blame? Do you still see it when you turn color
management in your image viewer off?

Reported by entertheyoni on 2015-04-26 21:12:33

[Beep6581]

Re #6:

They are the same? Really!?? Then try again in the dark and a calibrated monitor ;).
The RT's one has the biggest flat black area (on the left of the cat, but see also
the top right corner), the other ones has posterization as well, but quite less. IrafanView's
image, on its side, doesn't produce any block in the very dark areas!

Re #4:

The process to get the Irfan View image was:
1. load image in RT
2. generate a tiff output, 8 bit, with the supplied pp3
3. load the tiff file in IrfanView
4. generate the jpeg output, with 90% compression

The process to get the RT's image was:
1. load image in RT
2. generate a jpeg output, 8 bit, with the supplied pp3, and 90% compression as well

There would be no issue if I've used a very different compression ratio, but they are
the same, and the subsampling may be the same as well, so there is an issue, and a
big one: we don't have a floating point engine to get such flat dark areas, sorry (and
it seem to be in the dark area only).

Re #8:

I don't see why the monitor profile should play a role here: RT's tiff output and it's
conversion to jpeg with IrfanView produce an identical image, so producing a jpeg file
directly from RT should generate the same result than IrfanView, theoretically. May
there's an option of libjpeg that we haven't set?

Reported by natureh.510 on 2015-04-27 00:00:44

[Beep6581]

If you want to see things better, just load the images in IrfanView, RT or any basic
viewer, and increase luminosity and contrast.

Reported by natureh.510 on 2015-04-27 00:05:45

[Beep6581]

Re #7: I won't send it to libjpeg experts, since I don't know anything about jpeg compression
and its arcane. So I would be glad if DrSlony or you could do that, please.

Reported by natureh.510 on 2015-04-27 00:08:05

[Beep6581]

Please try with the attached icc as output profile instead of RT_sRGB.

Sample conversions ..
https://drive.google.com/folderview?id=0B0NqktTgc54sfmNySGVMbE1Uc0RweGJDdnM4LW9YS2lZZmRHTUlleERHRXVmRlUyR1NBbE0&usp=sharing

Reported by iliasgiarimis on 2015-04-27 02:41:26

---

• Attachment: [sRGB Gamma22.icc](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2753/comment-13/sRGB Gamma22.icc)

[Beep6581]

#10: "I don't see why the monitor profile should play a role here"

Unfortunately it does with Irfanview, see this comment here: http://www.dpreview.com/forums/post/53696650
: "Be very careful if you enable IrfanView's color management. If you enable a display
profile and then open/save a file using IrfanView, the display profile will be applied
to the saved image. This is something you absolutely do not want happening."

Maybe this is the reason why the Irfanview file has different brightness?

Reported by stefan.ittner on 2015-04-27 06:24:51

[Beep6581]

#10 but I explained why IrfanView's image has less visible artifacts and it has nothing
to do with better quality compression... IrfanView screwed up the gamma making dark
areas lighter than they should be so don't compare against it. Chroma subsampling also
has nothing to do with it. The monitor profile could play a role when it brightens
dark areas in a way it shouldn't in one of many ways, e.g. using a setting designed
for CRT on an LCD, thereby making the darkest colors brighter and clearly visible.

Reported by entertheyoni on 2015-04-27 07:34:23

[Beep6581]

Inverted, and input levels set with a black point at 245 and white point at 255:
http://i.imgur.com/CWCsPMq.png
http://i.imgur.com/5vr5Ucj.png
http://i.imgur.com/aUGCerM.png
http://i.imgur.com/y6ympkc.png

I see no bug in RawTherapee.

Reported by entertheyoni on 2015-04-27 07:48:24

[Beep6581]

>I see no bug in RawTherapee.

For some reason shadows are clipping somewhat earlier on RT images - one extra stripe
is present on GIMP output.

Reported by pinhuer on 2015-04-27 09:02:37

[Beep6581]

DrSlony, you destroyed the g22 screenshot, it's impossible to compare images with different
gamma after manipulation which acts at the input data, because for example setting
a clipping at 245 in one is different than 245 from the other. These values come from
the input while we are interested in output (after gamma correction - linerization)

I found the best presentation is just an increase of exposure at +7 stops. Samples
will follow later ..

It's not RT's problem but jpeg's compression, same ugly posterization happens with
Photoshop when used on files with RT_sRGB.icc and Adobe's own sRGB_IEC61966-2.1.icc
.. i.e generally with complex gamma curve (linear at the darks, then exponential).

Interesting that jpeg compression works much better at the darks with pure gamma2.2
with both jpeg engines I tried (RT, Photoshop)

Reported by iliasgiarimis on 2015-04-27 12:47:47

[Beep6581]

First of all, which libjpeg? I use libjpeg-turbo-1.4.0 (1.3.1 at the time of testing).
Does Photoshop use libjpeg at all?

As I see it, the posterization/blockiness is normal at the darkest tones, tones which
you are not supposed to be able to differentiate between so easily as for this to be
a problem. The visibility of these artifacts can be amplified by monitor profiles which
lighten shadows for a multitude of reasons such as black point compensation, black
point correction (LCDs at 0 input do not produce 0 output), rendering intents, matrix
vs LUT, etc. JPEG compression makes use of psychovisual redundancy to discard information
which should not be visible without tweaking. It may be that this is not the cause,
but I would start by putting into question the monitor calibration and profile before
I decided to call this a libjpeg* bug. Either way this is not a bug in RawTherapee.

Reported by entertheyoni on 2015-04-27 16:07:43

[Beep6581]

> IrfanView screwed up the gamma making dark areas lighter than they should be so don't
compare against it.

Yes, it looks like Irfan View does lighten dark areas because the Color Management
is active, as well as in Firefox, where I can see the same result. If I disable color
management of IrfanView, I get deeper darks than shown in RT's preview (so it's not
trustworthy), and I get the same untrustworthy result than exporting jpeg from RT w/o
embedded profile and displayed with a color managed IrfanView.

If I had a better jpeg output out of IrfanView, it's certainly because I exported first
in Tiff, w/o embedded color profile(?). Could there be a double conversion in RT? i.e.
it convert the image to RT_sRGB (or whatever has been set) AND embed the color profile?
(just a noob's thought)

For sure, RT's output with "noICM=sRGB" != ( "RT_sRGB", which seems equal to the "system
sRGB", same result ). That's not a little difference, but a strong one in the dark
areas. So I'd like to know where the problem is if it's not in RT? ... just to be able
to use the ICM panel in a predictable way (if possible).

I must add that both RT and IrfanView use the same monitor color profile, the one attached
below, as generated by Spider2/Pro 2.3.5.

I'll make 

Reported by natureh.510 on 2015-04-27 20:28:13

---

• Attachment: [Philips 274E - Lumiere du jour.icm](https://storage.googleapis.com/google-code-attachments/rawtherapee/issue-2753/comment-20/Philips 274E - Lumiere du jour.icm)

[Beep6581]

Hi Hombre, sorry I did not get time to test this, but your last comment contains an
important detail that may be the root cause. It appears that you exported Tiff from
RT without assigning a color profile and then used that to futher convert  for comparison.

No meaningful comparisons can be made outside of fully color managed process;)
I would use standard sRGB or AdobeRGB1998 in all software used for comparison.

Reported by michaelezra000 on 2015-04-27 21:40:06

[Beep6581]

Yeah, that's what I finally found out, but please, tell me why a color managed software
can produce images that, seen by another color managed software, produce so ugly results?
What's the point of color management then?

Reported by natureh.510 on 2015-04-27 23:39:50

[Beep6581]

If ICC version is supported it should work. RT_sRGB TRC is more detailed, may be it
is not correctly interpreted by other software?

Reported by michaelezra000 on 2015-04-28 03:57:26

[Beep6581]

I compared a version from dcraw with three from RT: one with no profile to which I later
applied OpenICC's sRGB in GIMP, another with a gamma=1 profile which i later converted
to sRGB in GIMP, and one with the normal RT_sRGB profile. No significant differences.
I am on a color-managed laptop screen though.

Reported by entertheyoni on 2015-04-28 06:41:23

[Hombre57]

Made some test again with a different colorimeter driver (Argyl CMS instead of DataColor one), set all settings to identical values (97% quality for jpeg) and then get identical result. I don't know what was involved in my initial experience, but I'm closing the issue, non relevant.

[iliasg]

@Hombre57 @Beep6581

I don't know what was involved in my initial experience,

Whith recent RT there is a significant improvement regarding posterization for 8-bit exports .. I bet it comes from better rounding in place of an ungly truncation before .. https://github.com/Beep6581/RawTherapee/pull/3432/files

[Hombre57]

So there really was an issue ? Glad that it is solved, thanks !

[iliasg]

@Hombre57 @Beep6581

There is still a difference between tiff-8bit vs jpeg-8bit exports .. tiffs are less posterized ;)

If a developer could take a look in Libjpeg or call libjpeg guys .. to investigate if low precision maths are involved there .. maybe jpeg posterization can become even less with some easy tweeks ..

From RT's part we can use non standard gamma curves which give more codes at the darks An easy solution for less posterization is to use the "free gamma" option and define there gamma = 2.40 - Slope = 15.0 (max) and it would be better if we had more headroom for the slope .. Or use a linear-log curve .. https://github.com/Beep6581/RawTherapee/issues/3333#issuecomment-224711875