sRGB export wrong without linear scaling calculation

[mediatechnologic]

The export of photos from DT 4.2 with downscaling or upscaling is calculating WRONG with nonlinear instead of linear RGB values. This does decrease brightness and does change colors between pixels and all fine details in photos are affected. The effect is well known as "Gamma Error in Picture Scaling". See www.ericbrasseur.org/gamma.html

You can reproduce this error directly with a checkerboard file. See my PNG test file in another post https://github.com/darktable-org/darktable/issues/12310#issuecomment-1374626148 about the same issue of DR color picker, DT darkroom scaling and DT photo preview. Here are additional explanations: discuss.pixls.us/t/wrong-color-picker-calculation/34204/19

The export has to result in the same perceived color as the 100 % checkerboard file with 1x1 pixel. This is a light yellow color (188,128,128) and NOT light red (188,188,128) for the 1x1 pixel checkerboard with light green (188,255,128) and magenta (188,0,128). This because the "average" of the green values 0 and 255 is not 128 but 188!

The image processing and viewing tools including all browsers who do result in light red or moiree colors are calculating wrong. The correct calculation would be to convert nonlinear RGB values to linear, then performing any manipulations (like calculation of average values) and finally calculating back to nonlinear RGB values. The formulas for this are well defined in IEC 61966-2-1 since 1999 and defined in chapter "Transformation" of en.wikipedia.org/wiki/SRGB

[todd-prior]

I look forward to seeing comments on this.....

[TurboGit]

Looks like the proper correction is done in gamma.c (last module in pixelpipe) for display purpose and that's why at 100% the picture is properly displayed yellow. But there is probably some missing conversion to cover other cases.

[TurboGit]

@mediatechnologic : By any mean if you are a programmer fill free to hack around dt and propose a PR. This is an advanced issue that will be difficult to fix without the knowledge you have.

[todd-prior]

display purpose and that's why at 100% the picture is properly displayed yellow

Would that track with 200 400 800 percent views being correct as well... Are those values that are in the drop down computed differently than any other zoom. 100 and each doubling of the zoom looks more or less correct...any other goes to the rose...

[jenshannoschwalm]

@mediatechnologic : By any mean if you are a programmer fill free to hack around dt and propose a PR. This is an advanced issue that will be difficult to fix with the knowledge you have.

I guess you mean without instead of with. Indeed a very annoying issue, i think a "must be fixed for 4.4" :-)

[mediatechnologic]

Thank you Jens-Hanno for highlighting the missing "without". I speculated also on "will not be difficult to fix". Or my lack of knowledge :-) In fact I never digged in any code in github. Should I start to learn it (aged>60)? I am not a real programmer, but only experienced with dos-batch, html, css, fortran and deeply with physics, math, optics, electronics, engineering and IT. Aren't the lin-RGB-pixel-conversions used nearly everywhere in DT?

Here is my small (programmed) contribution with a new "gamma" test file: Open, view, edit, scale, save it in any software including browsers and you will see if pixel calculations are done correctly.

[mediatechnologic]

I was asked to provide a raw file, which does show the export issue. A good idea! Best would be a photo taken by night of the dark sky full of stars. But that is not my passion. Please take instead of another synthetic image the REAL real world NASA image of the real world "Earth's City Lights". The scaling effect is described very good at: • https://legacy.imagemagick.org/Usage/resize/#resize_colorspace

However the link there is outdated. Here is the corrected link to various file formats up to TIF 40 MB TIF with 16382 x 8192 px: • https://visibleearth.nasa.gov/images/55167/earths-city-lights/55173l You can see immediately, when a software does not scale images correctly with this photo. All the bright pixel areas will loose brightness on the borderline between bright and dark, when the scaling is not calculated with linear RGB values.

You can mathematically get the "scaling error" by calculating the mean linear RGB value of all pixels. That is quite easy with ImageMagick (IM) and the msdos batch commands: set img=input.jpg echo input file: "%img%" magick %img% -colorspace RGB -scale 1x1! -colorspace sRGB -format "sRGB(%%[fx:floor(255*u.r)],%%[fx:floor(255*u.g)],%%[fx:floor(255*u.b)])" info: echo = mean linear RGB -^> sRGB value

With this commands you get just for comparison the wrong value of the correct image: magick %img% -scale 1x1! -format "sRGB(%%[fx:floor(255*u.r)],%%[fx:floor(255*u.g)],%%[fx:floor(255*u.b)])" info: echo = nonlinear sRGB mean value (incorrect!)

I have done 100 % scaled export in Darktable and scaled down/up exports. I you check original, well scaled and wrong scaled images with the above IM commands you see, that DT is scaling the exports wrong. The mean linear RGB value should stay constant for scaled images. For this NASA photo and other images with details, it is not with most image tools including DT. With the synthetic checkerboard image the effect and the RGB numbers of original and scaled export are obvious and exactly as I mentioned in previous post.

The scaling can be done with ImageMagick e. g. like this:

1. magick earth_lights_4800.tif -resize 1200 scale-1200x600_wrong.png
2. magick earth_lights_4800.tif -gamma 0.454545 -resize 1200 -gamma 2.2 scale-1200x600_gamma.png
3. magick earth_lights_4800.tif -colorspace RGB -resize 1200 -colorspace sRGB scale-1200x600_colorspace.png

No 1 gives the wrong scaling result, no. 2 is the correct but approximate (old) scaling and no. 3 is the correct scaling.

[mediatechnologic]

Below are the scaled files. The first three created with ImageMagick v7.1.0 and the last one with Darktable v4.2.0. The first two are OK. The 2nd is precise. The last two are wrong and too dark. Basically the color with incorrect scaling is also wrong (see above test image), but in a real world images in most cases probably negligible.

Remark: Open the images in a new browser tab! Otherwise the wrong browser scaling does add additional errors.

scale-1200x600_gamma.png mean linear RGB ---> sRGB= (18,20,46)

scale-1200x600_colorspace.png mean linear RGB ---> sRGB= (18,20,46)

scale-1200x600_wrong.png mean linear RGB ---> sRGB= (12,15,45)

scale-1200x600_darktable.png mean linear RGB ---> sRGB = (12,15,45)

The averaged nonlinear sRGB values are too low with the wrong scaling. The difference is not high, because most of the image is anyhow dark, but the difference is clearly visible. Vice versa the same will happen and will be visible for a bright image with small dark areas. Basically all details will be calculated too dark.

BTW: I am using Imagemagick not only for resizing all DT 100 % exports in batch mode, but also for the correct output sharpening. The sharpening parameters are not only a function of output size and expected viewing distance with display or print size, but also a function of the photo EXIF ISO readout value in order to avoid increased noise. I feel confirmed, that it is quite often better to have separate tools for separate tasks (DAM, editing, output processing, printing, publishing, presentation) and not one tool for everything.

May be DT should just use IM for exports? And use IM also for correct color picker area calculations?

[garagecoder]

Some things to consider:

• linear during downscale is generally better than sRGB, but not the only factor (e.g. filtering/interpolation type)
• linear during upscale is not necessarily better than sRGB (according to some image scaling experts), sRGB is even recommended
• example images help, but when scaling what works well for one extreme case may not be good for another
• imagemagick itself has been prone to scaling problems itself in older versions
• do you know what the default imagemagick downscale settings are?

Source of all of the above is the imagemagick docs.

[todd-prior]

@dtorop @TurboGit @mediatechnologic Has also proposed this issue to Aurelien over on his git... he took a look and actually agreed that there was an error

See https://github.com/aurelienpierreeng/ansel/issues/69#issuecomment-1399271366

and

https://ansel.photos/en/doc/modules/processing-modules/finalscale/

If there is agreement it would seem this would impact DT as well and should be considered....

[mediatechnologic]

Yes garagecoder, the upscaling is certainly a different topic. I can't imagine, why someone would ever need upscaling for high resolution raw/jpg images. For very big big posters/billboards?

Yes, ImageMagick had also a stony path for colorspace conversions, but today they are quite good. The default in scaling seems to be the input value, which is sRGB vor standard PNG/JPG files. This because IM cannot know, what is in the input file and what the user wants to do. So the user has to think about the correct colorspace conversions. If the input files contain other data (colorspace, metadata, dpi, gamma …) it turns even more complicated.

For nonlinear sRGB data the downscaling HAS to be calculated with linear values. I think there is no other way to do it correctly. But, after downscaling you CAN to do an output sharpening to compensate the downscaling effect. And this sharpening is not easy, because it depends on a lot of parameters including the image content and your intentions.

I am not happy, that I found at least one error in DT with this scaling issue. But I am happy that I was not wrong and not a PEBKAC. And I am happy, that Aurelien found a way to fix it.

There are a couple of reasons why good downscaling/sharpening is so important:

• The file size decreases dramatically.
• The image quality (sharpness) on display, TV, projector … is impressively better.
• The image resolution can be set exactly to the native resolution of the display. With this there is no need for the display device to scale (bad or wrong) and the speed of image loads/changes does improve a lot.

[TurboGit]

@mediatechnologic @todd-prior : As AP solution works actually? I tested the commit and made some tests changing the final-scale IOP order and didn't see any difference.

[todd-prior]

I haven't had any time to test it . I guess it would be most obvious on one of the test images used to demonstrate the issue and then to a varying degree image to image. I suspect @mediatechnologic can likely comment with more certainty wrt to the fix....

[dtorop]

It seems like the problem here is that colorout (which converts from linear working space to, in this case, sRGB) is running too early, and should move to after finalscale (which does the scaling). The problem is that there are still some modules (dither/borders/watermark, and clahe -- a deprecated "local contrast") which want to work in the output colorspace, and should run after the image is scaled. So these would need to continue to be after finalscale (and before colorout) and would most likely on the fly convert to/from the output colorspace during processing. This would also be a solution for #12310, if the color picker occurred before colorout -- but still occurred after any modules such as borders.

linear during upscale is not necessarily better than sRGB (according to some image scaling experts), sRGB is even recommended

If this is so, yet another wrinkle! We could allow for the user to move colorout to after finalscale in that case, excepting of course that finalscale is hidden from the GUI.

I can't imagine, why someone would ever need upscaling for high resolution raw/jpg images. For very big big posters/billboards?

Upscaling can be useful. For example, printing 26MP file at 12"x18" or larger on a 360dpi printer will require some upscaling. And I'd rather have this be done carefully by darktable rather than exporting it at 1:1 and relying on some other tool or the print driver to do this work.

There's been various discussions of iop reordering in the past, see #8051 and #8634, for example.

And also, colorout needs some attention to modernize, last I looked. I remember putting a bit of work into this, but it never made its way to the PR stage.

I haven't followed any of the recent AP work/thread on this. Unfortunately I'm back to being swamped, so if I were to work on this, it'll be months away.

[todd-prior]

It seems like the problem here is that colorout (which converts from linear working space to, in this case, sRGB) is running too early, and should move to after finalscale (which does the scaling). The problem is that there are still some modules (dither/borders/watermark, and clahe -- a deprecated "local contrast") which want to work in the output colorspace, and should run after the image is scaled. So these would need to continue to be after finalscale (and before colorout) and would most likely on the fly convert to/from the output colorspace during processing. This would also be a solution for #12310, if the color picker occurred before colorout -- but still occurred after any modules such as borders.

linear during upscale is not necessarily better than sRGB (according to some image scaling experts), sRGB is even recommended

If this is so, yet another wrinkle! We could allow for the user to move colorout to after finalscale in that case, excepting of course that finalscale is hidden from the GUI.

I can't imagine, why someone would ever need upscaling for high resolution raw/jpg images. For very big big posters/billboards?

Upscaling can be useful. For example, printing 26MP file at 12"x18" or larger on a 360dpi printer will require some upscaling. And I'd rather have this be done carefully by darktable rather than exporting it at 1:1 and relying on some other tool or the print driver to do this work.

There's been various discussions of iop reordering in the past, see #8051 and #8634, for example.

And also, colorout needs some attention to modernize, last I looked. I remember putting a bit of work into this, but it never made its way to the PR stage.

I haven't followed any of the recent AP work/thread on this. Unfortunately I'm back to being swamped, so if I were to work on this, it'll be months away.

Ya life has a way of keeping us busy. I know he has removed the histogram profile from his pipeline so it will be interesting to see the final result and how he feeds data to the colorpicker. I think its just good to be on the radar if nothing else

[saknopper]

If I understand correctly AP has (only) made the colorout and finalscale modules visible in the GUI and you should still manually reorder these if you wish to resolve this issue.

[todd-prior]

If I understand correctly AP has (only) made the colorout and finalscale modules visible in the GUI and you should still manually reorder these if you wish to resolve this issue.

I think you are correct. He has made mention to tweaking the pipeline down the road but for now I think you are correct.... so it is up to the users to move it before filmic....

[TurboGit]

@saknopper @todd-prior : But as I said I tried reverting the order of the module but for me this changes nothing. See my comment above.

[todd-prior]

@mediatechnologic @todd-prior : As AP solution works actually? I tested the commit and made some tests changing the final-scale IOP order and didn't see any difference.

Could be that for images where its an issue you can evaluate it but for many you won't notice much also sounds like due to the way the pipeline is sampled for previews etc that it is really only for exported photos...

AP's words "Resampling non-linear RGB is known to create artifacts such as fringes around sharp and contrasted edges, and should be avoided in general. However, in a majority of cases, it makes no perceptible difference, especially when processing low-dynamic-range images."

[mediatechnologic]

A little bit late, but after my normal daylight work I can use DT only at night, when its "dark" outside the "table". I really like to test now, but which version of DT or Ansel and where to download the win version? Sorry, I have no experience in finding commits and building executables.

[SoupyGit]

I really like to test now, but which version of DT or Ansel and where to download the win version?

I just tested the linux app image from https://ansel.photos/en/ and the final resampling module is visible in the pipeline. There is a windows .exe at the same site, but I haven't tested it.

As AP solution works actually? I tested the commit and made some tests changing the final-scale IOP order and didn't see any difference.

I have tested one image and there is a difference. RAW is downloadable from here: https://discuss.pixls.us/t/beneath-giants-how-to-make-it-look-good/31950 xmp used: DSCF9876.RAF.xmp.txt

Method: In Ansel, load xmp, move Final Resampling to between color balance rgb and Rgb Curve module. Both dt and ansel exports were 1600x, srgb (web safe), relative intent, jpg 100 quality, high quality resampling, lanczos 3 pixel interpolation, always use lcms2 off.

With darktable:

With ansel:

Difference is not really visible to the naked eye, but is if you load them in Krita and change blend mode to difference:

I chose that image because on pixls it had previously sparked a discussion on differences between resizing methods.

My 2c: Perhaps the biggest perk of ansel's implementation is the ability to do "post-resize sharpening" on linear data - that is, after Final Resampling and before Filmic (or other tone curve). But I did not do that here.

EDIT: For completeness, I also exported a version from ansel with Final Resampling in its default position (after Output Color Profile), and in Krita see almost no difference to dt export (just subtle spotty difference which looks like grain, and I guess is due to compression). This shows the difference we see above is indeed due to the different placement of Final Resampling in the pipeline, and not anything else.

[todd-prior]

I will have to get an updated Ansel.. I recall AP suggesting that it could go before filmic or sigmoid or after but before colorout. I wonder how much difference it makes to pick one of the other options in this case....I guess before ??

It would be good to see the impact on the checkerboard too.....

[jenshannoschwalm]

Just putting the module to an earlier place of the pipeline doesn't help for the mentioned issue here for me.