Highlights and shadows recovery like in rawtherapee

[brigazvi]

In the last version of rawtherapee they released very good and simple module for highlights and shadows recovery. We can create similiar one, or use the same?

[TurboGit]

Have you tested the shadows&highlight dt module? If so, what's wrong? What is better with rawtherapee? What do you expect from dt?

Please, stay a bit more focused otherwise we won't make progress and we won't be able to follow every issue. Have quite generic issue report like this may waste lot of our time. Thanks.

[ff2000]

I also wanted to ask if DT could implement that tool in a similar way RT does. RT now uses a guided filter which prevents halos and keeps a natural look. First RT. Left is "neutral" profile which doesn't apply any curves etc., right is just the highlight slider pulled to 100: DT also dialed back to "Original" and pulled back the highlight slider so clouds don't pop too much. You already can see a big halo going on. Can be somewhat softened by a bigger radius, but I never can eliminate that over-the-top HDR-look.

[TurboGit]

Try using a bilateral filter instead of the default gaussian one to avoid the halos.

[brigazvi]

In RT its more simple and easy and the result more natural.in DT the result seems a little bit flat.

[aurelienpierre]

darktable's H&S is wrong from the beginning. It's not just a matter of gaussian/bilateral/guided masking, it's the colorspace where it works. I plan on adding a tone equalizer, which would be a S/H module on steroids, with more channels, and using a laplacian pyramid as local contrast does. Stay tuned with #1904

[brigazvi]

@aurelienpierre This direction is welcome, but it complicates things further. What needs to end up is a module called shadows and highlighs recovery that includes two bars: shadows, and highlights. It just needs to work well in terms of colors and a more natural look in relation to the current module.

[aurelienpierre]

Think about it… What are highlights ? -1 EV under pure white ? -2 EV ? At -2.45 EV, you have mid-grey already.

Same applies for shadows, with a huge difference: black does not exist. So you would have to decide if your black is -8 or -10 or -12 EV from pure white, then how far from black the shadows begin.

Whatever you answer, it's a developer arbitrary choice. So, having a full equalizer with channels spaced evenly by 2 EVs is just making underlying choices accessible to the user. The logic is the same, you just get more control over the full range of luminance.

The current module is flawed from its core. The proper way to do that is using log-encoded RGB or xyY space at the beginning of the pipe, after exposure module, but it works in Lab at the end of the pipe. So don't hope a quick fix, it needs to be re-written.

[brigazvi]

@aurelienpierre I am very happy that you are working on a solution. I do not really understand the technological significance of things, I just know how it works in Lightroom and CaptureOne. There are not too many choices and yet it works amazing. Simple and amazing. At the end of the day most users use several basic modules and we need to make sure they are the best possible.

[aurelienpierre]

At the end of the day most users use several basic modules and we need to make sure they are the best possible.

I will never understand why users go on the raw path to take control on the pictures, while just wanting simple basic modules that just work. It's a paradox to me.

Lightroom and Capture One have users trapped in their engineering decisions. They have decided for you what "shadows" and "highlights" mean in terms of luminance values. Problem: these meanings assume you work in clipped display-referred RGB space, in an ICC workflow. That's rather restrictive.

If you work in HDR scenes, you are screwed, because that doesn't fall into the "correct" use-case they designed for you, on the assumption you will work on camera raws in an ICC workflow. So, that sums up to a very simple design dilemma: do we want a fixed tool, simple but limited in use because of all the underlying assumptions it makes without telling you, or an all-purpose tool, adaptable but with more parameters ?

The ICC workflow is not well-suited for HDR work, and we have to think of future-proof workflows that can use ICC but don't rely on it. In these setups, "highlights" can have very different meanings.

Having a tone equalizer is not much more complicated than a simple shadow/highlights. You will have whites, highlights, mid-tones, shadows, deep-shadows, blacks and deep blacks instead. More controls, but the same use principles apply, and nobody will decide for you how large the highlights range is.

[brigazvi]

@aurelienpierre I think people prefer simple modules because that's how they manage to get better results. Complex tools make them get lost. When a photographer wants to achieve a higher dynamic range than his camera allows, the only way is to shoot raw and work on it in an image processing software. I do not rule out the option of a simple default tool, with a hidden menu of optional professional options.

[aurelienpierre]

When a photographer wants to achieve a higher dynamic range than his camera allows, the only way is to shoot raw and work on it in an image processing software.

The point is not to achieve higher dynamic range than the camera offers (that's not possible), but take full advantage of what the camera has in its guts. Now, having more than 14 EV of dynamic range at 100 ISO is standard, and it's HDR already, since JPEG and most displays have only 8 EV available.

The whole ICC color-management chain was built assuming 100 % luminance was diffuse white, and everything above was clipped. So, middle-grey was at 18 %, and, with older cameras and color-film, black was at -8 EV from diffuse white. So, everything below 0.4 % luminance was assumed black. All in all, you exposed for mid-tones, got your grey at roughly 18 %, and there was not much variability in pictures, so you could just decide that highlights were above 50 % luminance, and shadows below 0.8 %, and that was about it.

Now, with 14-EV cameras, 100 % luminance can be direct sun-light, but since everything above is still clipped, you exposed to the right and recover shadows later in software. Meaning the actual mid-grey can be as low as 4 % luminance. Meaning your "physical" highlights can actually range from 12.5 % to 50 %, up to 100 %. And that depends on your scene, camera, ISO setting, etc. Because at 3200 ISO, most cameras have barely 8 EV of dynamic range remaining.

So, I understand that users love having 2 cursors doing magic for them without the need to understand what's going on, but how do you expect me to know, at the time I code, what is the luminance range of highlights for today's, yesterday's and tomorrow's cameras, at the current ISO ?

It's not even about having pro vs. basic options, is just about having physically-meaningful options. And the thing is "highlights" (as in LR or C1) can actually mean "whites", "highlights" or "midtones" depending on the picture base exposure. Is that reliable, systematic and reproductible ? Nope.

The only thing I know is cameras have a top dynamic range bounded at 16 EV, because they use up to 16 bits integer encoding. So I split the dynamic range in 8 evenly-spaced channels, every 2 EV, from 0 (white) to - 16 (black), I give you a cursor to manage each channel, and you will be able to adapt to every possible exposure scenario without having stupid devs making stupid choices for you, assuming you are more stupid than them and too lazy to read a manual.

Sorry, but in this case, reliability and simplicity cannot be achieved at the same time. Or you get a half-baked shadows-highlights module, that is simple to use (maybe) and messes up you colors at the same time.

[brigazvi]

@aurelienpierre

1. I feel the need to apologize if I sound too critical. I'm not a developer and I'm just trying to express my opinion. I greatly appreciate your work for Darktable.
2. How much do you suggest differs from the zone system module?
3. I am aware that it is not always possible to arrive at a simple solution. But I find it hard to believe that such large companies in the industry that millions of professional photographers use their software offer such a poor solution. I tend to believe that their solution (which is also very impressive) is pretty good.
4. Who knows, I have not seen your solution yet, maybe it's much better than I imagine?

[aurelienpierre]

Don't apologize, it's ok. Different points of view are to be expected between devs and users. I'm just trying to explain that what seems simple from an user-perspective is not necessarily future-proof and trying to replicate front-end features without undestanding the underlying assumptions they make can blow up in your face.

2. zone system works too late in the pipe, and in a way that need to take from one zone what you give to another. It can be also damaging for color preservation and local contrast preservation. What I propose is fully independant channels, working early in the pipe, like simple selective exposure adjustments, in a way that preserve both colors and local contrast.

3. Be extra careful with "simple". It's not because large companies are behind the soft that everything is clean under the hood. Especially when design choices have been made 25 years ago, for 25-years-old gear, and force-forwarded for today's workflow. Don't underestimate the power of habits and legacy.

My design politics rely on the ACES, which is a modern & future-proof standard. Basically the whole cinema industry is switching to that since 2014, but photography holds on legacy workflows because they "just work". How long do you think we will be able to afford things that just work ?

Image processing is hard. "Simple" solutions often fallbacks to limiting the features, making the code more rigid, and restricting the use. I find that it's sometimes better to not hide the intrinsic complexity of what's going on under the hood, and just educate users instead. That results in more efficient workflows, even if the UI might seem more impressive, because the tool can be adapted to various uses. Otherwise, you might end up fighting against the software to overcome its limitations, and get locked in its design choices, which is somewhat acceptable in proprietary software, but certainly not in free/opensource.

Good design is not things that just work now. It's things that are done the proper way so they work with as less side-effects as possible, and they keep working in the future, when cameras have maybe 24 EV of dynamic range. Because color-management in HDR setup is a serious and tricky issue.

4. See for yourself : https://discuss.pixls.us/t/a-tone-equalizer-in-darktable/10678

[brigazvi]

@aurelienpierre I still think it is important and worthwhile for things to be simple. I do not believe in educating the user. Still, it seems your idea is not so bad. The screenshots seem like a pretty simple tool. I hope it will really work well and replace the current module as default. Thank you very much for this amazing work. How much longer will it reach users?

[aurelienpierre]

How much longer will it reach users?

ASAP :slightly_smiling_face:

[brigazvi]

@aurelienpierre I would recommend changing the name of the module to new shadows and highlights module. Only for the purpose of stating intentions. This should replace the existing module. It may also be useful in the interface of the module to give up EV values ​​and leave only the simple language of shadows and highlights. what are you saying?

[brigazvi]

Many developers thinking that many modules is better. But sometimes it's just confusing. If the new module is better, I think the old one need to go home.

[aurelienpierre]

Bad idea. EVs units are actually quite explicit to state what's going on (perceptual logarithmic space), especially for those who are familiar with flashmeters (pretty much every professional photographer and old-school film shooter), and consistent with filmic and exposure settings. EV means it's a physical measurement you could acquire on set, using a lightmeter, so, again… making it "simple" will lose some accuracy and the link with the analog reality.

Actually, the labels are wrong in general, because they depend on the actual exposure setting/dynamic range. For example, in a 8 EV dynamic range scene, mid-tones are around -2.45 EV, so the -2 EV channel is high mid-tones already. But in a 12 EV one, the mid-tones are around -4.45 EV, so the -2 EV channel is truly highlights, and the -4 EV is indeed mid-tones. The only "true" part of the label is the EV measure, the rest is a "general audience" hint. Lose it, and it becomes a dumb software.

Also, a tone equalizer is something people already know: https://www.acdsee.com/en/lighteq That's far from a simple highlight/shadows module.

I do not believe in educating the user.

If so, users are invited to shoot in-camera JPEGs… They are pretty good when the camera is set properly, and firmwares do sometimes a better job than any raw editor, because manufacturers know their sensors better.

[brigazvi]

@aurelienpierre Thanks for your answers.

I did not really understand whether the bottom line is that you are offering an alternative to the existing module. The simple user, when he wants to recover shadows, will go to Shadows and highlights. In practice, Darkroom has a lot of modules that can recover shadows, the question is which one does it best, and which one we, the developers, recommend and say is the shadow recovery tool.

[aurelienpierre]

You are welcome.

I did not really understand whether the bottom line is that you are offering an alternative to the existing module.

It is an alternative than can do much more.

the question is which one does it best,

Define "best" and "simple user" first… I design for professionals. That is, people with understanding of what photography is, willing to have a reliable tool, working in a predictible way, giving reproductible results in a minimal number of steps, and grounded in the physics of light to have on-stage measurements transferable into software settings. In this context, for me, the best module is the one allowing luminance editing without affecting the color hue and saturation at all.

The modules that currently fail at this are :

• basecurve (hue shifts, desaturation of highlights, oversaturation of shadows),
• tonecurve (hue shifts, desaturation of highlights, oversaturation of shadows),
• highlights & shadows (hue shifts, haloing, desaturation, oversharpening),
• global tonemapping (oversaturation, unreproductible results between pictures and zoom level),
• levels (de/over-saturation, depending on the grey adjustement)
• local contrast (desaturation, hue shift towards greyish-blueish).

Incidentally, all of them perform non-linear operations, and all except basecurve work in Lab colorspace.

There is only one way to completely preserve color, while performing luminance adjustments: doing linear operations in a RGB linear space (that's what the exposure module does, and what the tone equalizer will do too). Some good trade-offs involve doing non-linear operations in normalized XYZ space (that is, xyY) or in a normalized RGB space (what filmic does). In Lab, the hue linearity is a lost cause from the beginning and there is nothing we can do, because this space has never been designed to be an editing colorspace.

To preserve colors, you have to keep things stupid simple and stick to multiplications of RGB values. If you can't (e.g. you want to apply contrast), you have to use a proper space and some sweeteners to deal with the mess you do on colors. Besides, using blurred masks in Lab does not make sense since you will get weird transitions between contrasted areas. See: https://www.youtube.com/watch?v=LKnqECcg6Gw

and https://www.visualeffectssociety.com/sites/default/files/files/cinematic_color_ves.pdf

[brigazvi]

@aurelienpierre

I'm not really able to fully understand your technical explanations. I agree with you that we appeal to professional photographers. A good tool is a tool that brings better results. That was what I meant.

for me, the best module is the one allowing luminance editing without affecting the color hue and saturation at all.

The least I know technically is that saturated red that becomes dark, in our case because of lack of exposure, loses its saturation. Dark red will always be less saturated than normal red. So I do not know if a change of luminance alone will give us the original color of the picture. I'm wrong? The examples cited in pixls do not show the function of the module in slightly more extreme situations. And again, thanks for your attention and investment in Darktable.

[aurelienpierre]

Here is a dynamic range compression:

With S&H:

With my tone equalizer:

See how the bush (right bottom corner) turns into magenta in the S&H version, while it remains green in the TE one ? Also, S&H resharpen too much, whereas it's not its purpose.

The least I know technically is that saturated red that becomes dark, in our case because of lack of exposure, loses its saturation. Dark red will always be less saturated than normal red. So I do not know if a change of luminance alone will give us the original color of the picture. I'm wrong?

Not sure what you mean here, it seems you mistake chroma with saturation. By definition, saturation is independent from luminance. But, if you move the luminance at a fixed chrominance, you might end pushing the color out of the gamut, which is a completely different issue.

[github-actions[bot]]

This issue did not get any activity in the past 30 days and will be closed in 7 days if no update occurs. Please check if the master branch has fixed it since then.

[aurelienpierre]

fixed by Tone EQ