RawTherapee and very high dynamic range support / dealing with highlights

[Beep6581]

Hello

I often run into trouble when I try to increase the general brightness of a photo while preserving details in the highlights. In such high dynamic range cases I always bracket and merge using HDRMerge so I'm sure that the floating-point DNG raw file contains details everywhere, but it seems that RawTherapee's tools are not tuned to dealing with such high dynamic ranges and so getting a good result can be very difficult.

Here is an example, the neutral version:

I want to make the photo much brighter and happier, but I also don't want to have a gigantic clipped (or badly crushed) blob on the wall:

The current "Highlight compression" slider as well as the highlights part of "Shadows/Highlights" is completely ineffective here. Try it, I included the PP3.

Here are the raw files (from five shots bracketed 2EV apart, combined in HDRMerge): http://filebin.net/ebwcyep3gj

I opened issue #2476 to get some popular tone-mapping operators into RawTherapee, maybe they would help, but I wonder, maybe you have other ideas? I hope that by opening an issue for this difficulty we can find a solution, or maybe tune the existing tools ("Highlight compression" or "Shadows/Highlights") to work better in such high dynamic range images.

[paulmatthijsse]

Hello, for the first one I used some sliders in the exposure tool plus wavelets, the second one is the same plus tone mapping.

pp3 here: https://www.dropbox.com/s/4gu4vzvh3oe2hh1/2015-12-04_karin_library_01_01-05.exposure%2Bwavelets.800px.jpg.out.pp3?dl=0

pp3 here: https://www.dropbox.com/s/oaqoaxjgeg61hww/2015-12-04_karin_library_01_01-05.exposure%2Bwavelets%2Btonemapping.800px.jpg.out.pp3?dl=0

[Beep6581]

The problem is that you cannot recover detail in the highlights. Try making the detail of the glass light cover at the back of the corridor visible. The problem is even more apparent in the other DNG, _02_01-05.dng

@Desmis @atorger maybe you have some ideas whether it's possible to tweak the code of RT's highlight recovery to deal with this?

[iliasg]

after disabling the tone curve and look table from the color profile the highlight recovery is much better. My try .. https://drive.google.com/file/d/0B0NqktTgc54sUTNXTDExcDllMGs/view?usp=sharing

[Beep6581]

These examples should explain the problem as clearly as I can put it.

Here is the image with the Neutral profile, no DCP tone curve, everything neutral apart from the curve and black.

The image is generally in the right ballpark, but bland. I add a second curve to fix that (I could use L_a_b* or CIECAM02, doesn't matter, I'll stick to the Exposure tool for this demonstration). There, fixed (let's say), now the photo doesn't look depressing:

Just one problem left: the clipping. Setting "Highlight compression" from 0 to 500 does nothing. 0: 500:

It looks like Shadows/Highlights is my only hope, but requires extreme values with this HDR DNG, and the difference between 100 and 99 is drastic: I can't use local contrast at all, it goes immediately crazy.

Note that I'm not asking for help developing these photos, I want to bring attention to the lacking control over highlights. Basically there is no way of dealing with highlights in HDR images in RawTherapee. The tools are not tuned to dealing with the dynamic range offered by HDR raw files (whether from HDRMerge of high end cameras).

[raceviper13]

the way i process photos is like the following. don't know if you like the results, but i primarily use curves for all photos as I can get the best results. the sliders don't do much for me except exposure compensation. this is similar to Beep6581's method. unfortunately, because i max out the top curve to get the image to where i can then really start adjusting it, I am a little limited as I have to complete the processing with exposure compensation and the 2nd curve. open full size for some notes.

[wilq32-pwitkowski]

I don't add anything valuable to the discussion, but maybe you will like this idea xD

[Beep6581]

Off-topic, but @wilq32 I do like it :] What approach did you use?

[Hombre57]

What approach did you use?

He used LightRoom ? :) Otherwise, I'm interested by the technique as well.

Here is mine, 100% done in RT, and which try to retain some details in the HL (see the lamp in the center of the image).

2015-12-04_karin_library_01_01-05-2.jpg.out.pp3.txt

[wilq32-pwitkowski]

LightRoom? Nooo Lightroom, no DxO pro, no Captrue One, no Camera Raw, i tried many different RAW converters and best results only in RT - this program is really best, I haven't yet answer cause I left my pp3 file in work so I couldn't upload it. I will do that in a few hours.

[wilq32-pwitkowski]

2015-12-04_karin_library_01_01-05.dng.pp3.zip

Here you go, keep in mind that weird stuff going on there, as I was using your DNG to playing abit with RT options. Most of the style come from curves in film-like style and Retinex. Other changes are subtle and not that important.

I didn't touch highligh problem at all and they still bit overexposed as I was working only with one file and... let's start overexposing topic from mentioning few things:

• "Highligh compression" as well as "Highligh/shadow" is used only when your changes in RT (increasing lightness, etc.) makes one or more channel blowed. Using this slider you can get back those area back to the normal. It does NOT recover highlights in areas that are already overblown on RAW file.
• For recovering highlights in one image you have an Highligh reconstruction option that tries reconstruct areas that are overexposed, yet it's just some aproximation - this cannot create information that is lost because of getting out of sensor range.
• I don't have enough knowledge to tell you how to merge bracketed exposures inside RT (not sure if it's implemented at all), but approach you've taken to make it outside of the application sound like something that could work. Yet make sure that after you're back to RT - there is no overexposed areas after merging. Brief search using google point me here (as I though you have to do it outside of the RT):

http://forum.luminous-landscape.com/index.php?topic=90208.0

Hope it helps,

Cheers

[Beep6581]

For the record, I tried processing this file (well several bracketed exposures of it saved from RT) in Luminance HDR and none of the 9 tone-mapping operators let me recover details in the highlights. I guess there is software which lets you do that based on the photos I've seen... If any of you have access to a recent Lightroom, please see whether the Highlights slider will recover detail in the highlights if you make the room as bright as in some of the versions above. If it doesn't then I suppose we can close this as I'm fantasizing about something which can't be done (or which can't be done automatically - one should be able to do this manually using luminosity masks in GIMP or Photoshop).

[wilq32-pwitkowski]

Oh sorry I just realised that you did not included bracketed images, can you attach also other files? When I look at the one you've provided in RawTherapee i see it still contains problems in light areas.

[Beep6581]

@wilq32 both DNG photos I uploaded are each bracketed from 5 shots 2EV apart. A part of the lamp in the foreground of shot 01 is clipped, the one in the background is not. In shot 02 there is no clipping (except in the filament of the lightbulb but that's fine).

The Right Thing to do was to use a strobe to match the intensity of the lamps, but I was curious whether instead of a strobe I could use RT.

[wilq32-pwitkowski]

In a zip file i see only 2x DNG and they are completly different, or am I miss something?

[raceviper13]

I use highlight recovery and total exposure compensation in order to recover highlights. The highlight recovery and threshhold sliders don't work as well to my liking, so I don't use those excepting strange cases. I tried again to get closer to what you guys were seeing to help show how to get these results using other methods. I even caved in and uses the dreaded tone mapping tool (only dreaded because I don't like the typical look of tone mapped images). However, the Shadow/Highlights tool still acts very strangely with these HDRMerge files. Maybe it's just a problem with that module.

2015-12-04_karin_library_01_01-05.dng.zip

[wilq32-pwitkowski]

Again... Your entry .DNG file is already clipped, take look at the image I've pasted - on historgram you have pike, with highligh clipping enabled i see they are overexposed. You can of course take exposure down but this doesnt mean that there is information there. Again highligh reconstruction only will try to aproximate this, yet if you have non overexposed bracketing image I would rather assume that HDRMerge was done incorrectly.

P.S. Didnt notice that I answering someone else xD Well to me results are not strange, but expected when it comes to overexposed areas :)

[raceviper13]

I disagree that the overexposed areas are of any concern. RT's highlight recovery is super-advanced and exceptional at recovering highlights. Is it perfect? By no means, as no algorithm can perfectly recover information that was lost. It can only be estimated. The estimates usually are imperceptible to me. Only when they are perceptible, do I discontinue further highlight recovery (usually more than 1EV recovery).

RT is very advanced, but there are areas obviously with room for improvement, and I believe this is one area.

I think we need to also consider we are trying to compress a very large range of brightness values into a much tighter range. this is difficult while also retaining some contrast. That's why people like local contrast so much because it can generate contrast where there wouldn't be any because of the compression.

[wilq32-pwitkowski]

RT for me is most advanced RAW converter that I know and gives flexibility that cannot be achieved in any other RAW converter.

Honestly I don't miss highligh reconstruction that much as It's same problem as with resizing - you can't get new information from 1 pixel, you can estimate/aproximate it - but it will never give you new information that simply doesn't exists. What I rather miss here is interface to make bracketing inside RT, maybe simple link to HDRMerge or so? Can't say.

[stefan-i]

Where Lightroom differs from RT is that it has a slider that can increase the contrast in the highlights ("decompressing" them if you like). I think this would be useful in RT as well, maybe it could be done e.g. by allowing negative values for the highlight compression slider. I've already opened an issue here a while ago: https://github.com/Beep6581/RawTherapee/issues/2216

[Beep6581]

There is clipping in the source images, but it's insignificant. If in doubt, use the second DNG. This issue is about recovering detail in highlights which are not clipped in the raw file, it's about pointing out that RT's tools are not tuned to dealing with HDR images and maybe someone has an idea how to tune them better. In many cases RawTherapee handles highlights very well and in a natural way, but I would also like to be able to compress highlights without washing out detail (as happens in the examples above, for example even though the lamp in the background has some detail, this detail is hard to discern, it's washed out to too compressed), and as the attempts in this issue prove, there is currently no way of doing that.

Someone in the forum mentioned last year that recent Lightroom versions have a completely reworked Highlights slider which manages to compress very bright clouds while retaining details whereas RT washes them out, that's why I'm curious whether anyone reading this with Lightroom could give these images a spin.

P.S. haha stefan-i mentioned it while I was writing this.

[wilq32-pwitkowski]

Second DNG is also overexposed? :)

Anyway I understand from this topic that the slider is not that magic-slider thingy as in Lightroom, and normal user doesn't want to go trough Curves to fix it?

[wilq32-pwitkowski]

Actually I did some more testing and it seems that you're right and Highlight recover doesn't do what it suppose to do - It doesnt show back details in highlights, for sure there is a room to improve here.

[iliasg]

Let me try put things in tact (in my mind) and perhaps help the discussion ..

With RAWs we start with a linear representation which for HDR content like DrSlony's sample has a range of around 20stops. But we have to export them for visualization on displays with less than 10stops DR. We need to compress ..
Any control or curve (in exposure panel and not only) we use is in fact a curve. Curves expand (i.e increase differencies) and compress sectors of the brightness range according to their shape. It's an axiom that having a finite brightness range any expansion of a sector will result in compression of another sector. In my sample above the first curve expands shadows and compresses light-highlight areas. At highlights a difference of 1% becomes around 0.1% i.e. invisible (1% is JustNoticableDifference). (I always like to use more control points so that I can keep the high range linear and not overcompress any sector). We can correct sectors of interest by using the second curve (as pictured in my sample above) and give at selected brightness ranges increased slope (contrast). I hope this technique gives a tool as asked by @stefan-i :) In this sample raw's histogram in fact we have

• interesting shadow areas (-20 to -8stops) which we want to make brighter and more contrasty,
• then a not interesting mid-light range which we want to (have to :) ) compress heavily
• highlight area where we want to not destroy tonality by overcompression.

After curve manipulation we need local contrast .. in RT we have many tools for this job .. In my opinion the local contrast in shadows/highlights tool is unusable. I prefer either CDBL (level 3-4-5) or/and Tone mapping (edge stopping 1.00, scale 0.5, iterates 5-9). I would add wavelets but I cannot master some color changes and artifacts there :(

Another aspect of the HDR issue is our display's brightness. I think that HDR content needs a different setting i.e. increased brightness-contrast (to maximize it's range) from our current 150-200 IRE to the max our monitor can provide (mine desktop monitor is at around 400 IRE) .. I will try when I will be in front of it

[raceviper13]

Like iliasg shows above, I use curves extensively and would love to have two more curves for exposure. I use them so much, it's hard to get exactly what I want with just two of them. Sometimes when a second point is added in "control cage" mode, it can really jack up the curve. Especially if the points are close to each other. The curve is apparently required to contact the control cage at the midpoint of the dashed line between each point. This causes me to only want to use one mid-point per curve if at all possible.

One other thing to consider, i am sure to the human eye, the original scene we are all looking at here looked rather different. I know the (stationary) eye can see better than a camera, but not as well as RT is presenting here.

[Beep6581]

@wilq32 see https://github.com/Beep6581/RawTherapee/issues/3061#issuecomment-171933942 As I wrote several times, this clipping is insignificant, it makes no difference - I don't expect to see the clipped filament, I do want to see the globe which is not clipped.

@iliasg

In my opinion the local contrast in shadows/highlights tool is unusable

Exactly, it's unusable with very high dynamic range images (effects are instantly crazy), it's somewhat usable with lower dynamic range ones.

[heckflosse]

@iliasg , @Beep6581 : Do you speek about local contrast in shadows/highlights with sharp mask enabled or disabled. The results are completely different. IMHO when sharp mask is enabled, lc in sh is usable. When sharp mask is disabled lc in sh is not usable

[Beep6581]

It looks much better when using the sharp mask, but it still brutally clips the darkest tones.

[wilq32-pwitkowski]

@Beep6581 Yes, I've mention that finally i get the point what you're talking about. Uhm I guess apart from alot of of compression using curves not much here can be done easily and there is a room for improvement here. What i suspect is happening right now is that a highligh recovery is kind of linear (something what @iliasg described before I guess?) so when we have like superbright detail and if we want to light up shadows it gets almost 100% white - this 'almost' is not 100% same as lighter pixels around because we can't see it, yet there is probably a slightly linear difference in place where details was there. I guess there is some room for improvement here, at least for this slider. Right now we have partialy control over it using curves in different areas (yet they are more about making sure that lighting things up does not affect bright part, rather than making details in highlights nonlinear?)

[heckflosse]

@Beep6581 I vote for disabling the lc slider in sh when sharp mask is disabled

[stefan-i]

Iliasg, thanks for your example of using the tone curves! That's indeed one good option to "decompress" highlights. Still, I find it less convenient than just pushing a slider. I find it somewhat difficult to create a tone curve that just decompresses higlights without also changing the darker tones.

Let me plug once more my idea of using the highlight compression slider for this purpose. Now it is used to compress highlights when used with positive values. Why not simply allow negative values, which then does the opposite, "decompressing" the highlights. It would not break backwards compatibility and would introduce a valuable feature without adding another slider :)

Still, I agree this would not solve the main problem of this thread, which IMO would require some kind of tone mapping operator / local contrast adjustment that is mainly targeted at the highlights.

[pygmalionshell]

Old thread but: do you guys think that with a flat DCP profile we could get better results? The VisionColor BMDfilm looks good on my high dynamic range photos. Unfortunately I can't show it because the DNG file is now deleted from the host. If anyone get it uploaded again I could do a test.

[Beep6581]

No.

[pygmalionshell]

Won't help the flat DCP? Well, at least it gives a good inverted S-shape curve, that's a start. The Retinex seems to work very fine now so I guess that's why it's an old thread. Thanks.

[heckflosse]

@Beep6581 Still valid with fattal?

[Beep6581]

Solved by the addition of the "HDR Tone Mapping" tool (Fattal) in 5.4.