darktable's haze removal tool?

[agriggio]

Darktable has a new haze removal tool. It seems to be very simple to use and effective. The code doesn't look too complicated. I'd like to "borrow" this for RT as well. Thoughts? (not for 5.4, too late)

[heckflosse]

@agriggio Where is that tool in darktable? Didn't find it

[agriggio]

https://github.com/darktable-org/darktable/blob/master/src/iop/hazeremoval.c

[heckflosse]

I meant in gui

[agriggio]

well you have to search in the big list of modules, I don't think it's in any of the tabs by default

[heckflosse]

Found it. Thank you.

[Beep6581]

Can it do something which curves and local contrast cannot? All examples I found when googling showed people using it to increase contrast in a way which curves would have done just as well, nothing to do with haze.

[agriggio]

I'm not claiming it is the only way, but it seems to be a simple way...

[morosakubek]

Something that specially removes blue cast from a distant plan (like mountains) or fog would be good.

[agriggio]

A quick update on this: https://filebin.net/3sdlzkzrsp6ufote/dehaze-preview.mp4?t=we3rfsvf

Not taken from darktable though. I got the algorithm directly from the paper: http://kaiminghe.com/cvpr09/

The implementation is trivial using the fast guided filter (see my pr #4849)

[sguyader]

@agriggio Sounds very interesting Alberto!

[Beep6581]

@agriggio could you show what happens when you run it on amsterdam.pef and soderasen_1.hdr.dng?

[agriggio]

@Beep6581 if you are looking for artifacts, they will be there :-) the tool is not yet done, this is only a preview... stay tuned for more news!

[Beep6581]

@agriggio no, I'm just curious to see its effects on photos I know well, and I wonder whether this could be used to add contrast to highlights, i.e. clouds.

[agriggio]

I don't think it's designed to work like that, but I'll give it a try as soon as I have the chance

[morosakubek]

@agriggio , I repeat, my attention, it should be combined for convenience with some blue cast removal.

[gaaned92]

Not taken from darktable though. I got the algorithm directly from the paper: http://kaiminghe.com/cvpr09/

@agriggio Yes introduction of dark channel prior dehazing should be a great improvement. As @morosakubek noticed, haze introduces also cast ( generally blue cast or orange cast in case of dust) and loss of sharpness proportional to the distance of viewed object. Is the method able to correct those two issues or a further correction using depth map as a mask should be added?

The implementation is trivial using the fast guided filter (see my pr #4849)

:open_mouth: :exclamation: :thumbsup:

[agriggio]

thanks for the comments. I am thinking about the colour cast, I have an idea that I want to try. I have no plans for sharpness though, as I think it's better addressed elsewhere. if you have sample images that you can share, they are most welcome!

[sguyader]

@agriggio here's a raw file showing haze and some light blue cast: https://drive.google.com/open?id=1rkwEcQ26-smpb_9N-pZSp8ynIIqvVWbZ

[gaaned92]

@agriggio Perhaps you know that: http://www.eng.tau.ac.il/~berman/NonLocalDehazing/ Searching for photos now.

[agriggio]

@gaaned92 thanks for the link, I didn't know that. I'll read it when I find some time. meanwhile I'll polish the code and push a branch so that other people can try it out...

[agriggio]

If someone is interested in trying this out, I've just pushed a dehaze branch. Here's a little demo on the raw provided by @sguyader (ignore the speed, this is from a debug build):

https://filebin.net/nqow4u9vbewd3nq7/Peek_2018-10-10_10-05.mp4?t=60661osf

As you can see, there is some haloing when pushing it too far. I've tried many things but for now this the best compromise I could reach between effectiveness and lack of artifacts. I also didn't do anything to correct the blue cast, I think it's better handled by the colour grading tools already available in RT.

Feedback welcome!

[gaaned92]

W10 build available: https://keybase.pub/gaaned92/RTW64NightlyBuilds/RawTherapee_dehaze_5.4-923-g14ac4babe_WinVista_64.zip

I notice 2 unexpected effects: 1- When strength is set to 0 there is still a strong effect as if one apply lightness.

2- the correction doesn't seem modulated by the depth map mask and seems uniformly applied. The effect should be minimal (or even null) for the near foreground (black regions of depth map)and maximal for far background (white region of depth map) with all intermediate correction strength as indicated by the depth map. Thus the strength should not have effect on near forground. I am not really able to observe that.

Beyond that, application of a luminance curve on the depth map mask should permit to modulate the correction.

Some hazy photos at https://keybase.pub/gaaned92/haze

[heckflosse]

@agriggio Alberto, there are some speedups possible. Or is it too early for speedups?

[agriggio]

Thanks for testing @gaaned92! I can assure you that the correction is applied following the depth map, exactly like specified in the paper. There is however a post-processing step that tries to equalize luminance, which is applied uniformly. Perhaps that is the culprit, I don't know. I will investigate more. Regarding setting the strength to 0, it seems to work as advertised here... but I'm working on a different branch, maybe I messed something up with rebasing -- I'll double check

[agriggio]

@heckflosse there are definitely many speedups possible -- if you want to have fun please go ahead, I will concentrate on smoothing the rough edges in the meantime

[agriggio]

@gaaned92 I confirm I messed something up when rebasing. I'll push a fix soon

[agriggio]

now it should work (more) as expected...: 75964dfd8b767ded7b92ea47f6f57159518e1be4

[agriggio]

@gaaned92:

Beyond that, application of a luminance curve on the depth map mask should permit to modulate the correction.

This is a nice idea, thanks! :+1:

[heckflosse]

@agriggio Alberto,

if you want to have fun please go ahead

Well, it's fun but it's also work, so I would prefer to avoid doubĺe work in case you change the code ;-)

I thought about an optimization in get_dark_channel and in the last loop where 'rgb2hslfloat' and vice versa is calculated. Is it likely that you change this parts of code soon?

[gaaned92]

RawTherapee_dehaze_5.4-924-g75964dfd8_WinVista_64.zip

uploaded at https://keybase.pub/gaaned92/RTW64NightlyBuilds/

It's OK now. thanks.

Using DSC_3445.NEF, with strength=75 I observe 2 effects:

• background is contrasted and the blue cast is corrected 👍

• but the foliage in the near foregroud is unnecessarily lightened. Is it possible to correct that?

Questions:

• Will it be possible to view the depth map?
• Surely somebody will ask to output the depth map.

And I dream of the possibilitty tu use in locallab such a mask in addition to present selection (which is a simple mask)

[agriggio]

@heckflosse those parts should be relatively stable, but I can't make any promise yet. did you take a look at the guided filter code? that's more stable, and I think there is room for speedups also there...

[agriggio]

but the foliage in the near foregroud is unnecessarily lightened. Is it possible to correct that?

I decided to apply an exposure correction that preserves the median luminance of the image, otherwise on most cases you would get a darker output. however, dehaze comes very early in the pipeline, so all the exposure controls can be applied on it...

Will it be possible to view the depth map? Surely somebody will ask to output the depth map.

both are possible with the 'trick' already adopted for wavelets, retinex and iirc also pixel shift. I don't think that approach is particularly user-friendly, but that's the only feasible way with the current architecture of RT. if that's ok, I can consider adding support for it. but I'd like to get the basics working well first

[heckflosse]

@agriggio Alberto, which trick for pixel shift?

[gaaned92]

so all the exposure controls can be applied on it... That seems ok

I can consider adding support for it. but I'd like to get the basics working well first Yes that can wait.

[agriggio]

@heckflosse if you show the motion mask, that will appear also in the output iirc. but maybe I remember wrong...

[gaaned92]

@agriggio Could you provide a patch that suppresses the exposure correction please?

[sguyader]

@agriggio Good job! When I apply the new Haze removal I often see strong halos, probably introduced by the blur mask. Would it be possible to control its radius? Also, it tends to blow highlights. Wouldn't it be better to prevent the filter to be applied to highlight regions, and maybe also to deep shadows, as often the parts we want to dehaze are rather in mid tone parts of the image.

[agriggio]

@gaaned92 here you go:

diff --git a/rtengine/ipdehaze.cc b/rtengine/ipdehaze.cc
--- a/rtengine/ipdehaze.cc
+++ b/rtengine/ipdehaze.cc
@@ -313,7 +313,7 @@

     if (newmed > 1e-5f) {
         const float f1 = oldmed / newmed;
-        const float f = f1 * 65535.f;
+        const float f = /* f1 * */ 65535.f;
 #ifdef _OPENMP
         #pragma omp parallel for if (multiThread)
 #endif
@@ -328,6 +328,8 @@
                 Color::hsl2rgbfloat(h, s, l, img->r(y, x), img->g(y, x), img->b(y, x));
             }
         }
+    } else {
+        img->normalizeFloatTo65535();
     }
 }

In fact, I started thinking that the brightness correction should not be applied. What is more important is the saturation correction, which is still applied. If you turn that off too, the results are really not nice...

[agriggio]

@sguyader please try with the above patch, it should avoid the blown highlights. Regarding radius, I can try exposing the parameter, but getting rid of the halos completely without sacrificing the effectiveness seems quite tricky

[gaaned92]

@agriggio Using DSC_3445.NEF with patch

Now the foliage is darkened. It is not what I expected. We can agree that for the foliage, the thickness of athmosphere penetrated by light is very small. So there is no haze affecting this foliage and thus no correction (or quite minimal) should be done on it. It seems that it is not the case.

neutral profile applied:

Haze Removal with strength=70 applied

Moreover there are artifacts on the foliage

[sguyader]

@agriggio It's better now, no more blown highlight. However, same remarks as André: the image is darkened overall. When pushed to 100, the results are quite uggly : some parts actually loose contrast behind the mask, see below the top of the foliage.

[Beep6581]

@agriggio maybe the issue title should be updated to describe the current algo?

[agriggio]

The algorithm used is the same as darktable, it's just the implementation that is different. As usual with research papers, the devil is in the details, so two different implementations of the same general idea might lead to quite different results, especially when tried on data sets that go beyond those used in the paper(s). Nothing new here :-) Regarding artifacts, halos, and darkening, well, I told you so, here and here. In general, I think this patch-based algorithm is not the best option when there is a detailed haze-free foreground that mixes with a hazy background, which is exactly what happens in your sample pictures. I tried several things, but the blanket is simply too short, so whenever I tweak something I get something else worse. From my quick reading, the non-local dehazing algorithm mentioned by @gaaned92 seems more promising, but it is also more complicated, and not something I can try to implement at the moment, sorry. Anyway, I am exposing more parameters, so that people have more sliders to play with -- it won't solve all the issues, but it will give me more excuses to blame the user's wrong choice of parameters for the bad results ;-) I'll let you know when I push.

[agriggio]

pushed

[gaaned92]

but it will give me more excuses to blame the user's wrong choice of parameters for the bad results ;-)

I fear I irritated you. Sorry. As I don't understand the algorithm, I am perhaps asking something impossible! :confused:

Forgive me if I insist. If you can get a depth map as the one below and use it as a mask to blend the corrected image with original image, there should not be any modification in the black region.

[agriggio]

@gaaned92 no, don't worry, everything is good. I'm not upset, just a bit frustrated because I've spent the last few days tweaking parameters and implementing various ideas, without much success. I feel like I need a break from this for a while, because I'm not being productive anymore. I hope someone else might give the paper a read (it's not hard), have a look at the code, and come up with some smart (or maybe obvious-but-not-for-me :-) idea...

[sguyader]

@agriggio thanks for your work. I understand your frustrations. I wasn't complaining when I showed the results I got. I was just reporting, in case you could find an "easy" solution. But I agree that it may not be the best tool and that you should not loose time and energy for marginal improvement. I still think that with your implementation, at strength 50-70 it improves hazy images, we just have to be careful and not go too far.

[agriggio]

fwiw, also darktable will produce artifacts if you push it too far. that said, I still have a couple of things to try before giving up. they are shots in the dark, but don't require too much work, so let's see...

[agriggio]

I've just pushed some more tweaks. If you have the chance, let me know if they improve the situation.

[sguyader]

@agriggio Yeas Alberto, it clearly improves the situation. Now I don't see the loss of contrast I mentioned in the earlier version. Playing with the depth and detail helps fine tuning according to the image. Great work Alberto!