Additional Median Filter for Denoise

[Beep6581]

Originally reported on Google Code with ID 2423

I have just tested a supplement to denoise by adding at the end of treatment a 3x3 median

It does not cost much processing time (15%)

I was surprised by the result!

It can be used alone or in addition to the rest.

What do you think ?

:)

Reported by jdesmis on 2014-06-18 21:18:24

---

• _Attachment: denoise_median.patch_

[Beep6581]

Jacques, when used in addition to the rest, processing time could be lower (less than
15%, but is almost ok). When used alone, it's unusable slow, because you forgot to
disable the normal denoising part in this case. Processing time for median only with
a D800 file was about 14 seconds with your patch!!!
Disabling 'normal' denoise if not used reduced the processing time to about 2.2 seconds.
Going back to old network-based median reduced the processing time to 0.67 seconds.

This is included in the modified patch.

And I bet it can be further reduced by changing this:

copy a => b
median b => c
copy c => a

to:

median a => b
copy b => a

;-)

(this part is not included in modified patch)

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-18 23:08:43

---

• _Attachment: denoise_median_op1.patch_

[Beep6581]

Here's the patch which doesn't need the additional copy for median calculation. Processing
time for 'median only' is now about 0.42 seconds with D800 file (was 0.67 seconds with
last patch). Memory consumption is reduced by width*height*3*4 byte compared to last
patch. Code is not beautified actually... Just a quick hack

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-18 23:50:38

---

• _Attachment: denoise_median_opt2.patch_

[Beep6581]

Next one: processing time for median reduced to 0.29 seconds with D800 file and memory
consumption reduced to width*height*1*4 byte ;-)

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-19 00:09:50

---

• _Attachment: denoise_median_opt3.patch_

[Beep6581]

Hello Ingo

Comme on dit en français...tu es comme Lucky Luke "plus rapide que son ombre"

Thank you for this very fast improvement...

I had seen #1, I thought to do today ... thank you yet done.

If I used the function "fq_sort2", in anticipation of using anything other than 3x3

Thank you very very much.

Jacques

Reported by jdesmis on 2014-06-19 05:34:16

[Beep6581]

Hello Jacques and Ingo!
I did a quick test before leaving for work and from what I could see, your's and Ingo's
patch from comment #3 looks really good: A real visible improvement compared to default
on a noisy D700/ISO 12800 file! Will test further though.
Thank you both :-)

Reported by johan@birkagatan.com on 2014-06-19 06:03:22

[Beep6581]

Here a new patch with some enhancements

1) thank you again, Ingo

2) I have introduced, a "combo box" to choose between 3 "median"
* standard 3x3
* soft 3x3 which seeks to preserve the details (a little) and consume a little more
time
* strong 5x5 - only for very noisy images

Used alone (sliders "luminance" and "chrominance master" to zero), time processing
is low, but result not realy good

In combination with FFT and Wavelet, the result seems better...and the processing time
is increase about 4% to 10%

:)

Reported by jdesmis on 2014-06-19 12:01:18

---

• _Attachment: denoise_median-4.patch_

[Beep6581]

you must read "less time" instead of "more time"

Reported by jdesmis on 2014-06-19 12:02:44

[Beep6581]

Jacques, works fine. But processing time for 3x3 increased by factor 2 with your patch.
Will post a new one in a few hours to get old speed back. It's almost ready, but I've
to leave now for some hours.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-19 13:01:35

• Status changed: PatchSubmitted
• Labels added: Type-Enhancement
• Labels removed: Type-Defect

[Beep6581]

Jacques and Ingo
I made some tests with denoise_median-4.patch. Very efficient with 3X3 soft on very
noisy images .  It permits also to strongly decrease denoise parameters. Details seem
not really degraded (but there is not a lot in these desperate photos).
5x5 seems even too strong.
I will wait for Ingo patch to test further.
André

Reported by gaaned92 on 2014-06-19 22:12:41

[Beep6581]

Here's the patch I announced in #8. It also includes a correction for the case that
RGB_denoise is called with src != dst, which actually is not the case, but it's better
to know that this case is handled too.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-06-19 23:09:35

---

• _Attachment: denoise_median-5.patch_

[Beep6581]

Thank you again, Ingo for these improvements :)

Here a new (little differences) patch, with improvements to "3x3 soft"..I think now,
it is better (a little more soft)

Gaaned92: I'll look to add, a 5x5 soft...

Jacques

Reported by jdesmis on 2014-06-20 05:00:22

---

• _Attachment: denoise_median-6.patch_

[Beep6581]

the same with "5x5 soft"

:)

jacques

Reported by jdesmis on 2014-06-20 06:49:09

---

• _Attachment: denoise_median-7.patch_

[Beep6581]

If no comments, I'll commit wednesday 25th.
:)

Reported by jdesmis on 2014-06-23 12:05:14

[Beep6581]

Hi Jacques, I will try to test this patch tonight EST

Reported by michaelezra000 on 2014-06-24 11:31:53

[Beep6581]

Jacques, I could not see what can be done different by the median filter vs an existing
Luminance NR when it is used with low values of the Luminance Detail slider. Am I missing
something?

Reported by michaelezra000 on 2014-06-25 01:57:16

[Beep6581]

Michael :
1) On very noisy images (D3x 6400ISO - D800 12800ISO) - there is a significant difference
in the solids

2) on low noisy images (blue sky) - the median increases performance

To deal with the noise, there are currently three types of functions, FFT, wavelet
and median ...(RT use only the 2 first)

In fact, when I was developing for Dcraw, I made ​​a "denoise" function...(4 years
ago), which used essentially the same methods as in RT, but at the end I added a small
median, which significantly reduces artifacts.

I applied the same thing here.

I think that leaving activate a small median (3x3 soft) generally will increase performance
without compromising the processing time

Reported by jdesmis on 2014-06-25 06:11:05

[Beep6581]

Median removes the little sharp pixels at 100% left over from NR, which maybe won't
make much difference on paper or when downscaled on screen but results in a ~30% smaller
file size as compared to noise reduction without median filtering when saving to JPEG/92/4:2:2
at 100% size.
http://i.imgur.com/OBS0e4f.png
http://i.imgur.com/cbaklNh.png

Reported by entertheyoni on 2014-06-25 07:11:13

[Beep6581]

Jacques, thanks for the explanation. OK to commit then!
Thanks for the new tool:)

Reported by michaelezra000 on 2014-06-25 11:26:03

[Beep6581]

Michael, Thanks

Commit to default :)

Reported by jdesmis on 2014-06-25 11:37:16

[Beep6581]

Some thoughts, lets first discuss before any time is spent implementing :)

1- Would it be possible (and useful?) to add a threshold selector for the median filter
like the one for USM, so that it would target only uniform surfaces (sky, water) and
very high frequency patches (like the sharp pixels left over from NR as in the screenshots
above) but not fine textures (brick wall, leaves)?
http://i.imgur.com/ovQaJEb.png a little noisy
http://i.imgur.com/jy0wyoX.png fine noise reduction, perfect, but I could reduce the
JPEG file's size further if I used median over the sky (and water, not visible in this
screenshot)
http://i.imgur.com/FEfGsss.png but it ruins the brick wall and tiled roof textures

2- The "Median" and "Median method" buttons could be collapsed into one drop-down:
Median: Off, 3x3 soft, 3x3 standard, 5x5 soft, 5x5 standard. So we'd get rid off the
on/off button.

Reported by entertheyoni on 2014-06-25 11:50:55

[Beep6581]

Have you considered placing the median filter before the detail recovery stage?

It seems to me that the artifacts come mostly from the first stage and details removed
by the median filter can be recovered with the recovery stage. EG the brick pattern
and tiled roof in the images from #20.

Reported by iain.fergusson on 2014-07-01 05:22:38

[Beep6581]

Is this new median also meant to be used with demosaicing algorithms that already apply
median filtering (LMMSE) at an earlier stage or is this not recommended?

Reported by stefan.ittner on 2014-07-01 09:11:56

[Beep6581]

DrSlony

#20  1. I think we must wait "local control" ??

DrSlony and Iain : I have add a "Lab" median after wavelet, you can choose between
 "L only" in middle treatment, and "RGB" in final

#22 : no problem: median for LMMSE acts on chroma RGB only, and acts on other things..

:)

Reported by jdesmis on 2014-07-01 11:03:37

---

• _Attachment: denoise_median_plus_8.patch_

[Beep6581]

@#21 iain.fergusson: fighting with the median filter to recover what it took away is
not a good approach, better to suppress it from taking away what one wants kept in
the first place, that's why I suggest a threshold selector so the median filter simply
has no effect at the targeted areas.

@#23 Jacques: As far as I'm concerned the patch can be committed as is without the
threshold selector, but if possible I would like to test whether such a threshold selector
would be useful.
I will try to test patch 8 later today.

Reported by entertheyoni on 2014-07-01 11:17:58

[Beep6581]

@#24 It's not just about recovering details lost by the median filter. In my experience
a median filter placed before an FFT detail recovery pass improves recovery of low-level
details/patterns. This might not be the case for RT's noise reduction, but I thought
it might be worth looking into. I should have explained more in my first post.

Reported by iain.fergusson on 2014-07-01 23:51:14

[Beep6581]

Jacques could you explain what the "Median - Luminance only" checkbox is supposed to
do?
1- How do I turn median filtering off completely?
2- The method options are "3x3 soft" "3x3 standard" "5x5 soft" "5x5". If there is no
difference between the "standard" and "" (nothing) suffixes, then "3x3 standard" should
be renamed to "3x3" only (or "5x5" should be renamed to "5x5 standard").
3- If the "Median - Luminance only" does not turn it off completely, then I think it
would be more intuitive to have this as a drop-down, not checkbox. Checkboxes in my
mind are for "on/off", and drop-downs for choices (other people please comment).
4- I asked for a drop-down in #20 so that there would be only one widget relating to
Median. Now there are two again. So maybe it's better if this is handled like "Sharpening
> Halo control": when disabled, the section is hidden. When enabled, the "amount" widget
appears, so here the "Method" and "Lab/RGB" options would appear.

Reported by entertheyoni on 2014-07-02 12:07:57

[Beep6581]

Hello DrSlony

I do not see how you could try the patch, I made a mistake by sending .. this is not
good. The one sent plant !

Je vais me permettre de répondre en français, pour qu'il n'y ait pas d'incompréhension...Je
rentre de la plage...

Pour empêcher complètement l'utilisation du median, il faut mettre la méthode sur « none »

Maintenant quelques commentaires généraux sur le bruit et les images :

1)les images contenues dans « http://rawtherapee.com/shared/test_images/ » ne sont
pas suffisamment significatives. Lorsque j'ai développé le module « denoise » pour
ma version de Dcraw (2008...2011), et lorsque Emil a développé celui de RT, nous nous
sommes servis d'images sur « Image ressources », dont :
*D3x 6400 ISO
*D700 25600 ISO
*on pourrait prendre aussi D800 12800 ISO
Ces images contiennent des aplats, des détails, des variations de textures dans chaque
couleur difficile, notamment les rouges

2) le bruit n'est pas un phénomène simple, il dépend de nombreux paramètres :
*prise de vue, intensité de la lumière, nature de la lumière, etc.
*caractéristiques de l'exposition, temps de pose
*type de capteur ; marque, âge de conception
*traitement par la camera
*demoisaicing 
*puis bien sûr le traitement du bruit, qui selon son emplacement (il est maintenant
tout au debut du processus), donnera des résultats différents
*puis bien sûr le processus qui amplifiera ou non le bruit (traitement des ombres,
accentuation...) 
etc.

Face à cela pour traiter le bruit on dispose de 3 type de traitements (il en existe
d'autres mais je ne sais pas les programmer!)
*wavelets – le principal
*complété par FFT ou median
Dans la version précédente il n'y avait pas de median..
j'en ai ajouté un car, il y a des images, et des niveaux de bruit pour lesquels le
median sera supérieur à FFT, sans bien sûr pouvoir le prédire...

Maintenant quelles sont ces caractéristiques :

Jusque ce patch, il n'y avait en fin de traitement un median « rgb » qui agit successivement
sur chaque canal « r », « g », puis « b ». Il agit sur les 2 composantes du bruit « luma »
et « chroma » et à 4 niveaux : soft , 3x3, 5x5 soft, 5x5

J'ai ajouté un median sur la luminance en mode « Lab » , qui n'agit que sur « L » et
n'a donc pas d'action sur le bruit chroma. Il a aussi 4 niveaux « soft, 3x3, 5x5 soft,
5x5

Si maintenant vous essayez sur par exemple l'image "D3x 6400 ISO", vous verrez d'importantes
différences selon ou non l'utilisation du type de median « Luminance » ou « RGB » et
selon sa puissance (soft ==> 5x5), et selon l'endroit (texture, caractères, etc.)

Bien sûr je peux changer l'interface GUI :
*en regroupant tout dans une même « combo box », mais cela ne me semble pas pertinent
*je peux aussi ajouter une fonction en plus au median, en ajoutant un median « ab »
(chroma) en mode Lab....mais cela me semble compliqué
*je peux aussi supprimer un des median (par exemple le RGB)

Autre chose, à propos du curseur similaire à celui utilisé dans "sharpening". Dans
sharpening les fonctions gauss utilisées sont continues. Ici le median est une fonction
discrète, on ne peut pas écrire f(3x3) = k * f(5x5)...Bien sûr on peut le programmer,
mais les résultats seront fantaisistes.

Cordialement

Maintenant le patch correct..J'espère ne pas me tromper à nouveau

Reported by jdesmis on 2014-07-02 17:01:13

---

• _Attachment: denoise_median_plus_9.patch_

[Beep6581]

To exhaust all possibilities. I can also replace the "checkbox" with a "combobox".
This "combobox" w'll have 4 choices: None, Luminance only, Lab, RGB (Lab remains to
be done) ["Luminance only" and "Lab" are inserted in threatment just after wavelet,
"RGB" at the end] 
The other combobox would have the choice "3x3soft, 3x3, 5x5soft, 5x5".

This would allow (on a series of images) to tests and determine:
1) What method(s) to choose between "L only", "RGB" and "Lab"
2) What are the choices for the type of median: 3x3soft, 3x3, 5x5soft, 5x5

Then I would remove everything that is superfluous.

:)

Reported by jdesmis on 2014-07-03 05:49:26

[Beep6581]

Here a patch with "combobox" for "method median" : none, L only, Lab, RGB

None = no median

And "old combobox" with "median type": 3x3soft, 3x3 , 5x5 soft, 5x5

We can test with some images:
eg :
1) amsterdam_moving_boat_1.pef ==> for low noise

2) D3x 6400 ISO (image ressource), "with bottles of vinegar and oil" = for high noise
and textures

3) another...??

And after, I suppressed if neccesary !

:)

Reported by jdesmis on 2014-07-03 08:58:08

---

• _Attachment: denoise_median_plus_10.patch_

[Beep6581]

Just a remark on low noisy images (amsterdam), compare:
* With "Luminance=20" "Detail=50"
* And "Luminance=0" with "Median method="Luminance only" and 3x3.

Substantially the same result (in my opinion), but the treatment time decreased from
4 seconds to 3 seconds

Reported by jdesmis on 2014-07-03 11:03:12

[Beep6581]

The same patch, but with time optimization...if user select only "median"...[slider
luminance =0 ; slider Chrominance master=0]

With same settings as #30, but with chroma=0....time decrease from 3 seconds to 1 second.
This time "Lonly" and "lab" is superior to time RGB, because need rgb<=>Lab conversion
!

I hope no error

Reported by jdesmis on 2014-07-03 16:22:03

---

• _Attachment: denoise_median_plus_11.patch_

[Beep6581]

Here a new patch - just before "1 mounth holidays" with my 2 grandchildren son:)

I limit action of "RGB" to 5x5soft (3x3 soft, 3x3, 5x5 soft)

I increase possible action in Lab mode (L only and Lab) to median 7x7 (3x3 soft, 3x3,
5x5soft, 5x5, 7x7). This median is slow, but in some case (rare), it can add effect,
and not suppress texture as "wavelet"

Reported by jdesmis on 2014-07-05 12:01:00

---

• _Attachment: denoise_median_plus_12.patch_

[Beep6581]

valgrind --leak-check=yes --track-origins=yes --error-limit=no --xml=yes --xml-file=rt_2423median12.xml
./rt_default_Debug_rt_2423median12/rawtherapee

From the File Browser I right-clicked on amsterdam_moving_boat_1.pef, applied Neutral,
opened it, enabled Noise Reduction with Median set to 3x3 soft, zoomed to 100% and
clicked through all the median methods: luminance only, Lab, RGB. Quit.
http://filebin.net/code9anubt/rt_2423median12.tar.bz2

Reported by entertheyoni on 2014-07-06 09:25:28

[Beep6581]

I checked the valgrind log. I didn't find anything relevant which is caused by the patches
from this issue.

Ingo

Reported by heckflosse@i-weyrich.de on 2014-07-06 11:08:32

[Beep6581]

Please give me a day (tomorrow) to check the patch again before committing, so I can
revise the label names and such.

Reported by entertheyoni on 2014-07-06 11:36:57

[Beep6581]

DrSlony
Another enhancement that should not change a lot for the stability of the code.

I introduce "iterations" for median (1 , 2 , 3) : Often, it is better to do iterations,
with small median, than one with big !

:)

Reported by jdesmis on 2014-07-06 15:57:51

---

• _Attachment: denoise_median_plus_13.patch_

[Beep6581]

Hi Jacques,
I just noticed that median leaves a very thin border untouched.
in patch 13 5x5-soft has action that is similar to what 5x5 was previously.
If you try on artificial test image with sharp 90degrees corner, the type 3 blurs the
corner, while 3x3 soft does not. with patch 13, type 5x5-soft also blurs the corner,
while previously it did not. May be a small bug here, typo or something?

About the method names.. the result of 3x3 is softer (more blurry) than 3x3-soft:)
It could be more intuitive to the user if methods are called in relation to the end
result. May be call it "3x3" and "3x3 strong", etc.?

Reported by michaelezra000 on 2014-07-07 11:46:21

[Beep6581]

The method types vary based on the Method - when RGB is selected 5, 7, 7soft disappear.
Also, I probably was confused by this and my remark in comment 37 about softness of
5x5soft may be incorrect (sorry:))

Reported by michaelezra000 on 2014-07-07 11:51:35

[Beep6581]

Hello Michael

The borders are never treated ! too complicated...and for me, not important

But there are differences :
3x3 soft : 5 pixels are treated (range 1)
3x3 : 9 pixels (range 1)
5x5 soft : 13 pixels (range 2)
5x5 : 25 pixels (range 2)
7x7 = 49 pixels (range 3)

For the label ! ? ! it is what you want. I call "soft" because it is less agressive,
but 3x3 leads to result more agressive (or soft or blurry...)

:)

Reported by jdesmis on 2014-07-07 11:54:41

[Beep6581]

I made changes for labels 3x3 strong and 5x5 strong

Then I commit...

I would be very little available for 1 month

:)

Reported by jdesmis on 2014-07-07 18:22:32

[Beep6581]

Jacques, will you commit anything the next days? Please tell me before I have to rework
the x-trans patch once again...........

Reported by heckflosse@i-weyrich.de on 2014-07-07 20:36:29

[Beep6581]

Ingo

No, nothing planned. I understand the "new" work !

:)

Reported by jdesmis on 2014-07-08 04:59:08

[Beep6581]

Jacques I hope you're still here :]

When I change "Median Method" I see "NR - median type Lab" in history.
Should be "NR Median - Method"

When I change "Median type" I see "NR - median method" in history.
Should be "NR Median - Type"

When I change "Iterations" I see "NR - median type RGB".
Should be "NR Median - Iterations"

Also I would change the label of "Iterations" to "Median Iterations" to make it clear.

Reported by entertheyoni on 2014-07-08 09:32:09

[Beep6581]

I'll fix that

Reported by heckflosse@i-weyrich.de on 2014-07-08 10:02:10

[Beep6581]

Committed, and the remaining things are handled in issue 2014 #314.
Need to be documented before we can close.

Reported by entertheyoni on 2014-07-09 12:18:32

• Status changed: NeedsDocumentation

[Beep6581]

I am finding the median stage useful but some artifacts appear when using L or Lab median.
See attached image and raw file.

I'm using:

Branch: default
Version: 4.1.38
Changeset: 72e1ff9f3153
Compiler: gcc 4.8.1
Processor: generic x86
System: Windows
Bit depth: 64 bits
Gtkmm: V2.22.0
Build type: Release
Build flags: -m64 -mwin32 -mthreads -msse -msse2 -mtune=generic -fopenmp -Werror=unknown-pragmas
-mwindows -DNDEBUG -O3 -Wno-aggressive-loop-optimizations
Link flags: -mwin32 -mthreads -static-libgcc  -mtune=generic -mwindows -s -O3
OpenMP support: ON
MMAP support: ON

Reported by iain.fergusson on 2014-07-13 05:05:58

[Beep6581]

Please don't attach any files here over 50kB Iain.

Reported by entertheyoni on 2014-07-16 17:56:39

[Beep6581]

My apologies for the large attachments. I have deleted them and added links below instead.

https://dl.dropboxusercontent.com/u/10782279/median%20noise%20artefact.jpg

https://dl.dropboxusercontent.com/u/10782279/DSC_5476.NEF

Reported by iain.fergusson on 2014-07-16 20:46:44

[Beep6581]

Reported by entertheyoni on 2014-07-18 11:15:46

• Blocking: #2447

[Beep6581]

#48

I will discuss the problem, but I think that - even without "median" - image already
present artifacts due to noise reduction.
In addition, it is known - no solution - the medians high values ​​(5x5 7x7 9x9 11x11...)
bring artifacts on the edges

:)

Reported by jdesmis on 2014-07-20 09:20:37