IGV and JDL interpolation for noisy images

[Beep6581]

Originally reported on Google Code with ID 1741

Here is a new patch ... which originates from the discussion:
http://www.rawtherapee.com/forum/viewtopic.php?f=2&t=4459

Thank you to Luis Sanz for updated code JDD (dcraw) :)

This new interpolation is recommended only in the case of noisy images.

Would not it be interesting to remove one (or two) interpolation(s) to avoid the multitude
of choices?

:)

Reported by jdesmis on 2013-02-28 18:34:36

---

• _Attachment: Igv_interpolate.patch_

[Beep6581]

Sounds interesting. Will give it a try tomorrow.

Reported by heckflosse@i-weyrich.de on 2013-02-28 20:01:54

[Beep6581]

Jacques, I see that you also have jdd in the patch, I am curious, what are expected
differences?

I don't nearly use all demosaic methods, but may be other users do... I suppose instead
of removing, we could rearrange the list by giving higher priorities to methods which
are likely used more frequently. 
Also, we should publishcomparison of diffrent demosaic options in rt and other programs.
I will try to get to it when I get back home next week.

Reported by michaelezra000 on 2013-02-28 20:14:12

[Beep6581]

I have to learn compiling RT for w32 ... :(

Jacques or Luis

Is it possible to merge IGV and amaze in the same file adaptively, use IGV for underexposed
areas and Amaze for the rest ??.

Transition could be calculated depending on ISO (exif) + sensor size (user input) +
a correction factor for adapting to old noisy CCD sensors or new and future with much
lower noise.  

Reported by iliasgiarimis on 2013-02-28 23:08:06

[Beep6581]

Here is the patch with:
1) IGV
2) JDL (I renamed JDD in JDL with "L" for Luis or Lassus)
3) change 2 pp3 "high/medium ISO"
4) change the order of interpolation: 1)Amaze 2)IGV 3)etc..

@ Ilias:
Of course it is always possible to mix, this is what has been done for Amaze (by Emil
and Luis), which is a very good compromise ... But you can not have everything.
I think it is better to have two interpolations: a) Amaze for the general case is probably
one of the best interpolations b) IGV for noisy images

:)

Reported by jdesmis on 2013-03-01 10:11:21

---

• _Attachment: igv_jdl_interp.patch_

[Beep6581]

Here is a quick comparison amaze-ivg-jdl
http://www.michaelezra.com/Projects/RT/images_shared/demosaic/demosaic_amaze-ivg-jdl_01.jpg

ivg and jdl images feel crisper but seem to lack antialiasing, jdl less so.

Reported by michaelezra000 on 2013-03-01 20:26:03

[Beep6581]

It's not so much anti-aliasing as 'zipper' artifacts (that's why the last part of AMaZE
stands for 'zipper elimination'.  It arises when the estimate of B-G or R-G is inaccurate;
many interpolation methods first estimate these color differences, then use the existing
Bayer raw data to calculate the individual R, G, B channels.  But suppose the |R-G|
estimate is too small or too large; then the G value inferred from the R sites will
differ from the G value actually measured at G sites in the Bayer array.  It's even
more noticeable in the R and B output; typically, G is interpolated first, then the
estimated R-G is added to it to infer R at sites where R is not measured data, but
the measured R is taken where it is actually observed.  Thus if the estimated R-G is
off, three sites out of four will have a wrong value of R (similarly for B), and this
is what you see in the output.  

The price paid for the smoother color channels in AMaZE is somewhat of a loss in detail,
partiularly at low contrast levels.  Maybe I could come back to this if speedups in
AMaZE processing are made possible by SSE and GPU coding, because I suspect that addressing
this issue would make AMaZE even slower.

Reported by ejm.60657 on 2013-03-01 22:25:00

[Beep6581]

Emil, it's always a pleasure, to read your explanations! Great!
Don't wait for speedups of AMaZE, let's work together. Make your mods, and I'll try
to speed up (using SIMD now and OpenCL later).

Ingo

Reported by heckflosse@i-weyrich.de on 2013-03-01 22:54:16

[Beep6581]

When time comes, for ease of comparisons within RT, could it be amaze-wip (work in progress,
or any other technical name - Amaze2 :)), to allow to switch between versions on the
same image and immediately observe the differences?

If the next version of amaze will resolve skin texture better, this would be Amazing!

Reported by michaelezra000 on 2013-03-01 23:20:01

[Beep6581]

You really want to be able to resolve all the pores and facial hair on your models?
  The women will love you  ;^)

Reported by ejm.60657 on 2013-03-01 23:33:23

[Beep6581]

Yes! :) except the facial hair - we will need a checkbox to avoid it:-D

Reported by michaelezra000 on 2013-03-02 00:41:24

[Beep6581]

:D

Reported by gyurko.david@e-arc.hu on 2013-03-02 11:57:17

[Beep6581]

The same patch after update default  :)

Reported by jdesmis on 2013-03-03 06:59:24

---

• _Attachment: igv_jdl_inter1.patch_

[Beep6581]

A quick comparison between Amaze and JDL on noisy image

Image: D3X_6400.ISO  (http://www.imaging-resource.com)
 
For luminance: Noise reduction ==> method=RGB luminance=0 chroma=50 gamma=1.7

http://jacques.desmis.perso.neuf.fr/RT/lum_AMA_JDL.jpg

For Chroma: Noise reduction ==> method=RGB luminance=50 chroma=0 gamma = 1.7
http://jacques.desmis.perso.neuf.fr/RT/chrom_AMA_JDL.jpg
:)

Reported by jdesmis on 2013-03-03 08:50:33

[Beep6581]

IGV with border interpolate  !

Reported by jdesmis on 2013-03-03 10:55:00

---

• _Attachment: igv_jdl_inter2.patch_

[Beep6581]

Hi Jacques, in the preview both IGV and JDL show small white squares

Reported by michaelezra000 on 2013-03-03 15:02:00

[Beep6581]

IGV with border interpolate "chroma".

Note:
It is not perfect (as JDL).. but we are with noisy images :)

Reported by jdesmis on 2013-03-03 17:08:46

---

• _Attachment: igv_jdl_inter3.patch_

[Beep6581]

This is a bit better, but now spots are colored:)

Reported by michaelezra000 on 2013-03-04 00:07:58

---

• _Attachment: demosaic_amaze-ivg-jdl_02.jpg_

[Beep6581]

This does not happen with all images, so may be it is specific to image dimensions.
This illustration is based on Canon 60D cr2.

Reported by michaelezra000 on 2013-03-04 00:10:30

[Beep6581]

Tested on raws from a few cameras but mostly from my Pentax K10D. Though very similar,
JDL seems a bit better all round - less noise (or the noise seems more finely broken
up) and less zippy dots.

Reported by entertheyoni on 2013-03-04 04:05:53

• Labels added: Type-Enhancement, Component-Quality
• Labels removed: Type-Defect

[Beep6581]

the same patch with optimization (less memory, more compact code), but similar results
:)

Reported by jdesmis on 2013-03-05 06:39:08

---

• _Attachment: igv_jdl_inter4.patch_

[Beep6581]

Works fine here :]
... but!!!!!!
Open your raw photo, let's say you're using AMaZE, you zoom to 100%, you turn Noise
Reduction on, you switch from AMaZE to IGV, you think you're seeing how well it performs...
You're not!
Take a screenshot (lossless), turn Noise Reduction off, turn it on again... Different!
Take another screenshot and flip between the two. Quite a big difference, especially
when comparing noise reduction/demosaicing algos!

See issue 1740.

Reported by entertheyoni on 2013-03-05 22:26:10

[Beep6581]

I made the change (LIM. ..) proposed by Luis.
I also limited the output values ​​to 0 .. 65535, although RT works with real
:)

Reported by jdesmis on 2013-03-06 07:25:55

---

• _Attachment: igv_jdl_inter5.patch_

[Beep6581]

here a patch with :
* IGV
* LMMSE 

I removed JDL that provides no more than IGV  (with more artifacts in highlights for
JDL)
LMSSE is slow despite OpenMP
:)

Reported by jdesmis on 2013-03-07 10:08:43

---

• _Attachment: igv_lmmse_interp.patch_

[Beep6581]

The same patch, but with update "default" and tags.

For my part we can add two interpolations (IGV and LMMSE), they all bring something
in noisy images. Should we remove interpolations in the list (ahd?)

The ranking of interpolations is based on his own evaluation criteria:
* Processing time
* Bad pixels
* False colors
* Ability to reproduce the details
* Moire (Nyquist)
* Ability to restore the diagonals
* Noise attenuation
* Signal to noise ratio (PSNR)
* chromatic aberration
* mazes
* DeltaE: colors respect
* edges
* Etc..

:)

Reported by jdesmis on 2013-03-08 09:13:06

---

• _Attachment: igv_lmmse_interp1.patch_

[Beep6581]

you can download a test file with 28 images  :)
http://filebin.net/5dctq1oxr0

Reported by jdesmis on 2013-03-08 09:56:30

[Beep6581]

Awesome :)

LMMSE seems to interpolate the border and highlights very well.

But it appears LMMSE crashes on some files, here's an example:

http://www.imaging-resource.com/PRODS/D90/D90hSLI6400.NEF.HTM

Reported by woznicamaciek on 2013-03-08 11:46:46

[Beep6581]

this should solve the problem??

:)

Reported by jdesmis on 2013-03-08 12:39:26

---

• _Attachment: igv_lmmse_interp2.patch_

[Beep6581]

It worked for the D90 files :)

Here's another one that crashes, Olympus E-PL5, though it's a low ISO shot i just thought
it might prove useful to you.

http://filebin.net/y5xwnufp4w

Reported by woznicamaciek on 2013-03-08 13:29:27

[Beep6581]

this should also solve the problem !
:)

Reported by jdesmis on 2013-03-08 15:16:05

---

• _Attachment: igv_lmmse_interp3.patch_

[Beep6581]

Nice, that solved it :)

Reported by woznicamaciek on 2013-03-08 20:00:54

[Beep6581]

if there are no further comments or modifications, I will commit tomorrow
:)

Reported by jdesmis on 2013-03-09 06:31:26

[Beep6581]

Please let me re-indent your code and check it. I'll need one hour only and will post
it back here, thanks.

Reported by natureh.510 on 2013-03-09 11:45:06

[Beep6581]

Hello Hombre

No problem...my code "est toujours égal à lui même" :)

Reported by jdesmis on 2013-03-09 12:47:28

[Beep6581]

Jacques both algorithms seem stable so far :) but i think there could be complaints
against IGV because of it's inability to properly demosaic the border, maybe the border
could be auto cropped? 
Otherwise it could be easily noticeable on big prints.

Reported by woznicamaciek on 2013-03-09 21:52:40

---

• _Attachment: 2013-03-09_22-51-22.jpg_

[Beep6581]

With the current code (and my images) I see nothing on the edges. However for the borders,
interpolation is "bilinear" and the noise is not treated. So in some cases it will
appear colored pixels ...

I will commit in the moments to come.

A big thank to Hombre has optimized the code and interface including other interpolations.

LMMSE works, but I disabled OMP, because in some (rare) cases it resulted in partially
non interpolated images. It takes about 6 seconds (on my system) for a D800 file, which
is acceptable (compared to amaze about 3 seconds)

More LMMSE consuming 2.8GB memory for D800 (compared to 3.9Gb consumed by Denoise).

In the coming time you may need to consider "OMP" and "Tiles"

:)

Reported by jdesmis on 2013-03-10 08:05:28

[Beep6581]

woznicamaciek please submit a raw file which shows the border problems.

Reported by entertheyoni on 2013-03-10 19:44:05

[Beep6581]

On this one for example it's very noticeable 

http://www.imaging-resource.com/PRODS/olympus-epl5/EPL5INBI12800.ORF.HTM

Reported by woznicamaciek on 2013-03-10 19:57:12

[Beep6581]

I have done a quick patch  :)

Reported by jdesmis on 2013-03-10 21:05:28

---

• _Attachment: igv_border.patch_

[Beep6581]

I would rather qualify that as "dirty hack", sorry... There should be a way to avoid
demosaicing the image twice, shouldn't there?

Reported by natureh.510 on 2013-03-10 21:19:38

[Beep6581]

the same with only borders :)

Reported by jdesmis on 2013-03-11 08:31:50

---

• _Attachment: igv_border1.patch_

[Beep6581]

The same with code simplification  :)

Reported by jdesmis on 2013-03-11 17:00:47

---

• _Attachment: igv_border2.patch_

[Beep6581]

Here is a patch that combines the changes:
* for the borders (IGV)
* CLIP  raw data
* Other modifications to reduce artifacts in IGV
:)

Reported by jdesmis on 2013-03-12 17:01:38

---

• _Attachment: igv_lmmse_border2_clipraw.patch_

[Beep6581]

see comment issue1771

Reported by jdesmis on 2013-03-13 10:53:36

[Beep6581]

commit to default

Reported by jdesmis on 2013-03-14 08:16:17

[Beep6581]

Hi guys, thank you for your great work on image denoising.
Tried all algorithms that rawtherapee alows on D800 iso 6400, iso 12800 and none of
them is ideal quality wise. 
IMHO better denoiser before demosaic is needed.
Have you been thinking of neural network that can learn noise characteristics and then
apply them? Image pairs: RAW high iso (not demosaiced data) & low iso demosaiced data.
So the network will do both tasks: denoise & demosaic.
Another question is if i as end-user can implement any patch you are providing here.
thank you!
Tomas

Reported by tomas.hejnal on 2013-12-16 13:49:34

[Beep6581]

Aha .. Tomas you mean like Tritical's nnedi 2X resampler for avisynth ?. It's the best
to my eyes ..

Do you know anyone else who developed nn algorithm which works ?.

Reported by iliasgiarimis on 2013-12-16 17:53:52

[Beep6581]

Hi Tomas
This question is best for the forum ("Ideas, Feature Requests").
Anyone can apply patches if they know how to compile code. Instructions are in COMPILE.txt.
The IGV and LMMSE demosaicing methods from this issue have already been committed in
March and are available in all RT builds since then.

I'm closing this issue, please discuss your idea on the forum.

Reported by entertheyoni on 2013-12-16 19:48:09

• Status changed: Fixed