Pattern noise reduction

[Beep6581]

Originally reported on Google Code with ID 2677

https://imgur.com/OMo4Q5q

In the image above you may see dark strokes on the image which are 1px wide and up
to 5px long. There are no similar light strokes.

For whatever reason, I see this rectangular noise pattern in images from many cameras.

I never use luminance noise reduction. If I use noise reduction I just turn chrominance
noise reduction to 100. However, these patterns remain visible and become even more
visible after chrominance is denoised.

I have a suggestion.
1) Program noise reduction algorithm aimed exactly on these rectangular (vertical and
horizontal) patterns. It should be switched independently from luminance noise reduction,
maybe have its own slider.
2) May be program a profiling tool which will find these patterns on a sample image
and will adjust denoising accordingly.

This kind of NR should be non-destroying for sharpness and will still contribute to
image quality, at the very least - to the noise uniformity. As you see these dark strokes
affect midtones as well.

Reported by pinhuer on 2015-02-19 19:51:42

[Beep6581]

After chroma noise reduction these rectangular patterns are the only thing which makes
me hate the noise.

Reported by pinhuer on 2015-02-19 19:54:31

[Beep6581]

After chroma NR: https://imgur.com/3OGOwWG .

Reported by pinhuer on 2015-02-19 19:55:43

[Beep6581]

May be there are other noise patterns as well. This suggestion may be generalized more:
program a noise uniformity improver.

Reported by pinhuer on 2015-02-19 19:58:25

[Beep6581]

I don't see the pattern noise, if by "pattern" we mean something that is repeated.
Also, in your exemple image, I think the shadows are a bit overstretched, increasing
the noise.

Reported by sguyader on 2015-02-19 20:23:52

[Beep6581]

DrSlony suggested in IRC that one can use median filter for that. I tested it and it
aids the issue which I am discussing a lot.

However, there might still be an improvement to this.
1)The grain which is remaining after median filter leaves dark blotches of pixels intact
and the noise still appears more blotchy than the film noise. In my opinion if there
is a way of making the grain finer it should be explored.
2) Chroma noise reduction reduces saturation of uniform areas significantly. Rather
than reducing the noise "down" to the floor, it reduces the noise "up" towards the
noise ceiling. https://imgur.com/a/PU3Jz - see, the dark areas of green channel view
become lighter meaning that the saturation is dropped. Is there any rationale for this?
It is obvious that for fine detail it cannot be different but how it works for uniform
areas is questionable.

Also, samples: http://www.dpreview.com/reviews/nikon-d750/13 - you may select any camera
and download any RAW file. The mentioned sample is from RAW from D7100 at ISO3200.

>I don't see the pattern noise, if by "pattern" we mean something that is repeated.

One may strecth the meaning of "pattern" to include any other form of predictability
present in the noise. Would it be correct?

I do.

>Also, in your example image, I think the shadows are a bit overstretched, increasing
the noise.

As long as there is distinguishable information left they are not overstretched in
my own terms.

Reported by pinhuer on 2015-02-19 21:01:58

[Beep6581]

And 3.
3) https://imgur.com/a/SMs9e - nomedian vs weakest 3x3 (which leaves some noise defects
in other areas).
I cannot say that median filter leaves details intact. It smoothes fine details significantly
and I am almost sure in that there should be much more efficient way of reducing discussed
kind of noise.

So no, median filter is not a quality solution.

Reported by pinhuer on 2015-02-19 21:10:18

[Beep6581]

Have you tried using LMMSE or IGV demosaicing?

Reported by iain.fergusson on 2015-02-19 23:07:49

[Beep6581]

#8: yes, I surely did. 

I am using an ISO3200 RAW from D7100 as an example: AMAzE, IGV and LMMSE have equal
impact on the discussed pattern (not as in "repeating pattern").

AMAzE also did somewhat worse in super-noisy areas in my expirience but only in SUPER
noisy.

___________________

I also possibly overestimated the impact of denoising on the saturated areas, after
a fresh look I cannot fund an issue with it.

___________________

I still think that there is an efficient way of dealing with this noise non-uniformity
and that making noise uniform is as valuable as denoising.

Reported by pinhuer on 2015-02-20 14:26:27

[Beep6581]

Hi pinhuer,

Which exactly is the source file of your sample, is it from DSC_6591.nef or DSC_6600.nef
(the low light ISO3200 under tungsten illumination) ?

Is your sample a screen capture from RT's edit window or is it from a exported 8bit
jpeg or 8/16bit tiff ?.

I have something in mind about these files (dithering before demosaic ..) but no time
and data to come to a valid proposal at the moment .. 

Reported by iliasgiarimis on 2015-02-20 15:38:10

[Beep6581]

It is DSC_6591.nef

It is a screencap of RT window at 200% or greater. Everything is left neutral.

Exports feature same artifacts.

My clue is that there are no practical restrictions against doing it after demosaicing.
I even think that demosaicing makes these easier to recognize.

Reported by pinhuer on 2015-02-20 16:06:42

[Beep6581]

I cannot see any pattern noise in the examples above... could you illustrate this better
or exaggerate visibility of the pattern with additional processing?

Reported by michaelezra000 on 2015-02-20 16:11:23

[Beep6581]

Sure. Here is a 1000% of noise with overlined lines: https://imgur.com/jvJlLij

Here is the same but amplified and without marks: https://imgur.com/bHyn9rr

As I said, it is not pattern as in "repeating pattern" but pattern as a specifiable
behaviour. You will never see this kind of noise on the good scan of the film.

It looks bad.

Reported by pinhuer on 2015-02-20 16:46:34

[Beep6581]

http://filebin.net/2zof5k0cy4 - direct download.

Reported by pinhuer on 2015-02-20 16:52:06

[Beep6581]

Also, to make it clear that it is not a debayerization problem: "fast" method leaves
those linear traces as well.

Reported by pinhuer on 2015-02-21 20:50:38

[Beep6581]

https://www.flickr.com/photos/castorscan/9693023737/sizes/o/in/photostream/ - here is
an example of the grain which does not posess any patterns.

Reported by pinhuer on 2015-03-08 15:02:44