martin-marek / hdr-plus-swift

📸Night mode on any camera. Based on HDR+.
https://burst.photo
GNU General Public License v3.0
198 stars 9 forks source link

Banding Noise Model #18

Open Alex-Vasile opened 1 year ago

Alex-Vasile commented 1 year ago

The original HDR+ paper assumes that the sensor noise at each pixel is only dependent on the intensity of the pixel and that the noise for each pixel is independent of that from any other pixel.

This is more true of newest, but many "older" cameras suffer from varying degrees of banding noise which results in the noise being in very distinct lines. This type of noise is clearly not independent at each pixel value, and is perceptually uglier than regular noise random noise.

While this app works great for newer cameras I think it's even more important for older ones, and I imagine that handling banding would improve the performance of the app with these older models.

chris-rank commented 1 year ago

Do you have an example image how the banding noise looks like? Does it occur at equal pixel positions in each burst image? Does the current app already improve the banding noise?

Alex-Vasile commented 1 year ago

First a correction to my original post, newer cameras also have this issue, though they have to be pushed further to see these problems. Apparently comes up in astrophotography.

I don't have any of my own raws that purposefully show it, but I'll see about capturing some with an older camera.

Here's a link with a discussion about the canon R5 displaying these issues: https://www.dpreview.com/forums/thread/4523050

And a shot directly from the above form post.

The horizontal striping pattern is clearly visible, even at the preview size here.

As for the other questions:

  1. I don't know if the pixel positions are equal, will have to gather my own raws to answer.
  2. I haven't tested explicitly yet, though I imagine it does improve it (I was more raising the issue since taking this into account may solve it even further)
Alex-Vasile commented 1 year ago

@chris-rank To answer the second part, the app does not handle banding noise well.

It does a good job removing the random parts of the noise, but it leaves the banding mostly untouched. If anything the banding is more pronounced because it's all that's left.

Here's a dark frame off my ancient 350D. All shots are at iso 1600 and exposure increased 4 stops in Capture One. I turned off all noise cancelling: dark frame - 1600 raw

Same region after a 10 burst stack (using the frequency branch) with NR=24: dark frame - 1600 nr=24

Same region after a 10 burst stack (using the frequency branch) with NR=25 (notice that it actually performs worse: dark frame - 1600 nr=25

Finally, the same region but merged with the current published version of the app, using the minimum robustness: dark frame - 1600 spatial

I don't know why the NR=25 performs worse than NR=24 but it must have something to do with how NR=25 is a special case and how

This does come up in real photos. Here's a crop of a photo (of a photo) taken again at iso 1600 and underexposed 2 stops. I then increased the exposure in Capture One by 2 stops. (again no noise cancelling): 1600 raw

Then the same region with same settings but after a 9 stack burst at NR=24: 1600 stacked

Notice the clearly visible lines in the stacked image, even if the noise is lower overall. Even after applying heavy spatial noise reduction in Capture One the banding still remains. Also, when taking the car photos as iso 100 (again underexposing and raising it in post) instead of seeing horizontal lines in the final stack I see both horizontal and vertical lines.

PythonNut commented 1 year ago

Does this image from the gallery also exhibit banding noise?

jozef from burst.photo gallery

Alex-Vasile commented 1 year ago

Does this image from the gallery also exhibit banding noise?

jozef from burst.photo gallery

I'm inclined to say yes. The reason for the noise may not be the same, since I haven't looked into the reasons for the banding in the first place, but the noise in the image has a pretty clear vertical pattern in the noise (both before and after noise reduction).