search

timothybrooks / hdr-plus

HDR+ Implementation
MIT License
587 stars 200 forks source link

Implement Handheld Multi-Frame Super-Resolution #40

Open brotherofken opened 5 years ago

brotherofken commented 5 years ago

Google researches published paper with details of implementation of their superresolution: https://arxiv.org/abs/1905.03277

It seems not trivial but implementable. Algorithm introduces new approach to merge and produces demosaiced image, so requires large changes in pipeline.

Development might be split into stages:

  1. Alignment refinement for sub-pixel accuracy using Lukas-Kanade optical flow iterations.
  2. Merge without considering robustness term
  3. Add robustness estimation.
JVision commented 4 years ago

Hi Brotherofken, I am working on an open source implementation https://github.com/JVision/Handheld-Multi-Frame-Super-Resolution of the paper.

would appreciate if anyone could join in with me. At very early stage though, please check it out at. @brotherofken

chencuber commented 4 years ago

Google researches published paper with details of implementation of their superresolution: https://arxiv.org/abs/1905.03277

It seems not trivial but implementable. Algorithm introduces new approach to merge and produces demosaiced image, so requires large changes in pipeline.

Development might be split into stages:

  1. Alignment refinement for sub-pixel accuracy using Lukas-Kanade optical flow iterations.
  2. Merge without considering robustness term
  3. Add robustness estimation.

Hi Brotherofken, in Fig.8 of the paper, it seems lamda1/lamda2 is in range of 0~1, however, we know lamda1 is the dominant eigenvalue and should be greater than lamda2. I am confused, do you know why is that?

JVision commented 4 years ago

The reason is simple. there are errors in the published work. The author means (lamda1-lamda2)/(lambda1+lambda2)

chencuber commented 4 years ago

The reason is simple. there are errors in the published work. The author means (lamda1-lamda2)/(lambda1+lambda2)

I think you are right, that form makes sense.

SuTanTank commented 4 years ago

Is it possible that lambda2/lambda1 is the correct one? Seems also fit the (0~1) range

Update: I was wrong, this value should should go up when the pixel is more likely on an edge, i.e. lambda1 >> lambda2.

brotherofken commented 4 years ago

@SuTanTank Thanks! That totally make sense. Look at the end of a section 2.2 in Anisotropic Diffusionin Image Processing. Axis description of Figure 8 says "Presence of an edge" (not coherence) and the book says that mu_1 >> mu_2 characterizes straight edges.