search

jcelaya / hdrmerge

HDR exposure merging
http://jcelaya.github.io/hdrmerge/
Other
363 stars 78 forks source link

Alignment issue (?) #61

Closed assaft closed 9 years ago

assaft commented 9 years ago

I'm getting some weird artifacts when I'm merging the attached 5 handheld exposures (1ev apart). I haven't encountered such issues (with handheld bracketing) so far so I'm a bit surprised to see these artifacts. Maybe it has to do with the fact that the pictures were taken using a fisheye lens or maybe something else is going on? I don't know if this is a bug or just an unavoidable result when the auto aligner fails (if alignment is the issue at all).

In the attached file there are raw and jpeg files. The raw files are before and after merging (default settings). The jpegs are crops of the relevant area in which I see the said artifacts.

https://dl.dropboxusercontent.com/u/21642925/Alignment.ZIP

Thanks for taking a look.

jcelaya commented 9 years ago

The alignment algorithm of HDRMerge is a bit limited, because it can only fix small horizontal/vertical translations. It must be this way, since we are working in RAW space. More complex alignments, like rotations or perspective corrections, must be done after the interpolation. In this case, the fisheye lense may be killing the alignment algorithm due to its deep perspective deformation. It doesn't look like a bug, just a limitation. In general, try to shot with a tripod, or be very careful if you shot handheld.

assaft commented 9 years ago

PhotoAcute, which also works in RAW, has managed to align this set without such artifacts: https://dl.dropboxusercontent.com/u/21642925/Alignment2.ZIP It claims to use a radiometric alignment algorithm. http://www.photoacute.com/tech/precise-radiometric-alignment.html So apparently more sophisticated alignment algorithms are possible in RAW space.

I'm not sure why HDRMerge didn't manage to align my set. The reason might be:

  1. Severe hand shake, to the extent that it necessitates more sophisticated alignment algorithms, such as the radiometric one.
  2. Mild hand shake, in the sense of small horizontal/vertical misalignments that HDRMerge is supposed to handle but failed to do so due to a bug.
  3. Fisheye deformation which in itself poses a challenge for HDRMerge's alignment algorithm.

I can rule out option 3 by testing the fisheye lens on a tripod. I hope to find time to do that soon.

I understand from your answer that you rule out option 2 and it sounds like you are hinting towards option 1. I'm not so sure that the alignment in this example is indeed so complex, but anyway notice that PhotoAcute works in RAW too and so more complex alignment algorithms are possible in RAW.

If you are certain that the problem happens as a result of a limitation in the current version of HDRMerge (rather than a bug) then I would mark this report as an open feature request rather than a closed bug.

jcelaya commented 9 years ago

Hello, sorry for the late answer.

Well, if PhotoAcute provides a correct output, then I cannot rule option 2 out. I would say it is not a bug, just that the algorithm I am using is not precise enough. I'll have a look at it (the day I find some spare time...)

Thanks for your feedback

Javi

2015-01-28 0:06 GMT+01:00 Assaf Toledo notifications@github.com:

PhotoAcute, which also works in RAW, has managed to align this set without such artifacts: https://dl.dropboxusercontent.com/u/21642925/Alignment2.ZIP It claims to use a radiometric alignment algorithm. http://www.photoacute.com/tech/precise-radiometric-alignment.html So apparently more sophisticated alignment algorithms are possible in RAW space.

I'm not sure why HDRMerge didn't manage to align my set. The reason might be:

  1. Severe hand shake, to the extent that it necessitates more sophisticated alignment algorithms, such as the radiometric one.
  2. Mild hand shake, in the sense of small horizontal/vertical misalignments that HDRMerge is supposed to handle but failed to do so due to a bug.
  3. Fisheye deformation which in itself poses a challenge for HDRMerge's alignment algorithm.

I can rule out option 3 by testing the fisheye lens on a tripod. I hope to find time to do that soon.

I understand from your answer that you rule out option 2 and it sounds like you are hinting towards option 1. I'm not so sure that the alignment in this example is indeed so complex, but anyway notice that PhotoAcute works in RAW too and so more complex alignment algorithms are possible in RAW.

If you are certain that the problem happens as a result of a limitation in the current version of HDRMerge (rather than a bug) then I would mark this report as an open feature request rather than a closed bug.

— Reply to this email directly or view it on GitHub https://github.com/jcelaya/hdrmerge/issues/61#issuecomment-71747833.

assaft commented 8 years ago

Just wanted to add here that the information I gave about PhotoAcute can be misleading. Indeed, PhotoAcute aligns in RAW, but it is a demosaiced RAW. It looks like it demosaics all bracketed files, aligns them, merges them, and then saves the merged file as a demosaicied ("linear") DNG.

Still, I'm trying to have a better understanding of the limitations mentioned in this comment. If I got things right, it is not possible to correct complex misalignment because we are dealing with non-demosaiced RAW files. But I wonder if this is because non-demosaiced data make it very hard to detect the type of misalignment and how to fix it (i.e. how to rotate/change perspective/etc in order to align the exposures), or because applying the needed correction to non-demosaicied data is impossible?

If the problem is mostly in the detection phase then HDRMerge can do a temporary internal demosaicing just for employing more powerful misalignment detection. This can make its aligner as good as any other open source aligner. If the problem is in the correction phase then that's a different story, but still I don't see why this is the case (i.e. why it is possible to apply rotation/perspective correction/etc. on demosaicied data and not on non-demosaiced).

jcelaya commented 8 years ago

The short story is that the shape of the mosaic in a RAW image, which comes from the way camera sensors are made, makes it impossible to apply such corrections, because each pixel only carries information about one color and lacks information of the other ones. Demosaicing fills those gaps.

OK, I'll try to be clear and concise... One of the main goals of HDRMerge is to merge exposures before demosaicing. Demosaicing can be a complex process that makes it very difficult to calculate some merging parameters precisely. However, the mosaic of RAW images is a rigid structure, and it limits the alignment of images. In such a mosaic, every pixel has only one color, either Red, Blue or Green (99% of cameras use such a mosaic, but there are other ones). Colors are distributed in the mosaic alternatively. For instance, this organization is very common: R G R G R G ... G B G B G B ... If you want to align two such images, pixels of the same color must align. So, displacements must be even, and rotations are impossible. The solution would be to interpolate color values with the other colors, but that's exactly what the demosaicing algorithms do.