Added support for bursts with bracketed exposure

[chris-rank]

closes #11

New features for bursts with Bayer pattern

• Bursts with bracketed exposure (e.g. -2EV, -1EV, 0EV, +1EV, +2EV) are now supported resulting in more efficient denoising (i.e. less images are needed for comparable results) and improved tonality and color fidelity in the shadows for scenes with very high dynamic range
• For underexposed bursts, optional exposure correction can be applied to further improve tonality in the shadows. RAWs of most cameras have a bit-depth of 12-14 bits while merging is performed with a bit depth of 32 bit. As a consequence, tonality of the deep shadows is much improved after merging. To preserve this improved tonality, shadows have to be lifted before converting the image back into a 12-14 bit DNG to prevent quantization errors. This can be achieved by applying a linear gain or a non-linear tone mapping operator that is close to linear in the shadows and midtones and then gets very flat in the highlights. Recovering the deep shadows in post is not possible as information already got lost during the conversion to 12-14 bit.
• Improved temporal averaging for uniform exposure with better handling of very bright highlights near the white point
• The new functionality described above only works when several metadata information can be extracted correctly from the DNGs: black level, white level, mosaic pattern, RGB color factors of the mosaic pattern and exposure bias set by the camera (e.g. -2EV, -1EV…). It does not work if the camera operates at extended ISO values.
• Different options for exposure correction can be chosen in the preferences under the new parameter 'Exposure control':
  • Off: Fallback solution with best file compatibility (no exposure metadata extraction required), but reduced functionality: Less efficient hot pixel suppression; Simple temporal averaging for uniform exposure can cause color cast in very bright highlights.
  • Linear (default): Linear exposure correction applying gain selected to leverage full bit range of RAW file; Can improve tonality of shadows without burning highlights; No correction if darkest frame of burst already has bright pixels very close to the white point.
  • Linear (relative +1 EV): Linear exposure correction with gain calculated as maximum of +1 EV and gain from linear (default) setting; Improves tonality of shadows, but can burn highlights.
  • Non-linear (target ±0 EV): Non-linear exposure correction with target ±0 EV based on camera’s exposure compensation setting; Improves tonality of shadows and compresses highlights to protect them; Always applies linear (default) correction first; Non-linear correction not applied if absolute exposure after linear (default) correction ≥ 0 EV.
  • Non-linear (target +1 EV): Non-linear exposure correction with target +1 EV based on camera’s exposure compensation setting; Improves tonality of shadows and compresses highlights to protect them; Always applies linear (default) correction first; Non-linear correction not applied if absolute exposure after linear (default) correction ≥ 1 EV.

Additional improvements

• Improved alignment of frames
• Improved hot pixel suppression
• Improved performance of image processing with calculation times reduced by ~10-20%
• Improved code structure
• Several small fixes

References used for inspiration

Google Research Blog: HDR+ with bracketing on pixel phones (https://ai.googleblog.com/2021/04/hdr-with-bracketing-on-pixel-phones.html) Reinhard et al. 2002: Photographic tone reproduction for digital images (https://www-old.cs.utah.edu/docs/techreports/2002/pdf/UUCS-02-001.pdf)

It is recommended to inspect the example at 100% zoom level.

[Alex-Vasile]

This is based on #16, and should be merged in only after that one.

[Alex-Vasile]

Does the current implementation work if the user manually specifies ISO/speed/ and f-stop? Related question, will it work if the f-stop is not stored in the metadata? (e.g. assume f-stop is the same between shots) I sometimes shoot either vintage or not-electronically coupled lenses and would still like to get the benefit from this approach.

[chris-rank]

Does the current implementation work if the user manually specifies ISO/speed/ and f-stop? Related question, will it work if the f-stop is not stored in the metadata? (e.g. assume f-stop is the same between shots) I sometimes shoot either vintage or not-electronically coupled lenses and would still like to get the benefit from this approach.

The required information is the exposure bias specified in the EXIF. You can typically manipulate it on your camera with the exposure compensation dial (e.g. -1EV, +1EV, ...). To shoot a burst with different exposure bias settings, many cameras have an automatic exposure bracketing mode. When you set the camera at base ISO and fix the f-stop, the camera should automatically adjust the exposure time depending on the exposure bias setting. So I assume it should also work with manual lenses as long as your camera can write the exposure bias into the EXIF information.

[Alex-Vasile]

The required information is the exposure bias specified in the EXIF. You can typically manipulate it on your camera with the exposure compensation dial (e.g. -1EV, +1EV, ...). To shoot a burst with different exposure bias settings, many cameras have an automatic exposure bracketing mode. When you set the camera at base ISO and fix the f-stop, the camera should automatically adjust the exposure time depending on the exposure bias setting. So I assume it should also work with manual lenses as long as your camera can write the exposure bias into the EXIF information.

I am considering a scenario where I have a static scene and I will manually change exposure (by changing shutter speed) to properly expose for different regions rather than use a uniformly spaced burst. Though I may be in a very small minority and most people may simply change the EV.

[Alex-Vasile]

I think it would be useful to provide the tone mapping equation for the different settings so that people could apply the inverse in their raw editor if they wish to get back the linear (or expected) response while still preserving the benefits of having the curve applied to the raw file.

[martin-marek]

Thank you @chris-rank and @Alex-Vasile for the amazing work! I've released this as v1.4 on GitHub and will submit it to App Store for review tomorrow. Please feel free to make any further documentation changes if you think of anything that might be helpful.