Ability to save aligned/merged image before demosaic

[Entropy512]

It would be beneficial to leverage the HDR+ align/merge algorithms but allow performing the remaining processing steps in the workflow using a tool such as RawTherapee.

An example proof-of-concept hack that saves to TIFF can be seen at https://github.com/Entropy512/hdr-plus/commit/135273bad4d75f9b0e9bd695f97c10120355bf13 - the resulting file can be renamed to .dng and then tagged with exiftool such that RT treats it like any other DNG file.

Some method of allowing this to be chosen via the command line instead of hardcoding the behavior is needed. The return type of process() could be a challenge for this.

Additional work would be to integrate an EXIF library so that the DNG could be tagged on export as opposed to in an external script.

[KillerInk]

i have made changes like this including a dng writer. you can split the pipeline easy into two parts. the algin and merge, after that you have the merged "bayer data" and maybe want to save it. the merged bayer data can then get feed into the demosaic/tonemap/etc part. but i only needed the first part so the jpeg creation is not added

https://github.com/KillerInk/hdr-plus/tree/dng-stack/

the dng writer use libtiff, should work with jpg too.

[brotherofken]

I'm going to take work on this issue. I did some research and found at least three ways to write merged RAW files. Writing custom RAW formats seems infeasible, but DNG should do the job. The options are:

1. DNG SDK supports writing via dng_image_writer. It's not a standard package, so it would be hard to maintain such dependency,
2. DngCreator from Android Project. It's not a standalone C++ library, but JNI for the corresponding class in the Android Camera API. It might require extra work to port this code because it relies on internal library. Also, this may require copying a lot of code from the Android project and may be undesirable.
3. Use libTIFF directly, like it was done by @KillerInk and, for example in synthraw. Available almost anywhere from Linux/MAC repositories. In all cases, one has to copy all RAW fields read from RAW to DNG manually.

I think that libTIFF allows making a quick and dirty solution, so I'm going to try it.

Here are a few problems that may arise:

1. Opcode List Processing. Sometimes DNGs contain info about lens shading correction, e.g GainMap OpCode which determines parameters for lens shading correction. I'm not sure that it's would be easy to write this field correctly for RAW formats other than DNG.
2. To be continued.

[KillerInk]

About opcode. Itself is stored as bytearray, so copy paste the the tag is not a problem. The real problem are lenses with a heavy vignette. Due the merge it get burned to the final image and the provided gainmap is not strong enough to fix it. Just as side note, there is a second opcode to fix vignette, radial distortion. To workaround the burned in vignette, the best solution is to apply the lens correction bevor starting the align and merge.

[brotherofken]

Thanks for sharing your thoughts about vignetting @KillerInk !

I have some progress so far. I've found one cool project which does raw2dng conversion: Fimagena/raw2dng

The bad thing is that it's not a library, but an application that does raw2dng conversion. So I forked this project and added library target in a branch brotherofken/raw2dng. Then, I linked that library as 3rd party to hdr-plus.

Finally, I was able to convert RAWs from Timoty to DNG using new binary called 'stack_frames' Current progress is here: https://github.com/brotherofken/hdr-plus/tree/44-save-merged-dng The application is not finished yet, but it's not hard to do now.

Unfortunately, this means that now we have a new heavy dependency - raw2dng. On the other side, now it's possible to write valid DNG files, which is much better than TIFF or PNG.

I hope, that I'll be able to finish work shortly, but unfortunately I'm very limited in my free time.

[Entropy512]

Hmm... raw2dng itself may be kind of overkill for this and/or duplicating code that may already be in hdrplus.

For the most part, I believe libtiff allows you to set the relevant metadata tags AND I think libraw already allows you to retrieve relevant data (colormatrix, CFA pattern, etc).

I do plan on returning to this at some point, but I'm in the process of car shopping at the moment and that's draining my soul.

[brotherofken]

For the most part, I believe libtiff allows you to set the relevant metadata tags AND I think libraw already allows you to retrieve relevant data (colormatrix, CFA pattern, etc).

I agree with you and concerned that raw2dng is a bit overkill too. libtiff should be enough to write valid DNG, but it would require writing a bunch of boilerplate code that copies data from LibRaw into TIFF tags. And it would require more time to code and debug as well. So I intended to reuse some existing open-source that does the job (quite boring job, TBH) and saves time for algorithmic changes.

[brotherofken]

Maybe it would be Okay to use raw2dng as a temporary (he-he) solution until the libtiff version of DngSaver is not available.

[brotherofken]

Okay, raw2dng is not that cool to use as I thought before. :-)

So I prepared a prototype which writes DNGs using libtiff: https://github.com/brotherofken/hdr-plus/tree/44-save-merged-dng

It does not produce correct DNGs yet, but it's close. @Entropy512 I would appreciate it if you review the code.

[KillerInk]

im not entropy but, wondering whats wrong with the dng writer i wrote?^^ its fully working a produce working dngs. also it takes black lvl etc from libraw what i have yet seen:

1. you miss colormatrix2. its needed. 2.bevor writing the rawdata you should use TIFFCheckpointDirectory(tif); it writes out the tags a simple dng should contain these tags: TIFFTAG_SUBFILETYPE TIFFTAG_IMAGEWIDTH TIFFTAG_IMAGELENGTH TIFFTAG_BITSPERSAMPLE TIFFTAG_PHOTOMETRIC TIFFTAG_COMPRESSION TIFFTAG_SAMPLESPERPIXEL TIFFTAG_ORIENTATION TIFFTAG_PLANARCONFIG TIFFTAG_EP_STANDARD_ID TIFFTAG_DNGVERSION TIFFTAG_DNGBACKWARDVERSION TIFFTAG_COLORMATRIX1 TIFFTAG_COLORMATRIX2 TIFFTAG_ASSHOTNEUTRAL TIFFTAG_CALIBRATIONILLUMINANT1 TIFFTAG_CALIBRATIONILLUMINANT2

[brotherofken]

Hmmm, it seems that the wrong thing is that I didn't look through your code carefully. :-( Would you mind against careful copypasting? :-)

[Entropy512]

I won't be able to look in much detail until Saturday or maybe tonight.

colormatrix2 is NOT needed, and in fact is rarely available. libraw in, nearly all cases I'm aware of, will only have a D50 color matrix/profile.

exiftool -DNGVersion="1.4.0.0" -DNGBackwardVersion="1.4.0.0" -ColorMatrix1="0.5271 -0.0712 -0.0347 -0.6153 1.3653 0.2763 -0.1601 0.2366 0.7242" \
         -IFD0:BlackLevelRepeatDim="2 2" -IFD0:BlackLevel="512 512 512 512" -IFD0:WhiteLevel=16380 \
         -PhotometricInterpretation="Color Filter Array" -CalibrationIlluminant1=D65 \
         -SamplesPerPixel=1 -IFD0:CFARepeatPatternDim="2 2" -IFD0:CFAPattern2="0 1 1 2" $1

is what I use to tag Sony A7M3 files after saving. One thing that can be really confusing is if you accidentally tag a preview image instead of IFD0 if the file has a preview, or you saved the raw data somewhere other than IFD0.

Although in many cases, I copied all exif from one of the ARWs first. I need to retest to see what happens if I tag a blank image with no exif.

[Entropy512]

@KillerInk my main issue last time I looked at your DNG writer is that the commits also included a rework to use libraw for input that is significantly different architecturally than what landed here on upstream. https://github.com/KillerInk/hdr-plus/commit/c3e0c5134c287b453ca53202cd1a9cfd52d449d5 is a massive commit that, since it's achieving 4-5 goals in a single commit, is really hard to read and it's a bit difficult to separate out what is needed now that libraw is upstream with a somewhat different implementation.

[brotherofken]

Okay, guys, I think it's ready for review now. Please, test it on your RAW files or send me them for testing. It produces seemingly correct DNGs for CR2 from Timothy and DNGs from HDR+ dataset.

I decided not to put all tags to DNG as @KillerInk did because it might be an overkill for the moment, but I can do it on request.

[KillerInk]

libraw is a bit special in collecting the necessary informations. loading arw files fills different locations with the infos, then loading a dng. as sample using a dng you can get the black lvl from raw.imgdata.color.dng_levels.dng_cblack for cr2/arw it use raw.imgdata.black.

One thing that can be really confusing is if you accidentally tag a preview image instead of IFD0 if the file has a preview, or you saved the raw data somewhere other than IFD0.

dngs have the NewSubFileType Tag. 0 should get used for raw data. 1 for preview/thumb

SubIFD Trees DNG recommends the use of SubIFD trees, as described in the TIFF-EP specification. SubIFD chains are not supported. The highest-resolution and quality IFD should use NewSubFileType equal to 0. Reduced resolution (or quality) thumbnails or previews, if any, should use NewSubFileType equal to 1 (for a primary preview) or 10001.H (for an alternate preview). DNG recommends, but does not require, that the first IFD contain a low-resolution thumbnail, as described in the TIFF-EP specification.

colormatrix2 is NOT needed, and in fact is rarely available

its usefull for wb^^ adobes dng converter provide it normaly if the camera is not to exotic

my workflow looks like this with ARWs from my A6000. Convert my raws to dng with adobes dngconverter. Apply the LensCorrection. Drag Drop the corrected dngs onto hdr+ binary.

I know that my input is significantly different architecturally^^ I did not mean to take the full commit as it is. Just copy out the dngwriter inlcuding its needed classes. If the code is to ugly, i understand that^^ I stopped on it after i was able to build the pipe with gradle. Only commited it that it get not lost.

[brotherofken]

Bump. Could you make a review and check it, please?