search

jcelaya / hdrmerge

HDR exposure merging
http://jcelaya.github.io/hdrmerge/
Other
367 stars 78 forks source link

Copy embedded lens profiles #49

Closed assaft closed 10 years ago

assaft commented 10 years ago

Micro four thirds cameras embed a lens profile in their RAW files (ORF/RW2). When OEM converters / ACR open such a file, they correct the lens aberrations based on the information stored in the included profile.

I noticed that Lightroom doesn't correct the distortion of DNG files created by hdrmerge (from several ORF files). It could be because hdrmerge doesn't include the lens profile when merging several RAW files into a DNG. I believe that the profile values can be copied as is from the original RAW files but probably the profile should be encoded somewhat differently in the DNG container. It would be great if hdrmerge could support this.

Information on the location of the profile in RW2 files and about its internal components can be found here: http://syscall.eu/#pana

jcelaya commented 10 years ago

All the metadata is copied from the input files to the output DNG, using Exiv2 (http://exiv2.org/). Some metadata is standard, while other is proprietary and depends on the maker. If Exiv2 is able to detect this metadata (called the "Makernote") for your camera maker and model, there should be no problem. However, I have seen at least one Olympus camera that declares to be made by Ricoh (who bought Olympus some years ago). This fact misleads Exiv2 until v0.24 (it should be fixed in the latest development snapshot) and the Makernote is not copied. Maybe you are experiencing a similar problem. Can you send me an example picture, or tell me your camera maker and model?

assaft commented 10 years ago

Thanks for the quick reply. It's an Olympus E-M10.

I prepared a zip file with an example: https://dl.dropboxusercontent.com/u/21642925/rawfiles.zip It contains:

  1. A RAW file - PA180692.ORF + a DNG export from LR. The RAW file contains the lens profile and LR transfers the correction values to the DNG format. I'm not an expert but I see that this DNG file contains: A. The exif tags: Camera Profile : Adobe Standard Camera Profile Digest : CA7D08049AE0320C962C93896BD58813

B. When running dng_validate -v it shows: Parsing OpcodeList3: 1 opcode

Opcode: WarpRectilinear, minVersion = 1.3.0.0, flags = 0 Planes: 3 Optical center: h = 0.498276 v = 0.500000 Plane 0: Radial params: 1.000000, -0.099675, 0.025612, -0.005231 Tangential params: 0.000000, 0.000000 Plane 1: Radial params: 1.000000, -0.100233, 0.026256, -0.005319 Tangential params: 0.000000, 0.000000 Plane 2: Radial params: 0.999999, -0.100157, 0.027315, -0.005481 Tangential params: 0.000000, 0.000000

This indicates that the DNG contains information for correcting lateral chromatic aberrations and I assume that also distortion information. A text file is included with this information.

  1. An hdrmerged DNG file - PA180691-180693_hdrmerge.dng + a DNG export from LR after tonemapping: PA180691-180693_hdrmerge_tonemapped_in_lr.dng. These DNGs are uncorrected and I think that this is because the correction values were not copied from the original RAW files used for the merging. Indeed, there are no corresponding exif tags (like Camera Profile Digest) and no WarpRectilinear opcodes.

As far as I know the opcodes are some sort of extension in the DNG file format. See page 16 in DNG Spec 1.4.0.0.pdf.

Let me know if there's anything else I can help with.

Thanks!

assaft commented 10 years ago

A workaround to this issue would be to convert the bracketed RAW files to DNG using Adobe's DNG Converter and then to use HDRMerge for merging them into a new DNG. This would utilize Adobe's tool for decoding the lens correction data from the proprietary RAW files and encoding this data into DNGs.

However, I tried this workflow and it doesn't work. I converted a group of ORF files using Adobe DNG Converter and verified (by dng_validate.exe) that the generated files have the lens correction information (i.e. the opcodes). When I merge them in HDRMerge I get a DNG that doesn't have the opcodes and not being corrected when opened in Lightroom.

assaft commented 10 years ago

I have some more info to give.

My goal was to find how to add the lens geometry correction data to a DNG file that HDRMerge produces.

Preliminary steps:

  1. I converted 3 ORF files to DNG using Adobe DNG Converer. This produces base1.dng, base2.dng and base3.dng.
  2. HDRMerge merged the files and produced merged.dng
  3. base1.dng and merged.dng were imported to Lightroom to verify that indeed the geometry distortion is corrected only for base1.dng.

At this point I started to look at the differences between these two DNG files. exiv2 was used to print all tags. This indicated that merge.dng is missing the tag Exif.SubImage1.OpcodeList3 - which is known to contain the lens correction information. This is the root of the problem.

To verify that, I tried to add the OpcodeList3 tag manually using exiv2. However, exiv2 did not support addition of such tag (explanation below) so first I had to alter its code and to rebuild it. I then added the missing tag and verified that now the DNG is opened in Lightroom with the lens geometry corrected.

Regarding the limitation of exiv2:

  1. The 'add' command doesn't work on tags that are considered 'image tags'. It works only on 'metadata tags'. This is due to some problems with complex NEF files.
  2. The reason why Exif.SubImage1.OpcodeList3 is considered an 'image tag' is that Exif.SubImage1.NewSubfileType has the value "Primary image". This is discussed here (with respect to OpcodeList1): http://dev.exiv2.org/boards/3/topics/1650 To overcome this problem, I patched the code in tiffimage.cpp (attached).

The upshot of this is that in order to preserve the lens geometry correction, HDRMerge needs to copy the tag Exif.SubImage1.OpcodeList3 from one of the input DNGs to the DNG that it produces. This can be done using exiv2 once it is patched.

Attachments:

  1. base1/2/3.dng - input DNG files
  2. merged.dng - merged DNG file
  3. base1.txt - a print of the tags in base1.dng using "exiv2.exe -b -pa base1.dng"
  4. merged.txt - a print of the tags in merged.dng using "exiv2.exe -b -pa merged.dng"
  5. merged_patched.dng - the merged DNG file with the addition of the OpcodeList3 tag.
  6. merged_patched.txt - a print of the tags in merged_patched.dng using "exiv2.exe -b -pa merged_patched.dng"
  7. before and after versions of "tiffimage.cpp".

To download: https://dl.dropboxusercontent.com/u/21642925/LensCorrectionTag.ZIP

Additional notes:

  1. When I print the tags of these dng files I get an error message that I ignore: Error: Failed to read Olympus IFD Makernote header. 0000 41 64 6f 62 65 00 4d 61 6b 4e 00 16 50 12 49 49 Adobe.MakN..P.II
  2. My original files - base1/2/3 were taken hand-held; therefore the alignment is not perfect and HDRMerge had to crop a little in the merged file. As a result base1.dng and mareged_patched.dng are not perfectly aligned, but nevertheless they are both geometrically corrected.
  3. Everything I wrote here is based on experiments with the Olympus E-M10. I would expect RAW files of other makers that rely on software correction of lens aberrations to behave similarly.
jcelaya commented 10 years ago

Nice research! Using it, I found a workaround that allows me to copy the OpcodeList tags from the source DNG files to the merged DNG:

  1. Set the NewSubfileType tag of SubImage1 to something else than Primary Image using my own DNG writer.
  2. Copy the OpcodeList tags with Exiv2, then set the NewSubfileType tag to Primary Image. This fix does not need a modified version of Exiv2. I have commited it to the development branch.

I will mark this as fixed, although just copying the OpcodeList tags if they already exist is not a perfect solution. However, calculating the lens correction information from the maker metadata is way too complex. I could do it easily with the DNG SDK, but linking against it is a Total Pain In The Neck for serveral other reasons, so it is not an alternative.

assaft commented 10 years ago

Thanks. Nice solution. Some comments:

  1. I noticed that if I'm copying OpcodeList2 I get a DNG that has a greensish cast on the top left part of the image. This happened with a different set of DNGs (ones that were exported from lightroom after some edits) so I don't know how representative this is. Anyway, with this set, the base exposure had a single OpcodeList3 and two OpcodeList2. I tested all combinations and concluded that copying only OpcodeList3 was the right thing to do. According toe Adobe DNG spec (1.4): OpcodeList2: Specifies the list of opcodes that should be applied to the raw image, just after it has been mapped to linear reference values OpcodeList3: Specifies the list of opcodes that should be applied to the raw image, just after it has been demosaiced. So based on my limited knowledge in this field I would guess that lens correction will always be in OpcodeList3, and that copying OpcodeList2 from a dng before the merging to a dng after it would create a mess.
  2. Yes, interpreting the lens correction info from the maker metadata is way too complex. No point in spending time on that given that Adobe DNG Converter does the job and is available for free. The only issue is with Linux support. I've read that it can be executed with Wine, but only in command-line mode. That should be enough for most users, I guess.
  3. I don't think that you could use the DNG SDK to extract info from the maker raw files (i.e. lens correction info from an ORF file). Parsing the maker raw files is part of the closed-source DNG Converter, not part of the SDK. The converter's logic is hidden probably because it contains code or data from camera makers under an NDA. As far as I know there is no open-source converter that can interpret this type of maker metadata and no reverse engineering of the values. Only raw converters that obtained a license from the makers can do that and they are all closed-source.
  4. Do you plan to release a new official build soon? At least for users of Olympus/Panasonic cameras this is an important fix
jcelaya commented 10 years ago
  1. Be careful. In the dng you sent me, there was a SubImage1.OpcodeList3, a SubImage3.OpcodeList2 and a SubImage4.OpcodeList2. Both SubImage3 and SubImage4 are previews, not the raw image, so these tags must not be copied.
  2. Should be...
  3. Of course, you are right.
  4. Yes, but I wanted to include another couple of features. It should be out in some days.
assaft commented 10 years ago
  1. Correct. I thought one of the OpcodeList2 was in SubImage1. These are previews added by LR when I developed the file. I didn't encounter an OpcodeList2 in a direct ORF->DNG conversion using Adobe's DNG Converter, but based on the description in the DNG 1.4 spec I still wouldn't aim to copy this tag by default (even if it appears in SubImage1).
  2. Thanks, looking forward!
assaft commented 9 years ago

I verified that it works in 0.5. Thanks.