Broken InteropIFD after changing orientation

[StefanOltmann]

I got an original.jpg which I modified by calling addImageField() and Metadata.insertExif(). This is my Kotlin code:

            val metadata = Metadata.readMetadata(bytes.inputStream())

            val exif = metadata[MetadataType.EXIF] as? Exif

            exif?.addImageField(TiffTag.ORIENTATION, FieldType.SHORT, shortArrayOf(4))

            Metadata.insertExif(
                bytes.inputStream(),
                File("modified.jpg").outputStream(),
                exif,
                true
            )

The resulting file is modified.jpg.

Comparing these with ExifTool I spot two problems:

• The InteropIFD is broken
• The "Rating Percent" got deleted

[StefanOltmann]

In fact the current approach of insertExif seems to remove a lot of data. In this sample all the maker notes.

photo_1.jpg photo_1.txt

photo_1_rotated.jpg photo_1_rotated.txt

[dragon66]

@StefanOltmann are you using the latest code in the repository? There have been some code change not long ago related to InteropIFD and some attempt to keep makernote intact.

So at least InteropIFD should not be broken. For makernote, I haven't found a better way to handle it so it will only make sure to keep its original byte order and write it back as is. The consequence is unless the makernote is self contained - means the byte offset for it references the makernote itself, it will not be re-read correctly after Exif data change in the current implementation.

[StefanOltmann]

@dragon66

Thank you for responding.

Yes, I used the latest code in the repository. Do you have a different result if you try the code above with the provided sample photo? In my test the MakerNote was lost.

[dragon66]

@StefanOltmann As I mentioned in my previous comment, maker note is an issue with the current implementation. I tried to recreate your result by change the orientation with value 4. After that I read the metadata for this new image and get the following information for the IFD0 part. The orientation is now BottomLeft which should look like the number 4 from the attached image. 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - IFD0: Image Info 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Rating: [0] 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Copyright: CC-BY 3.0 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Resolution Unit: Inch 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Make: Canon 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Model: Canon EOS 70D 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Software: darktable 2.6.2 2023-04-30 11:45:12,379 [main] INFO com.icafe4j.test.TestMetadata - Orientation: BottomLeft 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - DateTime: 2019:07:15 20:09:19 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - Exif Sub IFD: [294] 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - XResolution: 300 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - GPS Sub IFD: [6512] 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - YResolution: 300 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - Artist: dxfoto.ru 2023-04-30 11:45:12,380 [main] INFO com.icafe4j.test.TestMetadata - DateTime Original: 2019:07:14 09:21:11

2023-04-30 12:02:41,381 [main] INFO com.icafe4j.test.TestMetadata - Interoperability SubIFD: Interoperability Info 2023-04-30 12:02:41,381 [main] INFO com.icafe4j.test.TestMetadata - Interoperability Index: R98 2023-04-30 12:02:41,381 [main] INFO com.icafe4j.test.TestMetadata - Interoperability Version: [0x30,0x31,0x30,0x30]

[dragon66]

You can see the interopIFD and the rating value are still there but there is no rating percent. This is consistent with the reading of the original image which also only shows rating. I am not sure why the text file you provided showing rating percent but it is not included in the original image.

The make note in this case, although it is still in the resulting image becomes unreadable by the metadata reader.

[StefanOltmann]

I‘m sorry, I should have mentioned how I created the text files in the first place. To test/validate how photo apps and/or libraries change the metadata of my photos I trust ExifTool by Phil Harvey (exiftool.org), which I found to be the most reliable tool for this.

You can use exiftool -g1 -a file.jpg > file.txt to create a TXT file showing all metadata.

If you do that and compare the files you will see that some information is lost.

I didn’t find the cause yet, but there must be either something that is skipped on reading the file or something that is skipped on writing.

[dragon66]

@StefanOltmann The mystery of the missing "Rating Percent" is solved. The data type for this field supposed to be unsigned short value but the actual data type is signed long value. The library contains a bug which is not handling a few of the signed data types for some reason so this field was dropped but the "Rating" field was not.

I have added the handling of the missing signed field types and now it is working as expect.

With that, the only thing that is wrong is the maker note IFD which is not decoded and is written back to the image without processing. This will make the maker note invisible to metadata readers after metadata re-write.

[StefanOltmann]

Thank you for investigating here. It’s very interesting.

I really like your project. It has a good code style and an easy API.

[StefanOltmann]

Is it possible to retain the MakerNote without decoding it when using pixymeta to modify orientation, rating, and keywords? I want to make sure the original metadata isn't damaged by removing any parts of it.

I noticed that pixymeta goes beyond EXIF by also examining the ICC profile and containing code for the Huffman tables. Is this because of ICAFE or is there another reason for including this data in a metadata-focused library?

For my purposes, I'm searching for a simple, pure-Java library that is easy to use and has minimalistic features to write metadata.

[dragon66]

@StefanOltmann The MakerNote after the metadata change is actually copied with the image as is. The issue is it is of IFD structure which means unless it is completely self-contained, there is no way it can be read again later. The only way we can make it right is when reading decode it (at least read it into an IFD) and re-wite the IFD back to the resulting image. The difficulty here is there is no uniquely defined rules for different cameras to write the MakerNote and the biggest issue is the offset value of the data inside it. It could reference the start of Exif or in some cases, even the start of the image itself. The best way is to reference the start of itself but the world is not perfect plus most cameras will have a header before the IFD which even worse.

The library is intended for more than Exif and Huffman tables are for reading JPEG metadata.

[StefanOltmann]

Thank you for educating me.

[StefanOltmann]

I close this issue as it is no longer relevant for me.

I have developed my own library called Kim, which can be found at https://github.com/Ashampoo/kim. It is built upon Apache Commons Imaging and specifically designed to retain MakerNotes.

I sincerely appreciate your assistance in clarifying this subject. Thank you once again.