search

Exiv2 / exiv2

Image metadata library and tools
http://www.exiv2.org/
Other
932 stars 281 forks source link

XMP with Google depth-map gets corrupted when metadata is modified #2150

Open josemoliver opened 2 years ago

josemoliver commented 2 years ago

Refer to Digikam Bug: https://bugs.kde.org/show_bug.cgi?id=451466 An image file with XMP metadata for Google depth-map (Images taken in Portrait mode) gets corrupted when metadata is modified in Digikam (likely exiv2 issue)

Reference: Google Dynamic Depth Specification https://developer.android.com/training/camera2/Dynamic-depth-v1.0.pdf

piponazo commented 2 years ago

Thanks for reporting this issue. I'll try to look at it

josemoliver commented 2 years ago

Here is a sample image containing Google Depth Metadata: PXL_20220313_165550133 PORTRAIT

clanmills commented 2 years ago

There are two issues with this file. The XMP uses "Extended XMP" which is not supported by the XMPsdk built into Exiv2. 

$ exiv2 -px ~/Downloads/GoogleDepth.jpg 
Xmp.xmpNote.HasExtendedXMP                   XmpText    32  2116AA3FE861E37BFBAFDC04ACB9413B
Xmp.GCamera.SpecialTypeID                    XmpBag      1  com.google.android.apps.camera.gallery.specialtype.SpecialType-PORTRAIT

Secondly, there is more that 2mb of XMP metadata spanning multiple JPEG segments. The maximum length of the JPEG segment is 64k because the length of the segment is stored as a short (16bit integer).

$ exiv2 -pS ~/Downloads/GoogleDepth.jpg
STRUCTURE OF JPEG FILE: /Users/rmills/Downloads/GoogleDepth.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe1 APP1  |   18704 | Exif..II*......................
   18708 | 0xffe1 APP1  |     636 | http://ns.adobe.com/xap/1.0/.<x:
   19346 | 0xffe1 APP1  |   65460 | http://ns.adobe.com/xmp/extensio
   84808 | 0xffe1 APP1  |   65460 | http://ns.adobe.com/xmp/extensio
...
 2245054 | 0xffe1 APP1  |   27826 | http://ns.adobe.com/xmp/extensio
 2272882 | 0xffe0 APP0  |      16 | JFIF.........
 2272900 | 0xffdb DQT   |      67 
 2272969 | 0xffdb DQT   |      67 
 2273038 | 0xffe2 APP2  |     612 | ICC_PROFILE......T........mntrRG chunk 1/1
 2273652 | 0xffc0 SOF0  |      17 
 2273671 | 0xffc4 DHT   |      31 
 2273704 | 0xffc4 DHT   |     181 
 2273887 | 0xffc4 DHT   |      31 
 2273920 | 0xffc4 DHT   |     181 
 2274103 | 0xffda SOS

Perhaps you could explain "...gets corrupted when metadata is modified in Digikam". It's very likely that the Extended XMP will be lost when you edit this file with Exiv2. It's probably a modest change to recognise this class of file in readMetadata() and store the XMP to be rewritten by writeMetadata().

If you wish to modify the XMP metadata, that can be done by extracting the XMP as a sidecar.

$ exiv2 -pX ~/Downloads/GoogleDepth.jpg > ~/Downloads/GoogleDepth.xmp
$ exiv2 -pa ~/Downloads/GoogleDepth.xmp
Xmp.GCamera.PortraitNote                     XmpText   152  SERSUALvZDVtXnAeLOrj/Wd8EXMtdAZ8h8twnicX2v3Os7mUjwfKeIErmghpcohPKrIgw5sh237nHyYnYhJuGb5U9M1AZJg+Vpjfr2GLot0iBMu6quUrhMaXyeBpnyO+dC+5r6B9TgjYKnCeDERCffw=
Xmp.GCamera.RelitInputImageMime              XmpText    10  image/jpeg
Xmp.GCamera.RelitInputImageData              XmpText   2163424  /9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAMDBAIFAQEEBAEBBAQEAQMEBAQDAwYGBQUFBgcICAcHBwcHCAcICQgICQoKCQkLCQcICwsLCgwMDAkKDAsLCwr/

...

Xmp.Device.Cameras[1]/Camera:ImagingModel    XmpText     0  type="Struct"
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:FocalLengthX XmpText    11  3193.716797
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:FocalLengthY XmpText    11  3193.716797
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:ImageWidth XmpText     4  4032
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:ImageHeight XmpText     4  3024
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:PrincipalPointX XmpText    11  1990.501831
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:PrincipalPointY XmpText    11  1580.797852
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:Skew XmpText     8  0.000000
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:PixelAspectRatio XmpText     8  1.000000
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:DistortionCount XmpText     1  4
Xmp.Device.Cameras[1]/Camera:ImagingModel/ImagingModel:Distortion XmpText    43  AACAPwAAAADGIJ89AAAAAFqEWr4AAAAARawhPgAAAAA
$
clanmills commented 2 years ago

I've read the correspondence with DigiKam. A lot of discussion that I don't understand concerning namespaces.

I edited Exif.Image.DateTime:

693 rmills@rmillsm1:~/Downloads $ cp GoogleDepth.jpg GD.jpg
694 rmills@rmillsm1:~/Downloads $ exiv2 -K Exif.Image.DateTime GD.jpg 
Exif.Image.DateTime                          Ascii      20  2022:03:13 12:55:50
695 rmills@rmillsm1:~/Downloads $ exiv2 -M'set Exif.Image.DateTime 2022:03:15 11:59:35' GD.jpg 
696 rmills@rmillsm1:~/Downloads $ exiv2 -K Exif.Image.DateTime GD.jpg 
Exif.Image.DateTime                          Ascii      20  2022:03:15 11:59:35
697 rmills@rmillsm1:~/Downloads $ ls -l GoogleDepth.jpg GD.jpg 
-rw-r--r--@ 1 rmills  staff  5514542 15 Mar 11:59 GD.jpg
-rw-r--r--@ 1 rmills  staff  5512499 15 Mar 11:44 GoogleDepth.jpg
698 rmills@rmillsm1:~/Downloads $

When the file was rewritten, the Extended XMP is in the file, however it's near the start of the file and doesn't immediately trail the APP1/http://ns.adobe.com/xap/1.0/ segment:

$ exiv2 -pS GD.jpg 
STRUCTURE OF JPEG FILE: GD.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe1 APP1  |   65460 | http://ns.adobe.com/xmp/extensio
   65464 | 0xffe1 APP1  |   65460 | http://ns.adobe.com/xmp/extensio
...
 2225710 | 0xffe1 APP1  |   27826 | http://ns.adobe.com/xmp/extensio
 2253538 | 0xffe0 APP0  |      16 | JFIF.........
 2253556 | 0xffe1 APP1  |   18704 | Exif..II*......................
 2272262 | 0xffe1 APP1  |    2679 | http://ns.adobe.com/xap/1.0/.<?x
 2274943 | 0xffe2 APP2  |     612 | ICC_PROFILE......T........mntrRG chunk 1/1
 2275557 | 0xffdb DQT   |      67 
 2275626 | 0xffdb DQT   |      67 
 2275695 | 0xffc0 SOF0  |      17 
 2275714 | 0xffc4 DHT   |      31 
 2275747 | 0xffc4 DHT   |     181 
 2275930 | 0xffc4 DHT   |      31 
 2275963 | 0xffc4 DHT   |     181 
 2276146 | 0xffda SOS  
704 rmills@rmillsm1:~/Downloads $ exiv2 -px GD.jpg 
Xmp.xmpNote.HasExtendedXMP                   XmpText    32  2116AA3FE861E37BFBAFDC04ACB9413B
Xmp.GCamera.SpecialTypeID                    XmpBag      1  com.google.android.apps.camera.gallery.specialtype.SpecialType-PORTRAIT
705 rmills@rmillsm1:~/Downloads $

In the DigiKam issue report, you said something about "adding a statue". I think you're referring to something in the DigiKam UI.

I'm optimistic that modest changes to readMetadata() and writeMetadata() can preserve the extended XMP and rewrite it in the correct location in the file. This will enable you to safely edit Exif tags.

Editing the data in the Extended XMP might be challenging. It's possible to test that on the extracted sidecar.

It would be much easier to arrive at a definition of the issue if you could use the exiv2 command-line program to modify the file. Life is simpler when we both use the same tools. I'd also like to explain my intention here. I have retired from Exiv2. I'm interested in the analysis of this issue. The implementation of necessary changes will rely on a Team Exiv2 Engineer being available and willing to work on the code.

josemoliver commented 2 years ago

Thanks Robin for your analysis.

In the Digikam bug, I started with a file already modified in GeoSetter, which uses exiftool version 12.40 (ref file attached to this comment)- my mistake, as I should have started with the original file as generated by the Google Phone. Using Digikam, I added the tag/keyword "statue" and that is when I noticed the problems with the file in which the namespace was missing. I am familiarizing myself with exiv2 command line.

Sample File modified in exiftool 12.40 TestPortrait_exiftool .