search

pulsejet / memories

Fast, modern and advanced photo management suite. Runs as a Nextcloud app.
https://memories.gallery
GNU Affero General Public License v3.0
3.08k stars 82 forks source link

Live Photo doesn't work with Pixel 6 Pro and Ultra HDR #965

Closed rhssk closed 7 months ago

rhssk commented 9 months ago

Describe the bug Pixel Camera 9.2 introduced "Ultra HDR" setting on previous Pixel devices (in my case, Pixel 6 Pro). This setting is on by default after the update. Motion Photos taken with this setting enabled aren't properly played in Memories.

To Reproduce I've taken 2 photos: one with "Ultra HDR" enabled, other with it disabled. When uploaded to Memories, only the second one has functioning video playback.

Ultra HDR:

Regular:

Platform:

Kdubs937 commented 9 months ago

I can confirm this issue with a Pixel 7 Pro as well and with the same results with turning ultra HDR on/off. The Nextcloud log shows the following error message with Ultra HDR Turned on when the live photo is uploaded from my phone to Nextcloud. No error with Ultra HDR turned off.

Undefined array key "Copy2:DirectoryItemLength" at /var/www/html/custom_apps/memories/lib/Db/LivePhoto.php#78

eikaramba commented 8 months ago

same for pixel 8 pro

eikaramba commented 7 months ago

i am trying to debug the issue and posting my findings in the meantime:

.\exiftool.exe -api -n -json -U -G4 PXL_20231217_175442599.MP.ultraHDR.jpg is showing me

[{
  "SourceFile": "PXL_20231217_175442599.MP.ultraHDR.jpg",
  "Unknown:ExifToolVersion": 12.73,
  "Unknown:FileName": "PXL_20231217_175442599.MP.ultraHDR.jpg",
  "Unknown:Directory": ".",
  "Unknown:FileSize": "6.4 MB",
  "Unknown:ZoneIdentifier": "Exists",
  "Unknown:FileModifyDate": "2024:01:19 16:57:45+01:00",
  "Unknown:FileAccessDate": "2024:01:19 17:50:21+01:00",
  "Unknown:FileCreateDate": "2024:01:19 16:57:42+01:00",
  "Unknown:FilePermissions": "-rw-rw-rw-",
  "Unknown:FileType": "JPEG",
  "Unknown:FileTypeExtension": "jpg",
  "Unknown:MIMEType": "image/jpeg",
  "Unknown:ExifByteOrder": "Little-endian (Intel, II)",
  "Unknown:Make": "Google",
  "Unknown:Model": "Pixel 6 Pro",
  "Unknown:Orientation": "Horizontal (normal)",
  "Unknown:XResolution": 72,
  "Unknown:YResolution": 72,
  "Unknown:ResolutionUnit": "inches",
  "Unknown:Software": "HDR+ 1.0.585804401zd",
  "Unknown:ModifyDate": "2023:12:17 19:54:42",
  "Unknown:YCbCrPositioning": "Centered",
  "Unknown:ExposureTime": "1/24",
  "Unknown:FNumber": 1.9,
  "Unknown:ExposureProgram": "Program AE",
  "Unknown:ISO": 1550,
  "Unknown:ExifVersion": "0232",
  "Unknown:DateTimeOriginal": "2023:12:17 19:54:42",
  "Unknown:CreateDate": "2023:12:17 19:54:42",
  "Unknown:OffsetTime": "+02:00",
  "Unknown:OffsetTimeOriginal": "+02:00",
  "Unknown:OffsetTimeDigitized": "+02:00",
  "Unknown:ComponentsConfiguration": "Y, Cb, Cr, -",
  "Unknown:ShutterSpeedValue": "1/24",
  "Unknown:ApertureValue": 1.9,
  "Unknown:BrightnessValue": -2.59,
  "Unknown:ExposureCompensation": 0,
  "Unknown:MaxApertureValue": 1.9,
  "Unknown:SubjectDistance": "0.373 m",
  "Unknown:MeteringMode": "Center-weighted average",
  "Unknown:Flash": "Off, Did not fire",
  "Unknown:FocalLength": "6.8 mm",
  "Unknown:SubSecTime": 599,
  "Unknown:SubSecTimeOriginal": 599,
  "Unknown:SubSecTimeDigitized": 599,
  "Unknown:FlashpixVersion": "0100",
  "Unknown:ColorSpace": "sRGB",
  "Unknown:ExifImageWidth": 4080,
  "Unknown:ExifImageHeight": 3072,
  "Unknown:InteropIndex": "R98 - DCF basic file (sRGB)",
  "Unknown:InteropVersion": "0100",
  "Unknown:SensingMethod": "One-chip color area",
  "Unknown:SceneType": "Directly photographed",
  "Unknown:CustomRendered": "Custom",
  "Unknown:ExposureMode": "Auto",
  "Unknown:WhiteBalance": "Auto",
  "Unknown:DigitalZoomRatio": 0,
  "Unknown:FocalLengthIn35mmFormat": "24 mm",
  "Unknown:SceneCaptureType": "Standard",
  "Unknown:Contrast": "Normal",
  "Unknown:Saturation": "Normal",
  "Unknown:Sharpness": "Normal",
  "Unknown:SubjectDistanceRange": "Macro",
  "Unknown:LensMake": "Google",
  "Unknown:LensModel": "Pixel 6 Pro back camera 6.81mm f/1.85",
  "Unknown:CompositeImage": "Composite Image Captured While Shooting",
  "Unknown:GPSVersionID": "2.2.0.0",
  "Unknown:GPSImgDirectionRef": "Magnetic North",
  "Unknown:GPSImgDirection": 278,
  "Unknown:Compression": "JPEG (old-style)",
  "Unknown:ThumbnailOffset": 1128,
  "Unknown:ThumbnailLength": 5075,
  "Unknown:JFIFVersion": 1.02,
  "Unknown:ProfileCMMType": "",
  "Unknown:ProfileVersion": "4.0.0",
  "Unknown:ProfileClass": "Display Device Profile",
  "Unknown:ColorSpaceData": "RGB ",
  "Unknown:ProfileConnectionSpace": "XYZ ",
  "Unknown:ProfileDateTime": "2023:03:09 10:57:00",
  "Unknown:ProfileFileSignature": "acsp",
  "Unknown:PrimaryPlatform": "Unknown ()",
  "Unknown:CMMFlags": "Not Embedded, Independent",
  "Unknown:DeviceManufacturer": "Google",
  "Unknown:DeviceModel": "",
  "Unknown:DeviceAttributes": "Reflective, Glossy, Positive, Color",
  "Unknown:RenderingIntent": "Perceptual",
  "Unknown:ConnectionSpaceIlluminant": "0.9642 1 0.82491",
  "Unknown:ProfileCreator": "Google",
  "Unknown:ProfileID": "61473528d5aaa311e143dfc93efaa268",
  "Unknown:ProfileDescription": "sRGB IEC61966-2.1",
  "Unknown:ProfileCopyright": "Copyright (c) 2023 Google Inc.",
  "Unknown:MediaWhitePoint": "0.9642 1 0.82491",
  "Unknown:MediaBlackPoint": "0 0 0",
  "Unknown:RedMatrixColumn": "0.43604 0.22249 0.01392",
  "Unknown:GreenMatrixColumn": "0.38512 0.7169 0.09706",
  "Unknown:BlueMatrixColumn": "0.14305 0.06061 0.71391",
  "Unknown:RedTRC": "(Binary data 32 bytes, use -b option to extract)",
  "Unknown:ChromaticAdaptation": "1.04788 0.02292 -0.05019 0.02959 0.99048 -0.01704 -0.00922 0.01508 0.75168",
  "Unknown:BlueTRC": "(Binary data 32 bytes, use -b option to extract)",
  "Unknown:GreenTRC": "(Binary data 32 bytes, use -b option to extract)",
  "Unknown:XMPToolkit": "Adobe XMP Core 5.1.0-jc003",
  "Unknown:Version": 1.0,
  "Unknown:HasExtendedXMP": "EA37C3527D39F4770F52117929217D57",
  "Unknown:MotionPhoto": 1,
  "Unknown:MotionPhotoVersion": 1,
  "Unknown:MotionPhotoPresentationTimestampUs": 1002801,
  "Unknown:DirectoryItemMime": "image/jpeg",
  "Unknown:DirectoryItemSemantic": "Primary",
  "Unknown:DirectoryItemLength": 10121,
  "Unknown:DirectoryItemPadding": 0,
  "Unknown:MPFVersion": "0100",
  "Unknown:NumberOfImages": 2,
  "Unknown:MPImageFlags": "(none)",
  "Unknown:MPImageFormat": "JPEG",
  "Unknown:MPImageType": "Undefined",
  "Unknown:MPImageLength": 10121,
  "Unknown:MPImageStart": 2310901,
  "Unknown:DependentImage1EntryNumber": 0,
  "Unknown:DependentImage2EntryNumber": 0,
  "Unknown:ImageWidth": 4080,
  "Unknown:ImageHeight": 3072,
  "Unknown:EncodingProcess": "Baseline DCT, Huffman coding",
  "Unknown:BitsPerSample": 8,
  "Unknown:ColorComponents": 3,
  "Unknown:YCbCrSubSampling": "YCbCr4:2:0 (2 2)",
  "Unknown:HdrPlusMakernote": "(Binary data 32837 bytes, use -b option to extract)",
  "Unknown:Aperture": 1.9,
  "Unknown:ImageSize": "4080x3072",
  "Unknown:Megapixels": 12.5,
  "Unknown:ScaleFactor35efl": 3.5,
  "Unknown:ShutterSpeed": "1/24",
  "Unknown:SubSecCreateDate": "2023:12:17 19:54:42.599+02:00",
  "Unknown:SubSecDateTimeOriginal": "2023:12:17 19:54:42.599+02:00",
  "Unknown:SubSecModifyDate": "2023:12:17 19:54:42.599+02:00",
  "Unknown:ThumbnailImage": "(Binary data 5075 bytes, use -b option to extract)",
  "Unknown:MPImage2": "(Binary data 10121 bytes, use -b option to extract)",
  "Unknown:CircleOfConfusion": "0.009 mm",
  "Unknown:DOF": "0.09 m (0.33 - 0.43 m)",
  "Unknown:FOV": "73.7 deg",
  "Unknown:FocalLength35efl": "6.8 mm (35 mm equivalent: 24.0 mm)",
  "Unknown:HyperfocalDistance": "2.94 m",
  "Unknown:LightValue": 2.4,
  "Unknown:LensID": "Pixel 6 Pro back camera 6.81mm f/1.85",
  "Copy1:Orientation": "Horizontal (normal)",
  "Copy1:XResolution": 72,
  "Copy1:YResolution": 72,
  "Copy1:ResolutionUnit": "inches",
  "Copy1:DirectoryItemSemantic": "GainMap",
  "Copy1:DirectoryItemMime": "image/jpeg",
  "Copy1:DirectoryItemLength": 4112093,
  "Copy1:MPImageFlags": "(none)",
  "Copy1:MPImageFormat": "JPEG",
  "Copy1:MPImageType": "Baseline MP Primary Image",
  "Copy1:MPImageLength": 2310377,
  "Copy1:MPImageStart": 0,
  "Copy1:DependentImage1EntryNumber": 0,
  "Copy1:DependentImage2EntryNumber": 0,
  "Copy2:ResolutionUnit": "None",
  "Copy2:XResolution": 1,
  "Copy2:YResolution": 1,
  "Copy2:DirectoryItemMime": "video/mp4",
  "Copy2:DirectoryItemSemantic": "MotionPhoto"
}]

Here we can see that Copy2:DirectoryItemLength is indeed missing. however it is there when you search for

"Copy1:DirectoryItemSemantic": "GainMap", "Copy1:DirectoryItemLength": 4112093,

with verbose output this is also visible .\exiftool.exe -v -G4 PXL_20231217_175442599.MP.ultraHDR.jpg

ExifToolVersion = 12.73
  FileName = PXL_20231217_175442599.MP.ultraHDR.jpg
  Directory = .
  FileSize = 6433115
  ZoneIdentifier = Exists
  FileModifyDate = 1705679865.72081
  FileAccessDate = 1705683256.0236
  FileCreateDate = 1705679862.26275
  FilePermissions = 33206
  FileType = JPEG
  FileTypeExtension = JPG
  MIMEType = image/jpeg
JPEG APP1 (6197 bytes):
  ExifByteOrder = II
  + [IFD0 directory with 11 entries]
  | 0)  Make = Google
  | 1)  Model = Pixel 6 Pro
  | 2)  Orientation = 1
  | 3)  XResolution = 72 (72/1)
  | 4)  YResolution = 72 (72/1)
  | 5)  ResolutionUnit = 2
  | 6)  Software = HDR+ 1.0.585804401zd
  | 7)  ModifyDate = 2023:12:17 19:54:42
  | 8)  YCbCrPositioning = 1
  | 9)  ExifOffset (SubDirectory) -->
  | + [ExifIFD directory with 43 entries]
  | | 0)  ExposureTime = 0.041626 (41626/1000000)
  | | 1)  FNumber = 1.85 (185/100)
  | | 2)  ExposureProgram = 2
  | | 3)  ISO = 1550
  | | 4)  ExifVersion = 0232
  | | 5)  DateTimeOriginal = 2023:12:17 19:54:42
  | | 6)  CreateDate = 2023:12:17 19:54:42
  | | 7)  OffsetTime = +02:00
  | | 8)  OffsetTimeOriginal = +02:00
  | | 9)  OffsetTimeDigitized = +02:00
  | | 10) ComponentsConfiguration = 1 2 3 0
  | | 11) ShutterSpeedValue = 4.59 (459/100)
  | | 12) ApertureValue = 1.78 (178/100)
  | | 13) BrightnessValue = -2.59 (-259/100)
  | | 14) ExposureCompensation = 0 (0/6)
  | | 15) MaxApertureValue = 1.78 (178/100)
  | | 16) SubjectDistance = 0.373 (373/1000)
  | | 17) MeteringMode = 2
  | | 18) Flash = 16
  | | 19) FocalLength = 6.81 (6810/1000)
  | | 20) SubSecTime = 599
  | | 21) SubSecTimeOriginal = 599
  | | 22) SubSecTimeDigitized = 599
  | | 23) FlashpixVersion = 0100
  | | 24) ColorSpace = 1
  | | 25) ExifImageWidth = 4080
  | | 26) ExifImageHeight = 3072
  | | 27) InteropOffset (SubDirectory) -->
  | | + [InteropIFD directory with 2 entries]
  | | | 0)  InteropIndex = R98
  | | | 1)  InteropVersion = 0100
  | | 28) SensingMethod = 2
  | | 29) SceneType = 1
  | | 30) CustomRendered = 1
  | | 31) ExposureMode = 0
  | | 32) WhiteBalance = 0
  | | 33) DigitalZoomRatio = 0 (0/1)
  | | 34) FocalLengthIn35mmFormat = 24
  | | 35) SceneCaptureType = 0
  | | 36) Contrast = 0
  | | 37) Saturation = 0
  | | 38) Sharpness = 0
  | | 39) SubjectDistanceRange = 1
  | | 40) LensMake = Google
  | | 41) LensModel = Pixel 6 Pro back camera 6.81mm f/1.85
  | | 42) CompositeImage = 3
  | 10) GPSInfo (SubDirectory) -->
  | + [GPS directory with 3 entries]
  | | 0)  GPSVersionID = 2 2 0 0
  | | 1)  GPSImgDirectionRef = M
  | | 2)  GPSImgDirection = 278 (278/1)
  + [IFD1 directory with 7 entries]
  | 0)  Compression = 6
  | 1)  Orientation = 1
  | 2)  XResolution = 72 (72/1)
  | 3)  YResolution = 72 (72/1)
  | 4)  ResolutionUnit = 2
  | 5)  ThumbnailOffset = 1116
  | 6)  ThumbnailLength = 5075
JPEG APP0 (14 bytes):
  + [BinaryData directory, 9 bytes]
  | JFIFVersion = 1 2
  | ResolutionUnit = 0
  | XResolution = 1
  | YResolution = 1
  | ThumbnailWidth = 0
  | ThumbnailHeight = 0
JPEG APP2 (610 bytes):
  ICC_Profile chunk 1 of 1
  + [ICC_Profile directory with 11 entries, 596 bytes]
  | ProfileHeader (SubDirectory) -->
  | + [BinaryData directory, 128 bytes]
  | | ProfileCMMType =
  | | ProfileVersion = 1024
  | | ProfileClass = mntr
  | | ColorSpaceData = RGB
  | | ProfileConnectionSpace = XYZ
  | | ProfileDateTime = 2023 3 9 10 57 0
  | | ProfileFileSignature = acsp
  | | PrimaryPlatform =
  | | CMMFlags = 0
  | | DeviceManufacturer = GOOG
  | | DeviceModel =
  | | DeviceAttributes = 0 0
  | | RenderingIntent = 0
  | | ConnectionSpaceIlluminant = 0.9642 1 0.82491
  | | ProfileCreator = GOOG
  | | ProfileID = 97 71 53 40 213 170 163 17 225 67 223 201 62 250 162 104
  | 0)  ProfileDescription = sRGB IEC61966-2.1
  | 1)  ProfileCopyright = Copyright (c) 2023 Google Inc.
  | 2)  MediaWhitePoint = 0.9642 1 0.82491
  | 3)  MediaBlackPoint = 0 0 0
  | 4)  RedMatrixColumn = 0.43604 0.22249 0.01392
  | 5)  GreenMatrixColumn = 0.38512 0.7169 0.09706
  | 6)  BlueMatrixColumn = 0.14305 0.06061 0.71391
  | 7)  RedTRC = para..ff...Y...[
  | 8)  ChromaticAdaptation = 1.04788 0.02292 -0.05019 0.02959 0.99048 -0.01704 -0.0092[snip]
  | 9)  BlueTRC = para..ff...Y...[
  | 10) GreenTRC = para..ff...Y...[
JPEG APP1 (1492 bytes):
  + [XMP directory, 1463 bytes]
  | XMPToolkit = Adobe XMP Core 5.1.0-jc003
  | Version = 1.0
  | HasExtendedXMP = EA37C3527D39F4770F52117929217D57
  | [adding XMP-GCamera:MotionPhoto]
  | MotionPhoto = 1
  | [adding XMP-GCamera:MotionPhotoVersion]
  | MotionPhotoVersion = 1
  | [adding XMP-GCamera:MotionPhotoPresentationTimestampUs]
  | MotionPhotoPresentationTimestampUs = 1002801
  | [adding XMP-Container:DirectoryItemMime] (Directory/Item/Mime)
  | DirectoryItemMime = image/jpeg
  | [adding XMP-Container:DirectoryItemSemantic] (Directory/Item/Semantic)
  | DirectoryItemSemantic = Primary
  | DirectoryItemSemantic = GainMap
  | [adding XMP-Container:DirectoryItemLength] (Directory/Item/Length)
  | DirectoryItemLength = 10121
  | DirectoryItemMime = image/jpeg
  | DirectoryItemMime = video/mp4
  | DirectoryItemSemantic = MotionPhoto
  | DirectoryItemLength = 4112093 <----------------------------------------------------------------------------------
  | [adding XMP-Container:DirectoryItemPadding] (Directory/Item/Padding)
  | DirectoryItemPadding = 0
JPEG APP1 (44161 bytes):
JPEG APP2 (86 bytes):
  + [MPF0 directory with 3 entries]
  | 0)  MPFVersion = 0100
  | 1)  NumberOfImages = 2
  | 2)  MPImageList (SubDirectory) -->
  | + [BinaryData directory, 16 bytes]
  | | MPImageFlags = 0
  | | MPImageFormat = 0
  | | MPImageType = 196608
  | | MPImageLength = 2310377
  | | MPImageStart = 0
  | | DependentImage1EntryNumber = 0
  | | DependentImage2EntryNumber = 0
  | + [BinaryData directory, 16 bytes]
  | | MPImageFlags = 0
  | | MPImageFormat = 0
  | | MPImageType = 0
  | | MPImageLength = 10121
  | | MPImageStart = 2258397
  | | DependentImage1EntryNumber = 0
  | | DependentImage2EntryNumber = 0
JPEG DQT (65 bytes):
JPEG DQT (65 bytes):
JPEG SOF0 (15 bytes):
  ImageWidth = 4080
  ImageHeight = 3072
  EncodingProcess = 0
  BitsPerSample = 8
  ColorComponents = 3
  YCbCrSubSampling = 2 2
JPEG DHT (29 bytes):
JPEG DHT (179 bytes):
JPEG DHT (29 bytes):
JPEG DHT (179 bytes):
JPEG SOS
  + [XMP directory, 44086 bytes]
  | HdrPlusMakernote = SERSUAPvZDVtXnAeLOrjTHc0Z6fIof306N5abuHeYp788T3k4EvBiFCSkvaWwVgm[snip]
  MPImage2 = SCALAR(0x41f438c)
pulsejet commented 7 months ago

Thanks for the legwork @eikaramba.

pulsejet commented 7 months ago

Fixed by https://github.com/pulsejet/memories/commit/e6760c7452915a98dfe6c230c4c466579678a75e

eikaramba commented 7 months ago

I was also fixing it, this is my code:

foreach ($extExif as $key => $value) {
                    if (str_ends_with($key, ':DirectoryItemSemantic')) {
                        if ('MotionPhoto' === $value || 'GainMap' === $value) { // GainMap according to https://developer.android.com/media/platform/hdr-image-format#locate_the_gain_map_image
                            if (isset($extExif[str_replace('Semantic', 'Length', $key)])) { //prevent undefined array key error
                                $videoLength = $extExif[str_replace('Semantic', 'Length', $key)];
                                if (\is_int($videoLength) && $videoLength > 0) {
                                    $videoOffset = $file->getSize() - $videoLength;

                                    return "self__traileroffset={$videoOffset}";
                                }
                            }
                        }
                    }
                }

i guess yours works as well :)

pulsejet commented 7 months ago

The issue is Google decided to exclude the DirectoryItemLength for the Primary semantic (I've little understanding of how exactly this work) and exiftool doesn't provide an API to semantically group these. So we're left to do stupid guesswork.