search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
45.08k stars 2.18k forks source link

[BUG] iPhoto/Apple Photos Sidecar Metadata Not Being Read On Upload #3383

Closed thegranddesign closed 1 year ago

thegranddesign commented 1 year ago

The bug

I see that there is a lot of work being done around metadata parsing on upload so this may be already tackled (I did look into many of the other issues).

My issue is that I did an export from iPhoto/Apple Photos and my library contains 20k+ images. All of them have already been tagged, descriptions added, lat/longs attached, faces detected, and custom dates assigned (note this is NOT the creation date but the date I've manually given them).

Obviously I can't go back and redo all that work. :)

When Photos exports the data, it writes an XMP file next to the image with all the information. For some images there is also data in the image (for example if it was a photo from a phone versus something that was scanned in). The data is retrieved from the images ok, but the XMP data should take priority. Always.

I have tried both the web uploader as well as the bulk uploader (which, other than the metadata issue, is amazing ;) and in both cases it appears as if the XMP file is ignored.

The format of the XMP file that Apple Photos exports looks a bit odd but exiftool does a decent job of getting the data out.

I'd be happy to supply some of the formatting of the exported XMP files if that would help.

Am I missing something? Is this feature not implemented yet? Is it a bug? I wasn't sure.

Thank you for such an amazing product. If I can get this working it will be exactly what I need.

The OS that Immich Server is running on

N/A

Version of Immich Server

v1.68.0

Version of Immich Mobile App

N/A

Platform with the issue

Your docker-compose.yml content

N/A

Your .env content

N/A

Reproduction steps

1. Modify Photo Date/Time/Tags/Description/Etc In Apple Photos
1. Export Photos From Apple Photos
2. Import Photos Into Immich
3. Note that the custom information does not populate in Immich
...

Additional information

No response

ericswpark commented 1 year ago

The server backend and CLI is capable of ingesting XMP sidecar files, but the clients haven't been updated to upload them. I suggest closing this issue and following this one instead: https://github.com/immich-app/immich/issues/1941

alextran1502 commented 1 year ago

Duplicated of #1941

thegranddesign commented 1 year ago

I have tried both the web uploader as well as the bulk uploader (which, other than the metadata issue, is amazing ;) and in both cases it appears as if the XMP file is ignored.

@ericswpark @alextran1502 I literally said I was using the CLI bulk uploader. Please reopen.

ericswpark commented 1 year ago

@thegranddesign sorry, I completely missed that part. Then either the CLI tool isn't uploading the XMP files or the server is ignoring them. Have you checked the number of files that get uploaded with the CLI tool?

jrasm91 commented 1 year ago

Uploading via the mobile app is a duplicate, as already mentioned. Uploading via the CLI is already supported, given the sidecar file is the same name as the original (with the additional .xmp extension).

Can you be more specific what exactly is not working for you that hasn't already been addressed? Do your xmp files follow the naming convention we use to automatically detect them?

ericswpark commented 1 year ago

@jrasm91 I just realized the CLI tool didn't upload any of my XMP files either, so some of my photos that have had the time/date modified all show up under the date I uploaded the photos. Taking a look at the file name, it looks like Immich is expecting the following:

IMG_0321.jpg
IMG_0321.jpg.xmp

However, Apple Photos exports with the .jpg extension removed from the file name of the XMP sidecar file, like so:

IMG_0321.jpg
IMG_0321.xmp

Does Immich back up the XMP files in this case?

jrasm91 commented 1 year ago

Ah that makes more sense. No, we only look for the added .xmp extension, like in your first example.

You could update your sidecar files to use this format or open a feature request to support this additional one as well.

ericswpark commented 1 year ago

Made a FR in https://github.com/immich-app/immich/discussions/3407

thegranddesign commented 1 year ago

Thanks guys. I can batch rename the XMP files in the meantime I think. It'll take some trial and error but should be faster than waiting for the FR. I appreciate you all looking into this. <3

thegranddesign commented 1 year ago

I did this and it looks like further Apple Photo/iPhoto compatibility is required. The XMP files utilize a Ref field for the GPS coordinates that Immich currently does not honor. https://github.com/immich-app/immich/discussions/3407