[Bug] Removing orphant LivePhoto video file does not re-upload it from iOS

mkarnebeek commented 7 months ago

The bug

After initially uploading all my photo's from iPhone, the repair dashboard mentioned about 170 offline paths. After running https://immich.app/docs/guides/remove-offline-files/ I expected the iOS app to start re-uploading the now missing assets, but it didn't. These all turned out to be the movie-part of LivePhotos.

The OS that Immich Server is running on

Debian GNU/Linux 12

Version of Immich Server

v1.99.0

Version of Immich Mobile App

1.99.0 build.145

Platform with the issue

[ ] Server
[X] Web
[X] Mobile

Your docker-compose.yml content

From https://immich.app/docs/install/docker-compose

Your .env content

From https://immich.app/docs/install/docker-compose

Reproduction steps

I managed to reproduce this by, on web:

going to a recent live photo,
looking up the livePhotoVideoId asset id, looking up the filename from that video asset, and removing that file from the filesystem
having the video-asset recognized by the python script at https://immich.app/docs/guides/remove-offline-files/, and having the video-asset removed.
Immich web now showing just the picture only, and now showing livePhotoVideoId=null for that photo.

Then on Mobile (iOS)

Re-opening and even re-installing the iOS app didn't upload the live photo video asset

Then on Web:

Deleting the photo from web and then also from trash

Then on Mobile (iOS)

re-opening the iOS app re-uploaded both the image and video asset of the LivePhoto

Additional information

No response

mkarnebeek commented 7 months ago

As a workaround, I'm thinking of removing all image-assets without a livePhotoVideoId (as I have almost all of my photo's with live photo's) to have the app trigger uploading of these missing live photo's now.

alextran1502 commented 7 months ago

I think this is the mechanism we want, since you will have to delete the entire LivePhotos, both motion and image page, to have the file reuploaded again

mkarnebeek commented 7 months ago

I can imagine than. Makes sense. But how do we handle upload failures of the movie-asset of LivePhoto's?

One way to handle that is that https://immich.app/docs/guides/remove-offline-files/ checks if it is a video-asset of a live photo, it also removes the parent image-asset, right? But there isn't a link back from that video-asset to the image-asset, at least, I couldn't find a field on the video-asset pointing to the image-asset

I did notice, the image-asset gets its livePhotoVideoId field null'ed when I delete the video-asset. This is probably handled by the database, and not something Immich itself is aware of, right?

mmomjian commented 7 months ago

Your questions asks "how do we handle upload failures of the movie-asset of LivePhoto's", which confuses me. Failed uploads, in my experience, always appear as untracked files in the upload folder, not offline files. In the case that an upload fails, as far as I know Immich will continue trying until the asset is uploaded.

mkarnebeek commented 7 months ago

Uhm.. Good point. That was an assumption on my end. I based that on the app needing multiple days to upload everything because it uploaded only when I had the app open. iOS interrupting the upload process by putting the app to sleep or something.

Then I don't know how those assets got to be offline.

I did manage to modify the python script so it queries all assets in Immich, filters the non-live image-type assets and deletes those. The app then started re-uploading the assets, including the video-assets of the live photo's. So, that sorta worked.

I might just try to do the whole upload process again, just to see if I can get it to break again, because the cause currently is unknown for me.

Looking at the "I think this is the mechanism we want": A question: Do we want Immich in the future to be able to recover from this? If the video-asset is a separate asset as it is, one could argue that the mobile app / immich should detect it to be missing on the server, and re-upload?

jum0n commented 6 months ago

Your questions asks "how do we handle upload failures of the movie-asset of LivePhoto's", which confuses me. Failed uploads, in my experience, always appear as untracked files in the upload folder, not offline files. In the case that an upload fails, as far as I know Immich will continue trying until the asset is uploaded.

I dont think missing files will be re-uploaded. I have 45 untracked files in uploads with unique file names. I cannot find any of these files elsewhere in immich file structure, so believe they are the only copy that was sent by my phone. I removed one of the untracked files from the file system but immich client does not upload it again.

immich-app / immich