search

gilesknap / gphotos-sync

Google Photos and Albums backup with Google Photos Library API
Apache License 2.0
1.97k stars 161 forks source link

Shared photos are removed and then re-added in version 2.14.2 #444

Closed Phusty closed 6 months ago

Phusty commented 9 months ago

Hi, I'm using your library and it's great! I'm specifically using the latest stable version available on snapcraft (2.14.2), which I run with cron once a night with the following parameters: --skip-shared-albums --do-delete --flush-index --omit-album-date. Recently, I've noticed that some photos shared with an album of mine, which are correctly downloaded by the library, are at some point removed and then re-added the next time the library is triggered (the day after, as I have set up cron). This also happens for old albums that have not been modified.

Below you can see an excerpt from the logs where the same photo is first deleted and then re-added.

09-23 01:02:48 gphotos.GooglePhotosIndex WARNING gphoto/photos/2022/08/IMG_20220809_105755.jpg deleted 09-23 01:02:48 gphotos.GooglePhotosIndex WARNING gphoto/photos/2022/08/IMG_20220809_105753.jpg deleted 09-23 01:02:48 gphotos.GooglePhotosIndex WARNING gphoto/photos/2022/08/IMG_20220814_143948.jpg deleted 09-23 01:02:48 gphotos.GooglePhotosIndex WARNING gphoto/photos/2022/08/IMG_20220809_105811.jpg deleted 09-23 01:02:48 gphotos.GooglePhotosIndex WARNING gphoto/photos/2022/08/IMG_20220814_144116.jpg deleted

and

09-24 01:09:23 gphotos.GooglePhotosDownload DEBUG COMPLETED 929 downloading photos/2022/08/IMG_20220809_105753.jpg 09-24 01:09:28 gphotos.GooglePhotosDownload DEBUG COMPLETED 943 downloading photos/2022/08/IMG_20220809_105755.jpg 09-24 01:09:28 gphotos.GooglePhotosDownload DEBUG COMPLETED 944 downloading photos/2022/08/IMG_20220809_105811.jpg 09-24 01:09:28 gphotos.GooglePhotosDownload DEBUG COMPLETED 950 downloading photos/2022/08/IMG_20220814_143948.jpg 09-24 01:09:28 gphotos.GooglePhotosDownload DEBUG COMPLETED 952 downloading photos/2022/08/IMG_20220814_144116.jpg

Since I'm using an old version of your library, but a newer stable version on snapcraft is not available, I was wondering if this is a known issue and if it is fixed in the new version. I also wanted to ask if you plan to release the new version on snap or if you recommend that I install it in another way. Thank you

gilesknap commented 9 months ago

Hi @Phusty.

The settings you are using mean that you are completely rescanning your whole library every night. Including removing any local copies of your files that were deleted online.

That's not how I use gphotos-sync because it means that accidental deletions would get propagated to my backup. But then if you are taking snapshots of your filesystem by other means then this would be a good way to notice such deletions I guess.

Anyway. I can't comment on ever seeing this and I'm inclined to blame Google for this one. There is an outstanding bug that delivers a less items per chunk than requested, sometimes even zero, sometimes zero several times before picking up again. See the last bullet on #119. My code tries to workaround this by retrying even though zero response is supposed to mean that the search is complete.

Since it's a bug in the server and has not been fixed for 4 years I'm not holding out much hope.

Do you need to use --do-delete all the time. Could you change your workflow a little to work around this?

Phusty commented 9 months ago

Hi,

I was using the library with --do-delete to keep the backup aligned with the photos on Google, and I set up another script that makes a tar of a folder if something changes, notifying me on Telegram of the previous and current number of files (which is why I noticed this behavior) precisely to avoid accidental deletions from the backup.

I will try to modify the workflow in order to be sure that the backup only occurs when I have already deleted, among the latest photos taken, those that I do not want to keep.

Thank you very much for your response!

gilesknap commented 9 months ago

makes a tar of a folder if something changes, notifying me on Telegram of the previous and current number

Nice - in fact really the only way to use gphotos-sync as Google has crippled the API - its not a true backup but it is a good way to make sure accidental deletions are not happening.

Of course being as diligent as you are you will catch out little bugs like the one I mention in the Google API.

gilesknap commented 6 months ago

closing as completed