Downloading Deleted Files... only to delete them?

[michaelkrieger]

image: boredazfcuk/icloudpd latest (pushed 19 days ago).

With the latest update I've noticed a large number of deleted images- many more than I would ever delete each day (60-80).

Looking at the logs, icloudpd is downloading images, and in the same run deleting the very same files it just downloaded (by file name). The logs have nothing interesting in the preamble or file list, except, after it reviews the deduplicated files and downloaded files, it reviews the deleted files and deletes a bunch of files it just downloaded (about 60-80 each time).

In summary:

• I add 30-40 photos to my phone
• icloudPD downloads 122
• icloudPD deletes 62
• The next day it downloads the same deleted images and re-deletes them.

Why is it downloading the recently deleted images and subsequently deleting them?

[michaelkrieger]

Posted to https://github.com/icloud-photos-downloader/icloud_photos_downloader/issues/511 as well. Not sure which would be the culprit.

From the sounds of it there, it would be in the docker scripts.

[boredazfcuk]

Hi,

Your iDevice stores photos in a database, so it can have multiple photos with the same name. icloudpd stores them in a standard file structure, so it's possible to have photos with the same name in the photo stream and also in the deleted items folder. icloudpd has no way of differentiating the two, so it will see a file in the cloud, for example IMG_1234.HEIC, and download it. After the download run is complete, it processes deletions, where it sees that there is a photo with the same name in Recently Deleted, so it removes the file it just downloaded.

I've also seen this behaviour of downloading and deleting files immediately with the Telegram messenger application. If you save a file from that app to the photo stream, it will save it as something like camphoto_1234567.jpg. But for some reason it will also detect it as a deleted photo. This results in it being deleted and downloaded each time.

I noticed that you said you "add" photos to your phone... Are you adding them via some method which isn't the camera on your phone? If they are bing added by an app such as Telegram, then it could be that which is causing the issue,

[michaelkrieger]

@boredazfcuk , this issue continues for me.

Looking at the logs, this continues each and every day (since my original post).

These files should have long expired from the "Deleted Items" folder, so I find the theory that there is something matching this in the deleted items folder hard to agree to. My phone is producing items in the 5000's at this point and adding them to iCloud via the usual iCloud sync. These photos are mostly a bit older.

Some of the ones it is saying it's deleting are files that haven't been touched in years. This only started with the latest upgrade in late 2022 and never happened before that. In some cases, they are HEVC.MOV with the related HEIC near it.

As you can see from the below example, it downloads 2022/IMG_2891.HEIC. It then deletes it 20 minutes later. When I look at the files on-disk, there does appear to be one with the -1951428 in this case (though that's not the same as all of them).

In all cases, it appears to be movies or similar files in prior years.

2023-04-27 05:27:24 INFO     Downloading /home/michael/iCloud/2022/IMG_2891.HEIC
2023-04-27 05:27:24 INFO     /home/michael/iCloud/2022/IMG_2891_HEVC.MOV already exists.
2023-04-27 05:36:40 INFO     /home/michael/iCloud/2021/IMG_2891.PNG already exists.
2023-04-27 05:46:52 INFO     /home/michael/iCloud/2018/IMG_2891.HEIC already exists.
2023-04-27 05:48:29 INFO     Deleting /home/michael/iCloud/2022/IMG_2891.HEIC!

After the run, this is the state of files including 2891

./2018/IMG_2891.HEIC
./2021/IMG_2891.PNG
./2022/IMG_2891_HEVC.MOV
./2022/IMG_2891-1951428.HEIC

I've deleted all files matching the pattern (ie 20??/IMG_2891*) and will see what they do on a re-download. I'm hopeful maybe some stray file on disk is causing this, but if it re-downloads these same files and then re-deletes them, I'd suggest maybe a more complex method may be required to find deleted items?

[mariomaz87]

Hi, I'm having the same issues with 3 photos that are downloaded and deleted every sync. The issue started when I decided to sync also my Shared Library and not only my Personal Library. The 3 photos have been moved from one library to the other, so I think this is causing the issue. Did you manage to fix it somehow? Thanks!