search

tommyblue / smugmug-backup

Makes a full backup of a SmugMug account
MIT License
88 stars 16 forks source link

Rerunning a backup downloads unmodified images again #59

Open Solliko opened 1 month ago

Solliko commented 1 month ago

Hello Tommaso,

Rerunning a backup re-downloads unmodified images and overwrites identical files already on disk.

Have been using this wonderful app for years and this has never happened before. To verify I downloaded a fresh copy of my smugmug account (about 400 GB) which completed successfully in 2 hours. Then I ran it again and it proceeded to download files again.

Using version 1.5.0 from Aug 2023 on MacOS 14.6.
Here is my config.toml file:

[store] destination = "/Volumes/Photos Backup/Album" file_names = "{{.FileName}}" use_metadata_times = true force_metadata_times = true concurrent_downloads = 10 concurrent_albums = 10

tommyblue commented 1 month ago

@Solliko does it download everything every time or just some photos?

I noticed that on my account recently, it kept downloading a bunch of photos. After some investigation I found out that those photos were duplicated in their folders (mostly they were photos I rotated with Preview) and they had different sizes. Since the APIs return albums' photos in random order, every time the order changed then the size check failed and they were downloaded again. I removed the duplicated photos and everything is clean now.

Can you check if you're keeping downloading the photos with the same name?

Solliko commented 1 month ago

I just ran it again and it's currently downloading from multiple folders. Some folders were created last month and had files moved to them, some folders were created 2 years ago and have not changed since.

This is on a new backup created today.

Solliko commented 1 month ago

I have let it run for 20 minutes now and it appears to be re-downloading everything (60GB so far). I see it downloading unmodified folders from 2019.

Also, I tried to re-open issue #15. I left you a new message there. Thanks!

tommyblue commented 1 month ago

2 questions: which version are you using? Can you paste some logs so I can try to understand what's going on?

Solliko commented 1 month ago

Using version 1.5.0 from Aug 2023 on MacOS 14.6. How do I create logs?

tommyblue commented 1 month ago

Using version 1.5.0 from Aug 2023 on MacOS 14.6. How do I create logs?

Ok, the last version is 1.5.5 but nothing should change about what you're facing now. Logs are printed to console while running the program. How do you run it? If you run it from terminal you see them