search

mandarons / icloud-docker

Dockerized iCloud Client - make a local copy of your iCloud documents and photos, and keep it automatically up-to-date.
BSD 3-Clause "New" or "Revised" License
1.07k stars 49 forks source link

Some already downloaded files are downloading again on every pass... #76

Closed andressoler closed 2 years ago

andressoler commented 2 years ago

First things first... Thanks so much @mandarons, this project is what I've looking for since years... I appreciate so much your efforts on this...

Describe the bug Some files are downloaded on every pass, even if the file already exists and no changes have occurred on it... Usually this issue is related to MacOS packages, which are downloaded as a file, instead of as a directory, but I have also noticed that it applies to many photos...

To Reproduce In my case, this behavior seems to be always the same, although I'm not sure you know which files it occurs with and which it doesn't.

Expected behavior I expect not to download any file if it is already downloaded and it has not changed...

Screenshots You can see some examples on "icloud.log" where unmodified files are downloaded several times (PHOTOS sync_interval is lower than DRIVE one): 

/app # more icloud.log |grep LOGO-Contacto
2022-02-26 17:02:56,650 :: INFO :: root :: sync_drive.py :: 90 :: Downloading /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm ...
2022-02-27 10:33:47,321 :: INFO :: root :: sync_drive.py :: 90 :: Downloading /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm ...
2022-02-28 04:38:14,972 :: INFO :: root :: sync_drive.py :: 90 :: Downloading /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm ...
/app # ls -la /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm 
----------    1 root     root        899758 Sep 21  2018 /app/icloud/drive/Pixelmator/LOGO-Contacto.pxm

/app # more icloud.log |grep "Imagen 175.JPG"
2022-02-26 23:20:18,431 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-26 23:47:35,904 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 00:59:13,071 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 17:27:57,205 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 17:50:31,021 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-27 18:41:28,510 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-28 10:42:34,244 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-28 11:05:29,406 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
2022-02-28 11:56:30,420 :: INFO :: root :: sync_photos.py :: 37 :: Downloading /app/icloud/photos/all/Imagen 175.JPG ...
/app # ls -la "/app/icloud/photos/all/Imagen 175.JPG"
----------    1 root     root        735560 Mar 13  2011 /app/icloud/photos/all/Imagen 175.JPG

Configuration My config YAML is:

app:
  logger:
    level: "info"
    filename: "icloud.log"
  credentials:
    username: "MY-ID"
    retry_login_interval: 600
  root: "icloud"
  smtp:
drive:
  destination: "drive"
  remove_obsolete: true
  sync_interval: 3000
  filters:
    folders:
    file_extensions:
photos:
  destination: "photos"
  remove_obsolete: true
  sync_interval: 300
  filters:
    albums:
    file_sizes:
      - "original"
github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

andressoler commented 2 years ago

Hello @mandarons...

Have you had a chance to take a look on this?

Thanks so much!!!

mandarons commented 2 years ago

That's weird. Files code compares date modified and size (local vs remote) to determine if file should be downloaded or not. For Photos, it compares file size (local vs remote).

I will see if I can reproduce it (somehow).

andressoler commented 2 years ago

Thanks @mandarons!

Yes, I agree, it is weird... I supposed your code compares date and size... I have been researching a little bit, and I can assert when the files are re-downloaded, they remain with the same size and date...

I am Spanish (located in Spain)... I was wondering if some local settings are involved here, like date format or special characters (I don't think so but, who knows), but I am not able to find any clue...

By the moment, I think your application is super useful for me... I you fix it, I think it will be perfect...

Thanks again and congrats!!!

mandarons commented 2 years ago

Thanks, @andressoler. Let's log more information to see what's exactly happening. I will create a dev build with more logging, for you. Which docker image are you using - linux/386, linux/amd64, linux/arm/v7 or linux/arm64?

andressoler commented 2 years ago

Hey @mandarons... That sounds great!!!

I'm running amd64 version on a Synology using docker-compose: 

    iCloud-Sync:
      image: mandarons/icloud-drive
      environment:
        - PUID=0
        - GUID=0
        - TZ=Europe/Madrid
      container_name: iCloud-Sync
      restart: unless-stopped
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - /volume1/docker/SERVICES/iCloudDocker/andres/config.yaml:/app/config.yaml
        - /volume1/Nubes/iCloud/Andres:/app/icloud
        - /volume1/docker/SERVICES/iCloudDocker/andres/session_data:/app/session_data

Those are the instance details:

Size: 86.4 MB
Created: 2022-02-21 18:58:43
Build: Docker on linux, amd64

Thanks!

mandarons commented 2 years ago

@andressoler the build is ready. Please use docker image tag mandarons/icloud-drive:main, set the logger > level to debug in config.yaml file. Share the logs here.

andressoler commented 2 years ago

Hello @mandarons,

I have just run the last version for about an hour (I have user another account with less files, just for simplify)...

I share the logs below:

iCloud_logs.txt

In my opinion, I think there is a problem comparing remote and local file size... I am not sure why... These are the changes so far:

/app # more icloud.log |grep Changes
2022-04-14 11:21:04,669 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1584701632, remote_modified_time is 1584701632, local_file_size is 0 and remote_file_size is None.
2022-04-14 11:21:05,406 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1596032480, remote_modified_time is 1596032480, local_file_size is 231121527 and remote_file_size is 413796189.
2022-04-14 11:23:11,691 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1645269246, remote_modified_time is 1645269246, local_file_size is 13294053 and remote_file_size is 15042805.
2022-04-14 11:23:20,188 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1647360717, remote_modified_time is 1647360717, local_file_size is 224414 and remote_file_size is 409474.
2022-04-14 11:23:23,961 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1647360677, remote_modified_time is 1647360677, local_file_size is 1063001 and remote_file_size is 1214759.
2022-04-14 11:23:26,572 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1614718389, remote_modified_time is 1614718389, local_file_size is 0 and remote_file_size is None.
2022-04-14 11:25:18,948 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1584701632, remote_modified_time is 1584701632, local_file_size is 0 and remote_file_size is None.
2022-04-14 11:25:19,738 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1596032480, remote_modified_time is 1596032480, local_file_size is 231121527 and remote_file_size is 413796189.
2022-04-14 11:27:14,530 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1645269246, remote_modified_time is 1645269246, local_file_size is 13294054 and remote_file_size is 15042805.
2022-04-14 11:27:21,356 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1647360717, remote_modified_time is 1647360717, local_file_size is 224414 and remote_file_size is 409474.
2022-04-14 11:27:24,324 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1647360677, remote_modified_time is 1647360677, local_file_size is 1063002 and remote_file_size is 1214759.
2022-04-14 11:27:26,575 :: DEBUG :: root :: sync_drive.py :: 85 :: Changes detected: local_modified_time is 1614718389, remote_modified_time is 1614718389, local_file_size is 0 and remote_file_size is None.

I hope this makes sense for you...

mandarons commented 2 years ago

It looks like remote file size is always greater than local file size. This means downloading is not getting completed for some reason. In second run, it looks like size of local file is increased only by 1 byte for some of the files.

Can you please check if all of these local files open fine? Also, can you download these files from iCloud.com and see if their local file size is exactly same as remote file size?

andressoler commented 2 years ago

Ok, that is what I found so far...

These are my local files (size is not the same as remote):

$ ls -la
total 14244
drwxrwxrwx+ 1 root root       130 Apr 15 09:11  .
drwxrwxrwx+ 1 root root       260 Apr 15 09:11  ..
-rwxrwxrwx+ 1 root root  13294054 Feb 19 12:14  20210926_123137162.band
drwxrwxrwx+ 1 root users       72 Apr 15 09:11  @eaDir
-rwxrwxrwx+ 1 root root   1063004 Mar 15 17:11 'Mi canción 2.band'
-rwxrwxrwx+ 1 root root    224415 Mar 15 17:11 'Mi canción.band'

They cannot be opened... They appear as corrupt...

The funny thing is when I try to download them from iCloud Web, as they are a package (instead of a file), iCloud download it as a ZIP file... Wen I unzip it locally, my Mac recognizes it as a package, and I can open it without problems...

A Mac OS Package (as far as I know) is a folder with some files inside... I am not sure if it can be download AS IS in a linux machine, or it must be downloaded as a ZIP file... By the moment, it seems that the application is trying to download it as a whole file...

What do you think? Am I missing something?

mandarons commented 2 years ago

Looks like problem is with .band files. Are there any extensions that you see problematic? Do you have sample files that you can share with me to reproduce the issue on my side? Or how can I create them?

andressoler commented 2 years ago

Hello @mandarons!

Sorry for the delay in the response...

I think the problem is related with all MacOS "package" files, that are really a folder with some files within it and iCloud website downloads them as a ZIP (I attach a Pixelmator example): 393066_00.pxm.zip

Regards!!!

mandarons commented 2 years ago

Thanks for sharing. I will try to reproduce it on my end.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

mandarons commented 2 years ago

@andressoler : are you still seeing this issue?

andressoler commented 2 years ago

@mandarons: Yes, unfortunately i'm still seeing it for some packages...

mandarons commented 2 years ago

@andressoler : I am able to reproduce the issue now. For some reason, downloaded package files are always less than the one on server. I am looking into the fix now.

mandarons commented 2 years ago

@andressoler : please try with docker pull mandarons/icloud-drive:main and see if that fixes the issue. I will make a new official release soon.

andressoler commented 2 years ago

@mandarons, I think something is going wrong here...

Have a look to the logs (some dummy songs from my son's iPhone saved with GarageBand for iOS):

2022-07-18 11:19:25,860 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/Downloads/Mi canción.band ...
2022-07-18 11:19:31,792 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/Downloads/Mi canción.band: /app/icloud/drive/Downloads/Mi canción.band.zip is not a zip file
2022-07-18 11:19:37,495 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/Downloads/www.sneakersales.es ...
2022-07-18 11:19:37,524 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/Downloads/www.sneakersales 2.es ...
2022-07-18 11:19:40,142 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/20210129_201905366.band ...
2022-07-18 11:19:47,787 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/20210129_201905366.band: /app/icloud/drive/GarageBand for iOS/20210129_201905366.band.zip is not a zip file
2022-07-18 11:19:51,185 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band ...
2022-07-18 11:20:01,325 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band: /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band.zip is not a zip file
2022-07-18 11:20:03,685 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band ...
2022-07-18 11:20:17,181 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band: /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band.zip is not a zip file
2022-07-18 11:20:19,589 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Jujutsu.band ...
2022-07-18 11:20:34,836 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Jujutsu.band: /app/icloud/drive/GarageBand for iOS/Jujutsu.band.zip is not a zip file
2022-07-18 11:20:38,671 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción.band ...
2022-07-18 11:20:43,129 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción.band: /app/icloud/drive/GarageBand for iOS/Mi canción.band.zip is not a zip file
2022-07-18 11:20:45,913 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 2.band ...
2022-07-18 11:20:50,768 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 2.band: /app/icloud/drive/GarageBand for iOS/Mi canción 2.band.zip is not a zip file
2022-07-18 11:20:53,031 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...
2022-07-18 11:20:58,988 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 3.band: /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.zip is not a zip file
2022-07-18 11:21:03,841 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 4.band ...
2022-07-18 11:21:11,500 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 4.band: /app/icloud/drive/GarageBand for iOS/Mi canción 4.band.zip is not a zip file
2022-07-18 11:21:13,805 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 5.band ...
2022-07-18 11:21:59,797 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 5.band: /app/icloud/drive/GarageBand for iOS/Mi canción 5.band.zip is not a zip file
2022-07-18 11:22:01,687 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 6.band ...
2022-07-18 11:22:05,144 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 6.band: /app/icloud/drive/GarageBand for iOS/Mi canción 6.band.zip is not a zip file
2022-07-18 11:22:07,051 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 7.band ...
2022-07-18 11:22:10,951 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 7.band: /app/icloud/drive/GarageBand for iOS/Mi canción 7.band.zip is not a zip file
2022-07-18 11:22:18,115 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/Mi canción.band ...
2022-07-18 11:22:30,134 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/Mi canción.band: /app/icloud/drive/Mi canción.band.zip is not a zip file
2022-07-18 11:22:32,492 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/Mi canción 2.band ...
2022-07-18 11:22:36,477 :: ERROR :: root :: sync_drive.py :: 151 :: Failed to download /app/icloud/drive/Mi canción 2.band: /app/icloud/drive/Mi canción 2.band.zip is not a zip file
2022-07-18 11:22:37,907 :: INFO :: root :: sync_drive.py :: 141 :: Downloading /app/icloud/drive/Shortcuts/.WorkflowHiddenFile ...
2022-07-18 11:22:37,910 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/Mi canción.band.zip ...
2022-07-18 11:22:37,912 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/Mi canción 2.band.zip ...
2022-07-18 11:22:37,915 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/Downloads/Mi canción.band.zip ...
2022-07-18 11:22:37,917 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/20210129_201905366.band.zip ...
2022-07-18 11:22:37,918 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band.zip ...
2022-07-18 11:22:37,918 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band.zip ...
2022-07-18 11:22:37,918 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Jujutsu.band.zip ...
2022-07-18 11:22:37,921 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción.band.zip ...
2022-07-18 11:22:37,921 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 2.band.zip ...
2022-07-18 11:22:37,921 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.zip ...
2022-07-18 11:22:37,922 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 4.band.zip ...
2022-07-18 11:22:37,922 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 5.band.zip ...
2022-07-18 11:22:37,923 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 6.band.zip ...
2022-07-18 11:22:37,924 :: INFO :: root :: sync_drive.py :: 179 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 7.band.zip ...

It seems like it tries to download a zip version of the package (which is ok), but something does not match and finally it removes all files...

It repeats the same process again on the next try...

mandarons commented 2 years ago

@andressoler : That's weird. I tried with a sample Project.band.zip file, the file your shared 393066_00.pxm.zip. It seems to work. But in your case, it is failing to unzip.

Is it working okay for other 'package' files? Can you share one of the failing files?

andressoler commented 2 years ago

Hummm... This is very weird... I agree...

That's what I found:

1- Application downloads zip files (I attached one of them)... I can unzip it from my Mac, and even I can open it and use it... So application seems to work fine... 2- I have noticed that some photos are downloaded again (this is a strange behavior, because pictures are identical)

20210129_201905366.band.zip

Those are the files that are downloaded before it deletes them...

ash-4.4# ls -la
total 102812
d---------+ 1 root root      562 Jul 19 09:23  .
d---------+ 1 root root      164 Jul 19 09:23  ..
----------+ 1 root root  6670477 Jul 19 09:21  20210129_201905366.band.zip
----------+ 1 root root  5607575 Jul 19 09:21 '20220124_221641098 (1).band.zip'
----------+ 1 root root  2270060 Jul 19 09:21 '20220409_003913347 (1).band.zip'
d---------+ 1 root root        0 Mar  2 14:30 'carpeta sin título'
----------+ 1 root root 39676813 Jul 19 09:21  Jujutsu.band.zip
----------+ 1 root root  1659035 Jul 19 09:22 'Mi canción 2.band.zip'
----------+ 1 root root  7415130 Jul 19 09:22 'Mi canción 3.band.zip'
----------+ 1 root root   154699 Jul 19 09:22 'Mi canción 4.band.zip'
----------+ 1 root root 37940032 Jul 19 09:22 'Mi canción 5.band.zip'
----------+ 1 root root   177981 Jul 19 09:22 'Mi canción 6.band.zip'
----------+ 1 root root  2037647 Jul 19 09:23 'Mi canción 7.band.zip'
----------+ 1 root root  1644202 Jul 19 09:22 'Mi canción.band.zip'

I also get this error (I don't remember to see it before)... I am not sure if it is important:

2022-07-19 09:20:25,602 :: WARNING :: root :: config_parser.py :: 288 :: app > region not found. Using default value - global ...
mandarons commented 2 years ago

@andressoler Thank you. I updated the main branch not to delete the zip file if extraction fails. This will keep the ball rolling. For some reason, package files created on iOS are zipped differently than package files created on Mac OS. I am seeking help on stackoverflow.com. Let's see.

Thank you very much for your patience and co-operation, @andressoler, to solve this problem. I appreciate it.

mandarons commented 2 years ago

@andressoler : I have the fix now (thanks to StackOverflow community). I will make another release for you to test, soon.

andressoler commented 2 years ago

That sounds great @mandarons!!... I will be here to help you to improve this amazing project!!! Thanks!!!

mandarons commented 2 years ago

@andressoler - okay, main branch is ready for you with the fix. Please give it a try and let me know if it fixes the issue.

andressoler commented 2 years ago

Hummm... Am I doing something wrong? Have a look to my docker log:

Loading config from /app/config.yaml ...
2022-07-21 17:52:57,107 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Gnome
2022-07-21 17:52:57,140 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Google
2022-07-21 17:52:57,143 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Windows (alt)
2022-07-21 17:52:57,146 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading file
2022-07-21 17:52:57,147 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading keyczar
2022-07-21 17:52:57,147 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading multi
2022-07-21 17:52:57,147 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading pyfs
Traceback (most recent call last):
  File "/app/./src/main.py", line 3, in <module>
    from src import sync
  File "/app/src/sync.py", line 14, in <module>
    from src import sync_drive, sync_photos
  File "/app/src/sync_drive.py", line 8, in <module>
    import magic
  File "/app/venv/lib/python3.9/site-packages/magic/__init__.py", line 209, in <module>
    libmagic = loader.load_lib()
  File "/app/venv/lib/python3.9/site-packages/magic/loader.py", line 49, in load_lib
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation
mandarons commented 2 years ago

@andressoler : please try now with mandarons/icloud-drive:main.

uwe-schwarz commented 2 years ago

I tried :main, but it is too slow to test a second run (at least with my normal config, downloading basically all of my drive). It now downloads all pages/numbers/swiftpm packages and extracts them. IMO it would be better to have an option to leave them as zip (as before) and don't use file size for comparison, but just modification time?

mandarons commented 2 years ago

@uwe-schwarz Thanks for testing the fix.

Making extraction optional for packages might be a good idea. For now, it is out of scope of this bug.

andressoler commented 2 years ago

Here is what I see from my side:

Loading config from /app/config.yaml ...
2022-07-24 20:23:18,455 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Gnome
2022-07-24 20:23:18,510 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Google
2022-07-24 20:23:18,513 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Windows (alt)
2022-07-24 20:23:18,516 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading file
2022-07-24 20:23:18,517 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading keyczar
2022-07-24 20:23:18,517 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading multi
2022-07-24 20:23:18,518 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading pyfs
Loading config from /app/config.yaml ...
2022-07-24 20:23:18,746 :: WARNING :: root :: config_parser.py :: 288 :: app > region not found. Using default value - global ...
2022-07-24 20:23:43,777 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/Mi canción.band ...
2022-07-24 20:23:48,474 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Downloads/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/Downloads/Mi canción.band'
2022-07-24 20:23:54,005 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/www.sneakersales.es ...
2022-07-24 20:23:54,005 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/www.sneakersales 2.es ...
2022-07-24 20:23:57,259 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20210129_201905366.band ...
2022-07-24 20:23:59,456 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20210129_201905366.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20210129_201905366.band'
2022-07-24 20:24:02,840 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band ...
2022-07-24 20:24:05,362 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band'
2022-07-24 20:24:08,130 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band ...
2022-07-24 20:24:10,283 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band'
2022-07-24 20:24:13,889 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Jujutsu.band ...
2022-07-24 20:24:19,011 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Jujutsu.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Jujutsu.band'
2022-07-24 20:24:21,742 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción.band ...
2022-07-24 20:24:24,846 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción.band'
2022-07-24 20:24:29,629 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 2.band ...
2022-07-24 20:24:33,239 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 2.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 2.band'
2022-07-24 20:24:36,556 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...
2022-07-24 20:24:40,605 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.zip.
2022-07-24 20:24:40,605 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.gz.
2022-07-24 20:24:40,606 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 3.band: [Errno 2] No such file or directory
2022-07-24 20:24:44,807 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 4.band ...
2022-07-24 20:24:48,813 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 4.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 4.band'
2022-07-24 20:24:51,280 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 5.band ...
2022-07-24 20:24:54,397 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 5.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 5.band'
2022-07-24 20:24:58,312 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 6.band ...
2022-07-24 20:25:01,181 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 6.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 6.band'
2022-07-24 20:25:04,228 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 7.band ...
2022-07-24 20:25:07,236 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 7.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 7.band'
2022-07-24 20:25:14,051 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Mi canción.band ...
2022-07-24 20:25:17,666 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/Mi canción.band'
2022-07-24 20:25:21,850 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Mi canción 2.band ...
2022-07-24 20:25:25,349 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Mi canción 2.band: [Errno 21] Is a directory: '/app/icloud/drive/Mi canción 2.band'
2022-07-24 20:25:26,763 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Shortcuts/.WorkflowHiddenFile ...
2022-07-24 20:25:26,765 :: INFO :: root :: sync_drive.py :: 199 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...

It seems it cannot download the package file again because a directory already exists...

I can see all my folders in my file system, but all of them are empty, so packages haven't been unzipped properly, I suppose:

ash-4.4# ls -la
total 0
d---------+ 1 root root 438 Jul 24 20:25  .
d---------+ 1 root root 192 Jul 24 10:54  ..
d---------+ 1 root root   0 Jul 24 10:54  20210129_201905366.band
d---------+ 1 root root   0 Jul 24 10:54 '20220124_221641098 (1).band'
d---------+ 1 root root   0 Jul 24 10:54 '20220409_003913347 (1).band'
d---------+ 1 root root   0 Mar  2 14:30 'carpeta sin título'
d---------+ 1 root root   0 Jul 24 10:54  Jujutsu.band
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 2.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 4.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 5.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 6.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 7.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción.band'
uwe-schwarz commented 2 years ago

I did one more test with a subset (just Numbers-Files) and got this (second run):

Loading config from /app/config.yaml ...
2022-07-25 05:43:48,879 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Gnome
2022-07-25 05:43:48,947 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Google
2022-07-25 05:43:48,953 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Windows (alt)
2022-07-25 05:43:48,960 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading file
2022-07-25 05:43:48,963 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading keyczar
2022-07-25 05:43:48,964 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading multi
2022-07-25 05:43:48,966 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading pyfs
Loading config from /app/config.yaml ...
2022-07-25 05:44:00,171 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Numbers/Familie/xxx1.numbers ...
2022-07-25 05:44:06,645 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/Numbers/Familie/xxx1.numbers.zip.
2022-07-25 05:44:06,646 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/Numbers/Familie/xxx1.numbers.gz.
2022-07-25 05:44:06,648 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Numbers/Familie/xxx1.numbers: [Errno 2] No such file or directory
2022-07-25 05:44:06,651 :: INFO :: root :: config_parser.py :: 72 :: Syncing drive every 14400 seconds.
2022-07-25 05:44:06,651 :: INFO :: root :: sync.py :: 127 :: Resyncing at Mon Jul 25 09:44:06 2022 ...
2022-07-25 05:44:06,652 :: INFO :: root :: config_parser.py :: 72 :: Syncing drive every 14400 seconds.

The other numbers files are downloaded as single file, this one seems to be a package. It didn't get successfully downloaded, instead there was this folder:

drwxr-xr-x 4 root root   4096 Jul 25 05:43 \003\v<\0022�T\016�4\003\020���4�v���4��w\006�\024���>i\023�\001Q��q�y�.numbers
mandarons commented 2 years ago

Here is what I see from my side:

Loading config from /app/config.yaml ...
2022-07-24 20:23:18,455 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Gnome
2022-07-24 20:23:18,510 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Google
2022-07-24 20:23:18,513 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Windows (alt)
2022-07-24 20:23:18,516 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading file
2022-07-24 20:23:18,517 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading keyczar
2022-07-24 20:23:18,517 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading multi
2022-07-24 20:23:18,518 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading pyfs
Loading config from /app/config.yaml ...
2022-07-24 20:23:18,746 :: WARNING :: root :: config_parser.py :: 288 :: app > region not found. Using default value - global ...
2022-07-24 20:23:43,777 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/Mi canción.band ...
2022-07-24 20:23:48,474 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Downloads/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/Downloads/Mi canción.band'
2022-07-24 20:23:54,005 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/www.sneakersales.es ...
2022-07-24 20:23:54,005 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/www.sneakersales 2.es ...
2022-07-24 20:23:57,259 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20210129_201905366.band ...
2022-07-24 20:23:59,456 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20210129_201905366.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20210129_201905366.band'
2022-07-24 20:24:02,840 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band ...
2022-07-24 20:24:05,362 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band'
2022-07-24 20:24:08,130 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band ...
2022-07-24 20:24:10,283 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band'
2022-07-24 20:24:13,889 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Jujutsu.band ...
2022-07-24 20:24:19,011 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Jujutsu.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Jujutsu.band'
2022-07-24 20:24:21,742 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción.band ...
2022-07-24 20:24:24,846 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción.band'
2022-07-24 20:24:29,629 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 2.band ...
2022-07-24 20:24:33,239 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 2.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 2.band'
2022-07-24 20:24:36,556 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...
2022-07-24 20:24:40,605 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.zip.
2022-07-24 20:24:40,605 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.gz.
2022-07-24 20:24:40,606 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 3.band: [Errno 2] No such file or directory
2022-07-24 20:24:44,807 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 4.band ...
2022-07-24 20:24:48,813 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 4.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 4.band'
2022-07-24 20:24:51,280 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 5.band ...
2022-07-24 20:24:54,397 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 5.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 5.band'
2022-07-24 20:24:58,312 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 6.band ...
2022-07-24 20:25:01,181 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 6.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 6.band'
2022-07-24 20:25:04,228 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 7.band ...
2022-07-24 20:25:07,236 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 7.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 7.band'
2022-07-24 20:25:14,051 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Mi canción.band ...
2022-07-24 20:25:17,666 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/Mi canción.band'
2022-07-24 20:25:21,850 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Mi canción 2.band ...
2022-07-24 20:25:25,349 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Mi canción 2.band: [Errno 21] Is a directory: '/app/icloud/drive/Mi canción 2.band'
2022-07-24 20:25:26,763 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Shortcuts/.WorkflowHiddenFile ...
2022-07-24 20:25:26,765 :: INFO :: root :: sync_drive.py :: 199 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...

It seems it cannot download the package file again because a directory already exists...

I can see all my folders in my file system, but all of them are empty, so packages haven't been unzipped properly, I suppose:

ash-4.4# ls -la
total 0
d---------+ 1 root root 438 Jul 24 20:25  .
d---------+ 1 root root 192 Jul 24 10:54  ..
d---------+ 1 root root   0 Jul 24 10:54  20210129_201905366.band
d---------+ 1 root root   0 Jul 24 10:54 '20220124_221641098 (1).band'
d---------+ 1 root root   0 Jul 24 10:54 '20220409_003913347 (1).band'
d---------+ 1 root root   0 Mar  2 14:30 'carpeta sin título'
d---------+ 1 root root   0 Jul 24 10:54  Jujutsu.band
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 2.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 4.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 5.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 6.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 7.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción.band'

@andressoler : please try removing all the *.band from your local folder and try again. It looks like previous incorrect downloads are causing some problem.

mandarons commented 2 years ago

files are downloaded as single file, this one seems to be a package. It didn't get successfully downloaded, instead there was this folder:

@uwe-schwarz can you please attach the problematic xxx1.numbers file for me to reproduce the issue on my end? It look like that file is something different from other .numbers file.

andressoler commented 2 years ago

Here is what I see from my side:

Loading config from /app/config.yaml ...
2022-07-24 20:23:18,455 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Gnome
2022-07-24 20:23:18,510 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Google
2022-07-24 20:23:18,513 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading Windows (alt)
2022-07-24 20:23:18,516 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading file
2022-07-24 20:23:18,517 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading keyczar
2022-07-24 20:23:18,517 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading multi
2022-07-24 20:23:18,518 :: INFO :: keyring.backend :: backend.py :: 154 :: Loading pyfs
Loading config from /app/config.yaml ...
2022-07-24 20:23:18,746 :: WARNING :: root :: config_parser.py :: 288 :: app > region not found. Using default value - global ...
2022-07-24 20:23:43,777 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/Mi canción.band ...
2022-07-24 20:23:48,474 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Downloads/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/Downloads/Mi canción.band'
2022-07-24 20:23:54,005 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/www.sneakersales.es ...
2022-07-24 20:23:54,005 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Downloads/www.sneakersales 2.es ...
2022-07-24 20:23:57,259 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20210129_201905366.band ...
2022-07-24 20:23:59,456 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20210129_201905366.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20210129_201905366.band'
2022-07-24 20:24:02,840 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band ...
2022-07-24 20:24:05,362 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20220124_221641098 (1).band'
2022-07-24 20:24:08,130 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band ...
2022-07-24 20:24:10,283 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/20220409_003913347 (1).band'
2022-07-24 20:24:13,889 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Jujutsu.band ...
2022-07-24 20:24:19,011 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Jujutsu.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Jujutsu.band'
2022-07-24 20:24:21,742 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción.band ...
2022-07-24 20:24:24,846 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción.band'
2022-07-24 20:24:29,629 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 2.band ...
2022-07-24 20:24:33,239 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 2.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 2.band'
2022-07-24 20:24:36,556 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...
2022-07-24 20:24:40,605 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.zip.
2022-07-24 20:24:40,605 :: INFO :: root :: sync_drive.py :: 147 :: Successfully unpacked the package /app/icloud/drive/GarageBand for iOS/Mi canción 3.band.gz.
2022-07-24 20:24:40,606 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 3.band: [Errno 2] No such file or directory
2022-07-24 20:24:44,807 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 4.band ...
2022-07-24 20:24:48,813 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 4.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 4.band'
2022-07-24 20:24:51,280 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 5.band ...
2022-07-24 20:24:54,397 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 5.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 5.band'
2022-07-24 20:24:58,312 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 6.band ...
2022-07-24 20:25:01,181 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 6.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 6.band'
2022-07-24 20:25:04,228 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/GarageBand for iOS/Mi canción 7.band ...
2022-07-24 20:25:07,236 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/GarageBand for iOS/Mi canción 7.band: [Errno 21] Is a directory: '/app/icloud/drive/GarageBand for iOS/Mi canción 7.band'
2022-07-24 20:25:14,051 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Mi canción.band ...
2022-07-24 20:25:17,666 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Mi canción.band: [Errno 21] Is a directory: '/app/icloud/drive/Mi canción.band'
2022-07-24 20:25:21,850 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Mi canción 2.band ...
2022-07-24 20:25:25,349 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/Mi canción 2.band: [Errno 21] Is a directory: '/app/icloud/drive/Mi canción 2.band'
2022-07-24 20:25:26,763 :: INFO :: root :: sync_drive.py :: 161 :: Downloading /app/icloud/drive/Shortcuts/.WorkflowHiddenFile ...
2022-07-24 20:25:26,765 :: INFO :: root :: sync_drive.py :: 199 :: Removing /app/icloud/drive/GarageBand for iOS/Mi canción 3.band ...

It seems it cannot download the package file again because a directory already exists... I can see all my folders in my file system, but all of them are empty, so packages haven't been unzipped properly, I suppose:

ash-4.4# ls -la
total 0
d---------+ 1 root root 438 Jul 24 20:25  .
d---------+ 1 root root 192 Jul 24 10:54  ..
d---------+ 1 root root   0 Jul 24 10:54  20210129_201905366.band
d---------+ 1 root root   0 Jul 24 10:54 '20220124_221641098 (1).band'
d---------+ 1 root root   0 Jul 24 10:54 '20220409_003913347 (1).band'
d---------+ 1 root root   0 Mar  2 14:30 'carpeta sin título'
d---------+ 1 root root   0 Jul 24 10:54  Jujutsu.band
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 2.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 4.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 5.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 6.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción 7.band'
d---------+ 1 root root   0 Jul 24 10:54 'Mi canción.band'

@andressoler : please try removing all the *.band from your local folder and try again. It looks like previous incorrect downloads are causing some problem.

@mandarons, when I remove the directories, it download the packages again, and it shows no errors unzipping them, but it creates all de directions again with no files inside:

ash-4.4# ls -lRa
.:
total 0
d---------+ 1 root root 438 Aug  1 11:34  .
d---------+ 1 root root 192 Jul 25 20:55  ..
d---------+ 1 root root   0 Aug  1 11:34  20210129_201905366.band
d---------+ 1 root root   0 Aug  1 11:34 '20220124_221641098 (1).band'
d---------+ 1 root root   0 Aug  1 11:34 '20220409_003913347 (1).band'
d---------+ 1 root root   0 Aug  1 11:32 'carpeta sin título'
d---------+ 1 root root   0 Aug  1 11:34  Jujutsu.band
d---------+ 1 root root   0 Aug  1 11:34 'Mi canción 2.band'
d---------+ 1 root root   0 Aug  1 11:34 'Mi canción 4.band'
d---------+ 1 root root   0 Aug  1 11:34 'Mi canción 5.band'
d---------+ 1 root root   0 Aug  1 11:34 'Mi canción 6.band'
d---------+ 1 root root   0 Aug  1 11:34 'Mi canción 7.band'
d---------+ 1 root root   0 Aug  1 11:34 'Mi canción.band'

./20210129_201905366.band:
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./20220124_221641098 (1).band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./20220409_003913347 (1).band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./carpeta sin título':
total 0
d---------+ 1 root root   0 Aug  1 11:32 .
d---------+ 1 root root 438 Aug  1 11:34 ..

./Jujutsu.band:
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./Mi canción 2.band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./Mi canción 4.band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./Mi canción 5.band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./Mi canción 6.band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./Mi canción 7.band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

'./Mi canción.band':
total 0
d---------+ 1 root root   0 Aug  1 11:34 .
d---------+ 1 root root 438 Aug  1 11:34 ..

Do you think there is a problem with unzip procedure?

@uwe-schwarz, are you experiencing the same with your package files?

thanks!

uwe-schwarz commented 2 years ago

@mandarons, sorry I already resolved this issue by recreating the spreadsheet.

I haven't had any time to get to a few other errors I still have: 2022-08-01 09:13:40,763 :: ERROR :: root :: sync_drive.py :: 171 :: Failed to download /app/icloud/drive/xx1/xx2/xx3.numbers: [Errno 2] No such file or directory

As far as I could debug the issue: not all packages unzip to the same name folder. As in my example above (with the totally broken folder name) I have some "Untitled.numbers" folders.

My guess for a fix would be to create a folder with the correct name, unzip the package, move everything one level higher. Or have a option to record the size in a second file and keep the zip (like x.numbers as zip and x.numbers.meta with the size/modification).