All Photos from all libraries

[dotnjet]

I have a shared library and I would like to download all libraries and all photos no matter what. Unfortunately I always miss some of the photos of videos timelapse or even Portraits with the following settings: photo_album="all albums" photo_library="all libraries"

have also tried with photo_library="All Photos"

still no success. I am always missing out on some of the photos in the download. I would prefer to finally have all the photos and videos in one single folder.

Many thanks in advance

Martin

[dotnjet]

checking on my pictures it also showed, that the selfies were not downloaded either.

[boredazfcuk]

Downloading from more than about 4-5 albums in one go will result in Apple forcing you to re-authenticate MFA. Not much point if you have many more than that as it's not possible to automate that.

You will also need separate containers to download albums, shared libraries and 'Recent' photos.

[dotnjet]

Thank you for the update on the correct process. I am still missing how to download for example portraits, they are all missing as well as selfies. On top of that, I would not know the correct folder name etc.

[boredazfcuk]

Are you sure they're missing or are you only downloading the "original" photos. Portraits (and possibly selfies) are just normal photos with a filter applied. The filter is just an affect applied to the original photo, like when you use the markup tool to draw on a photo.

You will need to download original and adjusted photo types to obtain a portrait photo comparable to what you see on your iDevice.

Not sure this is true anymore.

[dotnjet]

I have done several tests. On the iPhone, a portrait Image has for example the name IMG_1000 and you can see that the filters are applied. The same filename should be used for the download but does not appear. I have checked for the name and visible if the picture appears. If I download manually the portraits from icloud.com on the web, the file is converted to a JPG, and the filters are fixed applied.

[boredazfcuk]

I just took a selfie and a portrait. Selfie is no different from any other photo, just comes down as a HEIC (with same name as what's reported on my iDevice).

Portrait comes down as two files. A HEIC file which is a regular photo and a JPG version, which has the blurred background filter applied.

[AndreyNikiforov]

I have done several tests. On the iPhone, a portrait Image has for example the name IMG_1000 and you can see that the filters are applied. The same filename should be used for the download but does not appear. I have checked for the name and visible if the picture appears. If I download manually the portraits from icloud.com on the web, the file is converted to a JPG, and the filters are fixed applied.

@dotnjet you can use --size adjusted to download edited photos and portraits.

[AndreyNikiforov]

I have a shared library and I would like to download all libraries and all photos no matter what...

Curious how are you expecting photos from multiple albums to be downloaded? Any photos in any album is also present in All Photos meta-album. Downloading more than one album means either a) files need to be duplicated on local storage OR b) files need to be linked on local storage. Both approaches have pros and cons.

[dotnjet]

I have done several tests. On the iPhone, a portrait Image has for example the name IMG_1000 and you can see that the filters are applied. The same filename should be used for the download but does not appear. I have checked for the name and visible if the picture appears. If I download manually the portraits from icloud.com on the web, the file is converted to a JPG, and the filters are fixed applied.

@dotnjet you can use --size adjusted to download edited photos and portraits.

Many thanks! My goal is actually only to be able to download ALL pictures from iCloud. Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded. I palyed around with the two settings photo_album="all albums" photo_library="all libraries" but without great success.

You are absolutely right, if I download the folders then I have duplicates which I have to remove later :-( Even in this case I do not have the portraits.

Have tested with --size adjusted and with --size original. Unfortunately no success.

Here my current settings: agentid= albums_with_dates=false align_raw=as-is apple_id=test@test.com auth_china=false authentication_type=MFA auto_delete=false bark_device_key= bark_server= content_source_url= convert_heic_to_jpeg=false debug_logging=false delete_accompanying=false delete_after_download=false delete_empty_directories=false delete_notifications=true dingtalk_token= directory_permissions=750 discord_id= discord_token= download_notifications=true download_path=/iCloud fake_user_agent=false file_match_policy=name-size-dedup-with-suffix file_permissions=640 folder_structure={:%Y} force_gid=false gotify_app_token= gotify_https= gotify_server_url= group=group group_id=1000 icloud_china=false iyuu_token= jpeg_path= jpeg_quality=90 keep_unicode=false libraries_with_dates=false live_photo_mov_filename_policy=suffix live_photo_size=original media_id_delete= media_id_download= media_id_expiration= media_id_startup= media_id_warning= msmtp_args=--tls-starttls=off msmtp_from= msmtp_host= msmtp_pass= msmtp_port= msmtp_tls=on msmtp_to= msmtp_user= name= nextcloud_delete=false nextcloud_password= nextcloud_upload=false nextcloud_url= nextcloud_username= notification_days=7 notification_type=Telegram photo_album="All Photos" photo_library=all libraries photo_size=adjusted,original prowl_api_key= pushover_sound= pushover_token= pushover_user= recent_only=500 set_exif_datetime=false single_pass=false skip_album= skip_check=true skip_download=false skip_library= skip_live_photos=false skip_videos=false startup_notification=true synchronisation_delay=0 synchronisation_interval=43200 synology_ignore_path=false telegram_chat_id=111111111111111111111 telegram_http=false telegram_polling=true telegram_server= telegram_silent_file_notifications= telegram_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx touser= trigger_nextlcoudcli_synchronisation= until_found= user=user user_id=1000 webhook_https=false webhook_id= webhook_path=/api/webhook/ webhook_port=8123 webhook_server= wecom_id= wecom_proxy= wecom_secret=

[AndreyNikiforov]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?

If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u your@email.address -d /data --recent 1 --size adjusted

[dotnjet]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib?

If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u your@email.address -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

[dotnjet]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib? If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u your@email.address -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

[AndreyNikiforov]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib? If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u your@email.address -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

Once you take portrait, do you see it in icloud.com? If it is not uploaded from device to icloud.com, then icloudpd cannot download it. The fact that icloudpd tells you that "the picture was downloaded already" suggests that new portrait not visible in cloud.com.

If you confirm that portrait indeed exists in icloud.com, but icloudpd does not see it, you can try running the same command without --size adjusted parameter to isolate, hopefully, logic with selecting edits. Your portrait should be downloaded as regular photo (without bokeh) in this case.

[dotnjet]

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib? If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u your@email.address -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

Since I also have a shared library this was not that easy any more. I noticed that portraits for example did not get downloaded.

Do you see other assets from the same shared library? Do you have portraits and/or edits downloaded from your PrimarySync lib? If you have latest asset as edit/portrait, you may find it faster to iterate with just it:

docker run --rm -it -v your_local_storage:/data icloudpd/icloudpd:1.23.4 icloudpd -u your@email.address -d /data --recent 1 --size adjusted

Many thanks. I will a.s.a.p. look into it but probably not before next week.

Sorry for the delay. Now I have tested it. Ran your script and downloaded the last picture successfully. Then I took a portrait and started the script again and got told, that the picture was downloaded already. So it is not downloading portraits :-(

Once you take portrait, do you see it in icloud.com? If it is not uploaded from device to icloud.com, then icloudpd cannot download it. The fact that icloudpd tells you that "the picture was downloaded already" suggests that new portrait not visible in cloud.com.

If you confirm that portrait indeed exists in icloud.com, but icloudpd does not see it, you can try running the same command without --size adjusted parameter to isolate, hopefully, logic with selecting edits. Your portrait should be downloaded as regular photo (without bokeh) in this case.

Thank you Andrey I have now tested it again with the different settings but only with partial success. I can confirm the latest newly taken portrait pictures was visible on icloud.com All the pictures I take are going directly to a shared library as I shale as a default to the family. First it did not download, but when I moved it to the personal library it was downloaded. I can confirm, that the setting "--size adjusted" did make it work! So now I just need the right setting to make it work with the from my user managed shared library. MANY THANKS in advance

[dotnjet]

Here some more infos: When I ran the cmd "sync-icloud.sh --List-Libraries" I get two: 2024-09-20 20:57:12 INFO - SharedSync-5F104EB2-15AC-XXXX-XXXXXXXXXXXX 2024-09-20 20:57:12 INFO - PrimarySync

When I move the portrait back to the shared library and change the settings to --library "SharedSync-5F104EB2-15AC-XXXX-XXXXXXXXXXXX" then the portrait is downloaded as HEIC, but not the original without filter. When downloading with --library "PrimarySync" I do not get to download the portrait picture.

How can I set --library "all libraries"? It gets back with "ERROR Unknown library: all libraries"