[FEATURE] Support for Advanced Data Protection (iCloud Encryption)

[harveybolton]

Use case Now that Apple has rolled out Advanced Data protection globally it's only a matter of time before the masses adopt it and it has potential to be enabled by default for new iCloud accounts soon enough, it would be great to get this project compatible with ADP for people already using it and those who will start to use it in future.

Describe the solution you'd like Being able to download iCloud photos that are end to end encrypted.

Describe alternatives you've considered Disabling ADP, but this is less desirable as it's a long overdue feature from Apple

Additional context If you have ADP enabled on your iCloud account, this is what happens after authenticating (with 2FA) with this project:

Loading config from /app/config.yaml ...
Loading config from /app/config.yaml ...
2023-02-04 14:51:49,566 :: INFO :: root :: sync.py :: 71 :: Syncing drive...
2023-02-04 14:51:50,125 :: ERROR :: icloudpy.base :: base.py :: 186 :: Missing PCS cookies from the request (423)
Traceback (most recent call last):
  File "/app/./src/main.py", line 7, in <module>
    sync.sync()
  File "/app/src/sync.py", line 72, in sync
    sync_drive.sync_drive(config=config, drive=api.drive)
  File "/app/src/sync_drive.py", line 304, in sync_drive
    items=drive.dir(),
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 234, in __getattr__
    return getattr(self.root, attr)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 229, in root
    self, self.get_node_data("FOLDER::com.apple.CloudDocs::root")
  File "/app/venv/lib/python3.10/site-packages/icloudpy/services/drive.py", line 34, in get_node_data
    request = self.session.post(
  File "/app/venv/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/base.py", line 158, in request
    self._raise_error(code, reason)
  File "/app/venv/lib/python3.10/site-packages/icloudpy/base.py", line 187, in _raise_error
    raise api_error
icloudpy.exceptions.ICloudPyAPIResponseException: Missing PCS cookies from the request (423)

[mietzen]

Did you have a chance to take a look at this? I was wondering if the encryption key you get on initial setup might be able to decrypt the data.

[mandarons]

A bit, yes.

From Apple's website:

Advanced Data Protection and iCloud.com web access

When a user first turns on Advanced Data Protection, web access to their data at iCloud.com is automatically turned off. This is because iCloud web servers no longer have access to the keys required to decrypt and display the user’s data. The user can choose to turn on web access again, and use the participation of their trusted device to access their encrypted iCloud data on the web. After turning on web access, the user must authorize the web sign-in on one of their trusted devices each time they visit iCloud.com. The authorization “arms” the device for web access. For the next hour, this device accepts requests from specific Apple servers to upload individual service keys, but only those corresponding to an allow list of services normally accessible on iCloud.com. In other words, even after the user authorizes a web sign-in, a server request is unable to induce the user’s device to upload service keys for data that isn’t intended to be viewed on iCloud.com, (such as Health data or passwords in iCloud Keychain). Apple servers request only the service keys needed to decrypt the specific data that the user is requesting to access on the web. Every time a service key is uploaded, it is encrypted using an ephemeral key bound to the web session that the user authorized, and a notification is displayed on the user’s device, showing the iCloud service whose data is temporarily being made available to Apple servers.

I haven't turned on the advanced data protection mode yet. If you have Advanced Data Protection turned on, can you help with following questions -

1. Typically, how long does the iCloud.com session last once authenticated?
2. When session expires or on manual logout, does it ask for 2FA every time?
3. Is the behavior same/similar if you access iCloud.com from a non-Apple device such as a Windows laptop?

[mietzen]

I already found the answer to point 1 and it’s not looking great:

After you approve access from your trusted device, you can access your data at iCloud.com for the next hour. Each time you access a new category of data — such as photos, notes, or files — you’ll be asked to approve that access from your trusted device. Some data isn’t available on iCloud.com, such as Health data and passwords in iCloud Keychain.

https://support.apple.com/en-us/102630

I don’t see this feature to make a lot of sense if one has to constantly approve any kind of access. 🙁

[o-lukas]

After manual logout it asks for 2FA every time.

I tested it using a Windows virtual machine as well and it seems to behave the same as on Apple devices.

So only way to get this working seems to be if the session does never expire...

[mandarons]

Yeah, I am wondering if it is worth the effort to enable this feature in icloudpy given that most of the applications won't be able to utilize it without poor user experience (many 2FAs, expiring sessions etc.).

[mandarons]

I already found the answer to point 1 and it’s not looking great:

After you approve access from your trusted device, you can access your data at iCloud.com for the next hour. Each time you access a new category of data — such as photos, notes, or files — you’ll be asked to approve that access from your trusted device. Some data isn’t available on iCloud.com, such as Health data and passwords in iCloud Keychain.

https://support.apple.com/en-us/102630

I don’t see this feature to make a lot of sense if one has to constantly approve any kind of access. 🙁

I agree. Turning on more security features (like this one) will significantly degrade UX for applications built with icloudpy or similar.

[harveybolton]

The iCloud for windows app seems to remain permanently authenticated with support for Advanced Data Protection, I'm not a pro but I wonder if one could use a mitm proxy like burpsuite to intercept the traffic and see what's going on during the authentication stage, if icloudpy could behave similar to the desktop app rather than icloud.com sessions it could solve both the MFA challenge/expiry and advanced data protection problems

[mandarons]

@harveybolton Intercepting traffic won't give us the information we need. With HTTPS, all traffic is e2e encrypted. Once it leaves the application process, we will see encrypted traffic. Only way is to reverse-engineer the iCloud Windows client. I am guessing iCloud Windows client is shipped as binary code. If that's the case, it's a monumental effort (a.k.a. not worth) to decompile the executable.

[dmuiX]

So bad news? Unimplementable so far?

[mandarons]

Yeah. There's not enough ROI yet.

[dmuiX]

Too bad. But understandable. Seems like currently distributed via Microsoft Store. But I suppose it’s an exe file…what else do you use on windows?

[mandarons]

I don't use iCloud client on Windows. I just browse icloud.com.

[mietzen]

Too bad. But understandable. Seems like currently distributed via Microsoft Store. But I suppose it’s an exe file…what else do you use on windows?

You could ask around on Mastodons/Xitters security research community if anyone ever tried to reverse engineer the iCloud Windows client with ADP enabled.

[drpoutine]

Tbh for this to work going forward it’ll need to use Anisette Data to login so you don’t have to always enter a 2FA code and it authorizes you as a device. That part is mostly done. The real challenge is API calls for iCloud. It’s possible CloudKit would work in this scenario but I’ve been doing a local rework of it using my fork. Auth works. Rest needs to be redone 😂

[mandarons]

Yeah, I have been looking into that as well (did a bit of reverse engineering of Windows iCloud client). I didn't get time to try out some experiments yet. If auth works using anisette data, what's the problem that you're facing with iCloud.com API calls?

[drpoutine]

Yeah, I have been looking into that as well (did a bit of reverse engineering of Windows iCloud client). I didn't get time to try out some experiments yet. If auth works using anisette data, what's the problem that you're facing with iCloud.com API calls?

Yea some calls can be different. But for anisette login, a good example is used in pypush since it supports accounts that are both developer organization accounts and have ADP enabled. Basically took that. I’d love to DM you on it and see if its workable since time is pretty stretched out atm.

[dmuiX]

Woah would be so amazing when I could backup/sync my iCloud with my NAS with ADP on. Thanks a lot for answering =) =).

[mandarons]

@drpoutine Sounds great. Please send me an IM on Discord.

[TheIronboy]

any updates for this?

[i814u2]

subscribing, but also double-checking if you're still investigating this one, and need any help/testers. (I finally got around to setting up iCloud-docker, but no joke: just enabled ADP yesterday. What are the odds...)

[dmuiX]

as this is still open And My last information with testing it 10 months ago: when adp is enabled => no access to icloud from this docker container.