[BUG] ICloudPyFailedLoginException: 'Missing apple_id field'

[clstaudt]

Describe the bug

Previously successful login code now fails with icloudpy.exceptions.ICloudPyFailedLoginException: ('Invalid authentication token.', ICloudPyAPIResponseException('Missing apple_id field')):

  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 277, in __init__
    self.authenticate()
    │    └ <function ICloudPyService.authenticate at 0x1189cf2e0>
    └ <iCloud API: harry@tuttle.gov>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 346, in authenticate
    self._authenticate_with_token()
    │    └ <function ICloudPyService._authenticate_with_token at 0x1189cf370>
    └ <iCloud API: harry@tuttle.gov>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 368, in _authenticate_with_token
    raise ICloudPyFailedLoginException(msg, error) from error
          │                            └ 'Invalid authentication token.'
          └ <class 'icloudpy.exceptions.ICloudPyFailedLoginException'>

icloudpy.exceptions.ICloudPyFailedLoginException: ('Invalid authentication token.', ICloudPyAPIResponseException('Missing apple_id field'))

To Reproduce Steps to reproduce the behavior: Run this code snippet

        icloud_connector = icloudpy.ICloudPyService(
            apple_id=apple_id,
            password=password,
        )

Expected behavior

iCloud login.

Configuration

icloudpy 0.3.2

[mandarons]

It looks like the error is from _authenticate_with_token method. This is used for authentication using session_data. It is likely that the session data that you are reusing might be corrupt(?) for some reason. Try deleting the existing session and recreating a new one.

[clstaudt]

Try deleting the existing session and recreating a new one.

@mandarons Could you advise how to do that? As a user of icloudpy I am not aware that I am reusing session data when I try to initiate a new one.

[mandarons]

Try setting the cookie_directory in ICloudPyService constructor. Here is one example: https://github.com/mandarons/icloud-drive-docker/blob/a83bee5f3fe133f356f5ce2e18f2aae71b1d201a/src/sync.py#L37

[clstaudt]

@mandarons

I am setting the cookie_directory and cookies get stored there. But that seems to give me the same error as before:

2023-02-06 20:45:50.544 | ERROR    | timetracking.intent:connect_to_cloud:117 - ('Invalid authentication token.', ICloudPyAPIResponseException('Missing apple_id field'))
Traceback (most recent call last):

  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 362, in _authenticate_with_token
    req = self.session.post(
          │    │       └ <function Session.post at 0x102120d30>
          │    └ <icloudpy.base.ICloudPySession object at 0x11f54cd90>
          └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/requests/sessions.py", line 635, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
           │    │               │         │          │       └ {}
           │    │               │         │          └ None
           │    │               │         └ '{"accountCountryCode": null, "dsWebAuthToken": null, "extended_login": true, "trustToken": ""}'
           │    │               └ 'https://setup.icloud.com/setup/ws/1/accountLogin'
           │    └ <function ICloudPySession.request at 0x119a235b0>
           └ <icloudpy.base.ICloudPySession object at 0x11f54cd90>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 158, in request
    self._raise_error(code, reason)
    │    │            │     └ 'Missing apple_id field'
    │    │            └ None
    │    └ <function ICloudPySession._raise_error at 0x119a23640>
    └ <icloudpy.base.ICloudPySession object at 0x11f54cd90>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 187, in _raise_error
    raise api_error
          └ ICloudPyAPIResponseException('Missing apple_id field')

icloudpy.exceptions.ICloudPyAPIResponseException: Missing apple_id field

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
    │    └ <function Thread._bootstrap_inner at 0x100e06950>
    └ <Thread(Thread-305 (<lambda>), started daemon 6275526656)>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
    │    └ <function Thread.run at 0x100e06680>
    └ <Thread(Thread-305 (<lambda>), started daemon 6275526656)>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
    │    │        │    │        │    └ {}
    │    │        │    │        └ <Thread(Thread-305 (<lambda>), started daemon 6275526656)>
    │    │        │    └ (<flet.control_event.ControlEvent object at 0x11f54d000>,)
    │    │        └ <Thread(Thread-305 (<lambda>), started daemon 6275526656)>
    │    └ <function NewTimeTrackPopUp.__init__.<locals>.<lambda> at 0x12153c280>
    └ <Thread(Thread-305 (<lambda>), started daemon 6275526656)>

  File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/view.py", line 121, in <lambda>
    on_click=lambda e: on_use_cloud_acc_callback(
                    │  └ <bound method TimeTrackingView.on_login_to_cloud of <timetracking.view.TimeTrackingView object at 0x11ea101f0>>
                    └ <flet.control_event.ControlEvent object at 0x11f54d000>

  File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/view.py", line 284, in on_login_to_cloud
    result: IntentResult[CloudConnector] = self.intent.connect_to_cloud(
            │            │                 │    │      └ <function TimeTrackingIntent.connect_to_cloud at 0x11a95bd00>
            │            │                 │    └ <timetracking.intent.TimeTrackingIntent object at 0x11ea100d0>
            │            │                 └ <timetracking.view.TimeTrackingView object at 0x11ea101f0>
            │            └ <class 'tuttle.cloud.CloudConnector'>
            └ <class 'core.intent_result.IntentResult'>

  File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/core/abstractions.py", line 294, in wrapped
    return attr(*args, **kwargs)
           │     │       └ {'account_id': 'mail@clstaudt.me', 'provider': 'iCloud', 'password': '******'}
           │     └ ()
           └ <bound method TimeTrackingIntent.connect_to_cloud of <timetracking.intent.TimeTrackingIntent object at 0x11ea100d0>>

> File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/intent.py", line 100, in connect_to_cloud
    connector: CloudConnector = self._cloud_calendar_source.login_to_icloud(
                                │    │                      └ <function TimeTrackingCloudCalendarSource.login_to_icloud at 0x11a95b880>
                                │    └ <timetracking.data_source.TimeTrackingCloudCalendarSource object at 0x11ea10b80>
                                └ <timetracking.intent.TimeTrackingIntent object at 0x11ea100d0>

  File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/data_source.py", line 124, in login_to_icloud
    icloud_connector = icloudpy.ICloudPyService(
                       │        └ <class 'icloudpy.base.ICloudPyService'>
                       └ <module 'icloudpy' from '/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/__init__.py'>

  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 277, in __init__
    self.authenticate()
    │    └ <function ICloudPyService.authenticate at 0x119a237f0>
    └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 346, in authenticate
    self._authenticate_with_token()
    │    └ <function ICloudPyService._authenticate_with_token at 0x119a23880>
    └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 368, in _authenticate_with_token
    raise ICloudPyFailedLoginException(msg, error) from error
          │                            └ 'Invalid authentication token.'
          └ <class 'icloudpy.exceptions.ICloudPyFailedLoginException'>

icloudpy.exceptions.ICloudPyFailedLoginException: ('Invalid authentication token.', ICloudPyAPIResponseException('Missing apple_id field'))

[mandarons]

Do you have 2FA enabled for your Apple account?

[clstaudt]

As far as I know yes, logging in usually triggers a 2FA code.

[mandarons]

@clstaudt : sadly, I am not able to reproduce the issue. Something unique about your apple ID account is causing this error.

It looks like this issue is blocking your product development. However, without being able to reproduce the issue, I have limited options to help you.

Let's try adding the missing field and see what happens.

In your site-packages/icloudpy/base.py add the following line below line 358:

"apple_id": self.user["accountName"],

So, the data object should look something like this:

data = {
            "accountCountryCode": self.session_data.get("account_country"),
            "dsWebAuthToken": self.session_data.get("session_token"),
            "extended_login": True,
            "trustToken": self.session_data.get("trust_token", ""),
            "apple_id": self.user["accountName"],
        }

Try reproducing the issue now.

[clstaudt]

@mandarons That gives me a different error now: missing password field

  File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/data_source.py", line 124, in login_to_icloud
    icloud_connector = icloudpy.ICloudPyService(
                       │        └ <class 'icloudpy.base.ICloudPyService'>
                       └ <module 'icloudpy' from '/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/__init__.py'>

  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 277, in __init__
    self.authenticate()
    │    └ <function ICloudPyService.authenticate at 0x117ad3880>
    └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 346, in authenticate
    self._authenticate_with_token()
    │    └ <function ICloudPyService._authenticate_with_token at 0x117ad3910>
    └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 369, in _authenticate_with_token
    raise ICloudPyFailedLoginException(msg, error) from error
          │                            └ 'Invalid authentication token.'
          └ <class 'icloudpy.exceptions.ICloudPyFailedLoginException'>

icloudpy.exceptions.ICloudPyFailedLoginException: ('Invalid authentication token.', ICloudPyAPIResponseException('Missing password field'))

[mandarons]

Try adding "password": self.user["password"] to data object.

[clstaudt]

Different error again:

> File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/intent.py", line 100, in connect_to_cloud
    connector: CloudConnector = self._cloud_calendar_source.login_to_icloud(
                                │    │                      └ <function TimeTrackingCloudCalendarSource.login_to_icloud at 0x13010f760>
                                │    └ <timetracking.data_source.TimeTrackingCloudCalendarSource object at 0x130328940>
                                └ <timetracking.intent.TimeTrackingIntent object at 0x130328910>

  File "/Users/cls/Documents/Work/Projects/PrototypeFund/Dev/tuttle/app/timetracking/data_source.py", line 124, in login_to_icloud
    icloud_connector = icloudpy.ICloudPyService(
                       │        └ <class 'icloudpy.base.ICloudPyService'>
                       └ <module 'icloudpy' from '/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/__init__.py'>

  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 277, in __init__
    self.authenticate()
    │    └ <function ICloudPyService.authenticate at 0x117fdf5b0>
    └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 346, in authenticate
    self._authenticate_with_token()
    │    └ <function ICloudPyService._authenticate_with_token at 0x117fdf640>
    └ <iCloud API: mail@clstaudt.me>
  File "/Users/cls/miniforge3/envs/tuttle/lib/python3.10/site-packages/icloudpy/base.py", line 370, in _authenticate_with_token
    raise ICloudPyFailedLoginException(msg, error) from error
          │                            └ 'Invalid authentication token.'
          └ <class 'icloudpy.exceptions.ICloudPyFailedLoginException'>

icloudpy.exceptions.ICloudPyFailedLoginException: ('Invalid authentication token.', ICloudPyAPIResponseException('Authentication required for Account. (421)'))

[mandarons]

Are you really seeing

'{"accountCountryCode": null, "dsWebAuthToken": null, "extended_login": true, "trustToken": ""}'

? Or did you redact these values for privacy reason?

[clstaudt]

This is the original stack trace, no redaction.

[mandarons]

That's unexpected. I am suspecting something is going on in your application. To remove that possibility, try -

1. Create a fresh virtual environment
2. Install the latest icloudpy
3. Run the following command follow the instructions: icloud --username=<icloud-username> --session-directory=./session_data It will ask you to enter password and 2FA code.

Let me know if it throws an error.

[clstaudt]

This seems to work, it requests a 2FA code and shows no error.

[clstaudt]

I have no idea what could be going on in my application. There is not much going on in this method besides passing apple id and password to ICloudPyService:

    def login_to_icloud(
        self,
        apple_id: str,
        password: str,
    ) -> CloudConnector:
        """Attempts to authenticate user with their icloud account"""
        # TODO: error handling - login may fail
        logger.info(f"Logging in to iCloud with {apple_id}...")
        icloud_connector = icloudpy.ICloudPyService(
            apple_id=apple_id,
            password=password,
            cookie_directory=Path.home() / ".tuttle" / "cookies",
        )
        return CloudConnector(
            cloud_connector=icloud_connector,
            account_name=apple_id,
        )

[clstaudt]

@mandarons What makes you sure it is not an issue in icloudpy?

[mandarons]

As it is working in CLI mode, it is not an issue with icloudpy. Take a look at https://github.com/mandarons/icloudpy/blob/e7edabd2f82fc217d8504cacf9e942e5d7a9808b/icloudpy/cmdline.py#L213. You need to check for if 2FA is needed, get the code and try authenticating using it.

We can continue to use this issue if you need further assistance.

[clstaudt]

Will investigate. What puzzles me is that the application code worked fine for a while. Including the 2FA code request. So the issue may be elsewhere.

[mandarons]

Then it must be some runtime issue, especially if you're using multithreading/multiprocessing etc.

[mandarons]

Good luck!