Two-step/two-factor authentication is required (2fa)

ADMiNZ commented 7 months ago

Hello. It is no longer possible to log in via SMS. Codes are not always received, but SMS messages are received instantly.

Previously, there was always a choice of how to send authorization. Is there any way to delete the authorization settings and log in again with confirmation?

kmorber commented 6 months ago

Same problem here, no 2fa code is received.

ADMiNZ commented 6 months ago

The problem is that there is no choice of sms or on the device (and it used to be)

ADMiNZ commented 6 months ago

How can I log in via SMS now? The device is broken and there is no way to get the code on the device

AndreyNikiforov commented 6 months ago

The problem is that there is no choice of sms or on the device (and it used to be)

SMS auth path was broken in 1.17.x when we did an emergency fix for Apple API changes.

AndreyNikiforov commented 6 months ago

... The device is broken and there is no way to get the code on the device

Curious how are you getting new pictures into iCloud if device is broken? Trying to understand the urgency for SMS code functionality. If you are not getting new pictures, then there is no urgent need to run icloudpd. If you are fixing device ASAP to start using it (and getting new pictures), then you'll be able to get MFA once device is fixed. What am I missing?

boredazfcuk commented 6 months ago

I guess there could just be a couple hundred photos in his iCloud which he doesn't want to go through manually and has bought an android phone or something.

I'd also like this feature but for a different reason. I back up 3 additional accounts, and with my phone being a trusted device for the other three accounts, SMS based MFA allows me to re-authenticate the other devices without them being present.

In addition to this, it seems that Apple have reduced the cookie validity period from 90 days to 30 days, so I'm having to grab phones to re-authenticate every couple of weeks. Means I can't just sort it when I have time, but have to organise it around them.

ADMiNZ commented 6 months ago

I make backups of all family members. And phones are not always at hand. I even extended my session via SMS (since it’s convenient). Also, one of the phones broke down, and I didn’t download all the last photos (the session ended), I had to look for a backup old phone - log into my account on it and then get a code...

It’s just that you can even log into this icloud.com account via SMS without any problems - but in the application it was and stopped working :(

ADMiNZ commented 6 months ago

It’s just strange that this was included in the program - and for some reason it was cut out.

AndreyNikiforov commented 6 months ago

@ADMiNZ @boredazfcuk for scenarios of supporting multiple account, pls see if #805 makes sense

AndreyNikiforov commented 6 months ago

v1.17.4 has a fix. Pls reopen the issue if something still not working

boredazfcuk commented 6 months ago

I've just removed my cookie and re-created it, but I wasn't prompted to choose between SMS or the iDevice Popup. It just went straight to Please enter two-factor authentication code: then gave me warning after I clicked Allow, then entered the code: WARNING Failed to parse response with JSON mimetype

AndreyNikiforov commented 6 months ago

I've just removed my cookie and re-created it, but I wasn't prompted to choose between SMS or the iDevice Popup. It just went straight to Please enter two-factor authentication code: then gave me warning after I clicked Allow, then entered the code: WARNING Failed to parse response with JSON mimetype

new logic checks for trusted devices and only offers sms option ifvthere are such devices.

ADMiNZ commented 6 months ago

new logic checks for trusted devices and only offers sms option ifvthere are such devices.

Can you be more specific? Just today I compiled the latest build for the nas server and upon authorization it immediately asks for a code - there is no choice for SMS. I would like it to be like it was before - when extending the session, you can immediately choose how to receive the code (similar to authorization on icloud.com)

AndreyNikiforov commented 6 months ago

Can you be more specific? Just today I compiled the latest build for the nas server and upon authorization it immediately asks for a code - there is no choice for SMS. I would like it to be like it was before - when extending the session, you can immediately choose how to receive the code (similar to authorization on icloud.com)

What version are you comparing behavior with? Since 1.17.0 (and before 1.17.4) MFA auth behavior is the following:

user enters password
iCloud sends MFA to iDevice
iCloud auth responds to icloudpd
icloudpd requests MFA from user
user enters MFA code
icloudpd continues with downloading

In 1.17.4 I changed the logic to the following:

user enters password
iCloud sends MFA to iDevice
iCloud auth responds to icloudpd
icloudpd checks trusted devices
requests MFA from user and gives an option to use SMS if list of trusted devices is not empty
user enters MFA code OR index of the trusted device (if there were trusted devices reported)
if user entered MFA, then icloudpd continues with download (stop auth)
if user entered trusted device index, then icloudpd request SMS code from iCloud
icloudpd requests MFA code sent to SMS from user
icloudpd continues with download

"Trusted devices" request is the same we used pre 1.17.0 to get the list of SMS devices from iCloud. I tested for my account and for single iDevice on the account, it returned one devices, so I can authenticate either way (code on Apple devices or SMS). If you know how to setup different combinations of devices/trusted devices for the account, let me know - I may be able to repro your case.

ADMiNZ commented 6 months ago

As I wrote earlier, I have this workset on a nas server. My task has stopped downloading files because the code has gone bad.

I enter:

(env) root@name:~# icloudpd --username 'e@mail.com' --password 'password' --directory /folder/iCloud 2024-04-18 14:09:26 DEBUG Authenticating... 2024-04-18 14:09:27 ERROR Authentication required for Account. (421) 2024-04-18 14:09:28 INFO Two-step/two-factor authentication is required (2fa) Please enter two-factor authentication code: 1 2024-04-18 14:09:31 ERROR Incorrect verification code. (-21669) 2024-04-18 14:09:31 ERROR Code verification failed. 2024-04-18 14:09:31 ERROR Failed to verify two-factor authentication code

What do I need to do to log in via SMS?

AndreyNikiforov commented 6 months ago

Can you be more specific? Just today I compiled the latest build for the nas server and upon authorization it immediately asks for a code - there is no choice for SMS. I would like it to be like it was before - when extending the session, you can immediately choose how to receive the code (similar to authorization on icloud.com)

What version are you comparing behavior with? Since 1.17.0 (and before 1.17.4) MFA auth behavior is the following:

user enters password

iCloud sends MFA to iDevice

iCloud auth responds to icloudpd

icloudpd requests MFA from user

user enters MFA code

icloudpd continues with downloading

In 1.17.4 I changed the logic to the following:

user enters password

iCloud sends MFA to iDevice

iCloud auth responds to icloudpd

icloudpd checks trusted devices

requests MFA from user and gives an option to use SMS if list of trusted devices is not empty

user enters MFA code OR index of the trusted device (if there were trusted devices reported)

if user entered MFA, then icloudpd continues with download (stop auth)

if user entered trusted device index, then icloudpd request SMS code from iCloud

icloudpd requests MFA code sent to SMS from user

icloudpd continues with download

"Trusted devices" request is the same we used pre 1.17.0 to get the list of SMS devices from iCloud. I tested for my account and for single iDevice on the account, it returned one devices, so I can authenticate either way (code on Apple devices or SMS). If you know how to setup different combinations of devices/trusted devices for the account, let me know - I may be able to repro your case.

I made a mistake. The second scenario worked for my old iPhone without phone number, with separate iCloud account, and with trusted phone number from my primary account. For my primary account with just one iPhone with a number, it does not provide SMS option - I assume we need to use new Apple APIs to get devices capable of SMS for such cases (I tested icloud.com and it has option for SMS for my primary account, so it is feasible).

boredazfcuk commented 6 months ago

Yeah, I'm sure that historically it used to offer a choice of auth.

When the icloud app was used to save the password into the keyring, the only option it would give you was SMS.

Then when the cookie was created, it would allow you to choose from SMS or Apple MFA.

If you check the docs on my project, it kinda confirms this previous behaviour: https://github.com/boredazfcuk/docker-icloudpd/blob/master/CONFIGURATION.md#multifactor-authentication

2020-08-06 16:45:58 INFO     Adding password to keyring...
Enter iCloud password for email@address.com:
Save password in keyring?  [y/N]: y
Two-step authentication required. Your trusted devices are:
  0: SMS to 07********
Which device would you like to use? [0]: 0
Please enter validation code: 123456
2020-08-06 16:47:04 INFO     Using password stored in keyring
2020-08-06 16:47:04 INFO     Generate MFA cookie with password: usekeyring
2020-08-06 16:47:04 INFO     Check for new files using password stored in keyring...
  0: SMS to 07********
  1: Enter two-factor authentication code
Please choose an option: [0]: 1
Please enter two-factor authentication code: 123456
2020-08-06 16:47:30 INFO     Multifactor authentication cookie generated. Sync should now be successful.

AndreyNikiforov commented 6 months ago

I think behavior of many Apple APIs changed (and we adjusted icloudpd for it with 1.17.0), because list of devices that was used to return phone number for SMS does not bring primary phone anymore. icloud.com uses different API for devices today, so the path forward I see is to implement new API for devices in icloudpd.

boredazfcuk commented 6 months ago

Would make sense to be honest. Number/SIM cloning is fairly easy now. So much so, that SMS auth is discouraged as it is less secure than app based. We're phasing it out at work in the next couple of months. Guess Apple have already, for main authentication purposes.

ADMiNZ commented 6 months ago

Would make sense to be honest. Number/SIM cloning is fairly easy now. So much so, that SMS auth is discouraged as it is less secure than app based. We're phasing it out at work in the next couple of months. Guess Apple have already, for main authentication purposes.

After replacing SIM cards, they do not receive incoming SMS for a day.

You can also find fault with the fact that all devices linked to the account receive the code :) And if an attacker steals, say, a Macbook, with a simple password and in the notes there will be a password for the account. He can download everything easily :)

This is the second authorization - you also need to know your login and password.

AndreyNikiforov commented 6 months ago

I see the security concern of SMS auth at Apple as a separate from supporting it in icloudpd. The fact that many companies dropping support may suggest that Apple will drop it too and we may decide to not invest in supporting it in icloudpd.

Currently, I do not commit to invest time beyond short trial into supporting new API for SMS.

boredazfcuk commented 5 months ago

So, I've just come to renew the cookie on a family member's device and the process went like this:

2024-04-27 10:37:37 INFO     Starting container initialisation
2024-04-27 10:37:37 DEBUG    2024-04-27 10:37:37 INFO     Correct owner on config directory, if required
2024-04-27 10:37:37 DEBUG    2024-04-27 10:37:37 INFO     Correct group on config directory, if required
2024-04-27 10:37:37 DEBUG    Generate MFA cookie using password stored in keyring file
2024-04-27 10:37:39 ERROR    Authentication required for Account. (421)
  0: SMS to ********01
  1: SMS to ********02
Please enter two-factor authentication code or device index (0..1) to send SMS with a code: 1
Please enter two-factor authentication code that you received over SMS: 703638
2024-04-27 10:38:18 INFO     Multifactor authentication cookie generated. Sync should now be successful
2024-04-27 10:38:18 INFO     Container initialisation complete

The trusted account number (my number) is the one that ends in 02. The number that ends in 01 is the phone number that is registered to the icloud account that I'm downloading photos from.

It's good that I can choose either number for authentication, but surely this means that anybody with a trusted number registered to their account is going to be forced to use SMS auth as there is no option for Apple iDevice auth?

AndreyNikiforov commented 5 months ago

It's good that I can choose either number for authentication, but surely this means that anybody with a trusted number registered to their account is going to be forced to use SMS auth as there is no option for Apple iDevice auth?

Apple devices receive code right after password is submitted to iCloud and there is no choice to receive on iDevice or not - that is why UI asks for code received on Apple device OR choice of SMS options.

boredazfcuk commented 5 months ago

When I attempt to login to icloud.com using my family member's Apple ID, it automatically performs the iDevice request after putting the password in: If I want to use SMS instead, I click 'Did not get a verification code?' and then 'Use phone number': Then select the number:

Can icloudpd not mimic this behaviour? Initially give the user two options, 1: Enter MFA code or 2: Use SMS? If option 2 is selected, present the trusted numbers for the user to select which they want to use, as currently happens.

AndreyNikiforov commented 5 months ago

@boredazfcuk I am lost. What auth actions/steps you can do through web and cannot do with icloudpd (and need them)?

From what I understand icloudpd supports a) login with MFA sent on Apple device and b) MFA through SMS on trusted device with a minimum amount of steps possible for each use case.

boredazfcuk commented 5 months ago

Sorry, it's me not picking up on the change of behaviour. Previously, it would ask something like:

0: MFA
1: SMS
2: SMS trusted

I was expecting the same prompts. I should read things properly and not skim read.

AndreyNikiforov commented 5 months ago

Sorry, it's me not picking up on the change of behaviour. Previously, it would ask something like:
0: MFA
1: SMS
2: SMS trusted

MFA choice is unnecessary, because you get the code anyways, that is why the choice you are given:

enter 6 digit code you already received on Apple device
get SMS with code on number xxx01
get SMS with code on number xxx02

Another way to look:

for users with apple devices it is one step - just enter the code (in old behavior there were two steps - select that you want to enter MFA from Apple and enter MFA itself)
for user who wants to use SMS there are still two steps - select device to send SMS to and enter the code itself

SurajViitk commented 5 months ago

Maybe a little deviating from topic, but I got a workaround for this. I was not getting the option to choose 2FA mode (sms or phone), it directly went to enter validation code step and no code was received. I requested the code by logging into icloud web, chose sms option and used that otp in the option and it worked!

AndreyNikiforov commented 5 months ago

Maybe a little deviating from topic, but I got a workaround for this. I was not getting the option to choose 2FA mode (sms or phone), it directly went to enter validation code step and no code was received. I requested the code by logging into icloud web, chose sms option and used that otp in the option and it worked!

FYI I have noticed recently that I stopped receiving list of SMS devices for either of my two accounts recently. It may be a change on Apple side or maybe the some nuances of the Apple API for trusted devices we do not understand yet.

guyank commented 5 months ago

I'm having a problem with authenticating a family member's account - mine works fine. When I enter the password, they never receive a code and it seems to go through:

2024-04-30 21:18:39 INFO Directory is writable: /config/python_keyring/ 2024-04-30 21:18:39 INFO Starting container initialisation iCloud Password: 2024-04-30 21:19:19 ERROR Authentication required for Account. (421) 2024-04-30 21:19:25 INFO Multifactor authentication cookie generated. Sync should now be successful 2024-04-30 21:19:25 INFO Container initialisation complete

However, when I look at the logs, I get the following:

2024-04-30 21:28:31 DEBUG Authenticating... iCloud Password: 2024-04-30 21:28:31 ERROR Failed to download new files 2024-04-30 21:28:31 ERROR - Can you log into icloud.com without receiving pop-up notifications? 2024-04-30 21:28:31 ERROR Error debugging info: 2024-04-30 21:28:31 ERROR getpass.py:91: GetPassWarning: Can not control echo on the terminal. Warning: Password input may be echoed. Aborted! 2024-04-30 21:28:31 ERROR Please report problems here: https://github.com/boredazfcuk/docker-icloudpd/issues 2024-04-30 21:28:31 INFO Web cookie expires: 2024-06-29 @ 20:19:23 2024-04-30 21:28:31 INFO Multifactor authentication cookie expires: 2024-05-30 @ 20:19:23 2024-04-30 21:28:31 INFO Days remaining until expiration: 29 2024-04-30 21:28:31 INFO Synchronisation ended at 21:28:31 2024-04-30 21:28:31 INFO Total time taken: 00:00:02 2024-04-30 21:28:31 INFO Next synchronisation at 21:28:29

AndreyNikiforov commented 5 months ago

Please report problems here: https://github.com/boredazfcuk/docker-icloudpd/issues

You need to report problems into another GH project as stated

boredazfcuk commented 5 months ago

Just digging this up again as I've noticed the behaviour where it doesn't list trusted devices again. When I attempt to reauthenticate my wife's device, I just get:

2024-05-16 21:46:53 ERROR    Authentication required for Account. (421)
Please enter two-factor authentication code:

Despite her account having two trusted SMS numbers available when logging into icloud.com.

I tried @SurajViitk's solution, but it did not work for me:

Maybe a little deviating from topic, but I got a workaround for this. I was not getting the option to choose 2FA mode (sms or phone), it directly went to enter validation code step and no code was received. I requested the code by logging into icloud web, chose sms option and used that otp in the option and it worked!

boredazfcuk commented 5 months ago

I've also noticed, that when I log in to my iPad's account, which doesn't have a SIM of its own, the UI is a little different: Instead of having "Use phone number" it just says "Text me" instead. When I click that, it sends the SMS code to my iPhone, which is the only trusted number for the iPad's Apple ID.

cyayon commented 4 months ago

Hi,

do you found a workaround for the 30 days MFA expiration cookie ?

On another project https://github.com/rclone/rclone we are looking for a workaround https://github.com/rclone/rclone/pull/7717 ...

thanks.

boredazfcuk commented 4 months ago

Expiry time is set by Apple so I don't think you'll have much much trying to extend it. It used to be 90-days, but they recently reduced it to 30-days.

I handle it in my container using a Telegram chat bot and the expect package. Basically the container polls the chat bot every minute, when it receives an authentication command, it starts the icloud authentication process. It messages the user with the authentication options (Apple/SMS). User responds with the choice which expect sends to icloud. It then asks the user for the MFA code, it then uses that to validate MFA and obtain a new 30-day cookie.

cyayon commented 4 months ago

Expiry time is set by Apple so I don't think you'll have much much trying to extend it. It used to be 90-days, but they recently reduced it to 30-days.

I handle it in my container using a Telegram chat bot and the expect package. Basically the container polls the chat bot every minute, when it receives an authentication command, it starts the icloud authentication process. It messages the user with the authentication options (Apple/SMS). User responds with the choice which expect sends to icloud. It then asks the user for the MFA code, it then uses that to validate MFA and obtain a new 30-day cookie.

thanks.

AndreyNikiforov commented 4 months ago

@cyayon there is also a discussion #805 about adding WebUI for password/MFA. It gives an opportunity for people carrying devices with AppleID used by icloudpd to react on MFA prompts right away and without external systems.

Where can I learn about iCloud photo support for rclone? Don't see that provider into the doc.

cyayon commented 4 months ago

Hi, thanks for your answer, I will forward on rclone thread. Currently, iCloud on rclone is in alpha/beta status : https://github.com/lostb1t/rclone/tree/feature/icloud, there is no photo support at this moment.

AndreyNikiforov commented 3 months ago

in 1.20.3 icloudpd switched to using latest icloud.com api for SMS MFA. Pls reopen if issue persists.

boredazfcuk commented 3 months ago

I've attempted to authenticate my own account using the option SMS for the number associated with my account and get this error:

2024-06-30 16:08:11 DEBUG    Generate MFA cookie using password stored in keyring file
2024-06-30 16:08:13 ERROR    Authentication required for Account. (421)
  0: ••••• ••••01
  1: ••••• ••••02
Please enter two-factor authentication code or device index (0..1) to send SMS with a code: 0
Please enter two-factor authentication code that you received over SMS: 123456
2024-06-30 16:08:31 ERROR    Please log into https://icloud.com/ to manually finish setting up your iCloud service (AUTHENTICATION_FAILED)
Traceback (most recent call last):
  File "starters/icloudpd.py", line 6, in <module>
  File "click/core.py", line 1157, in __call__
  File "click/core.py", line 1078, in main
  File "click/core.py", line 1434, in invoke
  File "click/core.py", line 783, in invoke
  File "icloudpd/base.py", line 499, in main
  File "icloudpd/base.py", line 983, in core
  File "pyicloud_ipd/base.py", line 546, in photos
  File "pyicloud_ipd/services/photos.py", line 279, in __init__
  File "pyicloud_ipd/services/photos.py", line 166, in __init__
  File "requests/sessions.py", line 637, in post
  File "pyicloud_ipd/session.py", line 168, in request
  File "pyicloud_ipd/session.py", line 186, in _raise_error
pyicloud_ipd.exceptions.PyiCloudServiceNotActivatedException: Please log into https://icloud.com/ to manually finish setting up your iCloud service (AUTHENTICATION_FAILED)
[718] Failed to execute script 'icloudpd' due to unhandled exception!

Works OK if I just enter the code which pops up on my phone though.

I then tested on one of my other accounts, for which my mobile number is only a trusted number for, and it doesn't actually prompt me for any authorisation info. It just just tells me I need to authenticate, then exits:

2024-06-30 16:14:50 ERROR    Authentication required for Account. (421)
2024-06-30 16:14:56 INFO     Multifactor authentication cookie generated. Sync should now be successful

I've restarted the container after this and it's actually fine.

AndreyNikiforov commented 3 months ago

I think my testing was not sufficient. I did --auth-only for my testing and it works, but actual downloading fails with the same error. Back to debugging...

AndreyNikiforov commented 3 months ago

another attempt with 1.20.4

boredazfcuk commented 3 months ago

I had a look at my code, and I was using a bit of a hack to get it to generate a cookie:

"/opt/icloudpd/bin/icloudpd --username ${apple_id} --cookie-directory ${config_dir} --directory /dev/null --only-print-filenames --recent 0 --domain ${auth_domain}"

This was how I managed to get it re-authenticating without downloading anything. It would still create the directory structure, so I chucked them in the bitbucket.

This is from the days before --auth-only so I've updated the script so it now runs this instead::

/opt/icloudpd/bin/icloudpd --username ${apple_id} --cookie-directory ${config_dir} --auth-only --domain ${auth_domain}

I believe I was being MFA authenticated without having to re-enter the code as, the cookie generation code was only removing the cookie. It seems a .session file is also being created, so I'm removing that too now, and all seems OK.

icloud-photos-downloader / icloud_photos_downloader

Two-step/two-factor authentication is required (2fa) #803