ICloud integration gives invalid auth w/ app-specific pw

jkrall commented 3 years ago

The problem

I'm attempting to setup the iCloud integration w/ an app-specific password, per the instructions here.

Upon hitting submit, I get a short spinner and then "Invalid authentication" — and I cannot proceed beyond this error to finish setting up the icloud integration. I've also tried the suggested troubleshooting step of removing the .storage/icloud directory, to no avail. (which is created upon attempting to setup the integration, but removing it has no affect on the issue)

I have confirmed that I've entered the app-specific password correct, and attempted several different passwords I've generated on appleid.apple.com.

What is version of Home Assistant Core has the issue?

core-2021.7.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

icloud

Link to integration documentation on our website

https://www.home-assistant.io/integrations/icloud/#app-specific-passwords

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2021-08-03 23:05:32 ERROR (SyncWorker_40) [pyicloud.base] Missing apple_id field
2021-08-03 23:05:32 ERROR (MainThread) [homeassistant.components.icloud.config_flow] Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

Additional information

No response

bcutter commented 1 year ago

I don't think so. I meanwhile kicked the integration after using it for only 4 weeks. Got almost hundred (!) mails every day.

Once pain > benefits, that's a no go for using an integration.

No idea if the custom one (iCloud3) performs/will be better (does it support app specific logins?).

N3rdix commented 1 year ago

Version 2 performs pretty well for me since more than 2 years with only a few mails (~1-2 a week max.) Definitely worth to invest some time in my opinion.

According to the author the app-specific password is on the list of things he wants to look into for version 3 which is currently in beta.

wolph commented 1 year ago

I just report those mails as spam since I can't unsubscribe from them. After a little while gmail learns to trash them :)

LewisSpring commented 1 year ago

I just report those mails as spam since I can't unsubscribe from them. After a little while gmail learns to trash them :)

Please set up a filter instead! This could cause false-positives for other people and may mean someone misses an indicator of an account breach.

emce commented 1 year ago

So - it raises concerns...

wolph commented 1 year ago

I just report those mails as spam since I can't unsubscribe from them. After a little while gmail learns to trash them :)

Please set up a filter instead! This could cause false-positives for other people and may mean someone misses an indicator of an account breach.

Any mail that you cannot unsubscribe from is spam in my book. Perhaps this way Apple will fix their broken system at some point

LewisSpring commented 1 year ago

I just report those mails as spam since I can't unsubscribe from them. After a little while gmail learns to trash them :)

Please set up a filter instead! This could cause false-positives for other people and may mean someone misses an indicator of an account breach.

Any mail that you cannot unsubscribe from is spam in my book. Perhaps this way Apple will fix their broken system at some point

I don't disagree!

polskikrol commented 1 year ago

Here is a link to the feature request in v3: https://github.com/gcobb321/icloud3_v3/issues/78. Doest appear that either the development branch nor the integrated feature in core will support app specific passwords anytime soon. Everything runs on an older python implementation (https://pypi.org/project/pyicloud/) which essentially recreates the web calls rather than using any modern API which Apple now supports. The lack of app specific passwords is also an issue in this python code as well (https://github.com/picklepete/pyicloud/issues/349). This is truly a double whammy of crud from a security perspective:

One doesnt want to use / encode / hard code / whatever their master icloud password in some config file on an automation server. App passwords are meant to over come this, where you have a different password for a specific application and can easily revoke or change, without affecting your other logins.
One doesnt want email spam from the python cloud re-auth occurring a few times daily as tokens age out, which are then ignored, and may result in a true compromise or attack being missed.

In summary, doesnt look good.

issue-triage-workflows[bot] commented 1 year ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

shailyglobal83 commented 1 year ago

Hi,

Thanks for the response, i Checked it on the latest version and it seems that this issue is not resolved by the team yet. So kindly do not close and it take it on high priority because no one would like to use actual password in configuration So we should have the option to login with app specific password.

polskikrol commented 1 year ago

Agreed. This is not fixed. Many other modules support API based authentication using app specific passwords, but this one does not at the moment. Given many people have iPhones, not sure why this is not prioritized.

wolph commented 1 year ago

Agreed. This is not fixed. Many other modules support API based authentication using app specific passwords, but this one does not at the moment. Given many people have iPhones, not sure why this is not prioritized.

It's not fixed because it can't be fixed. The available apple api's don't expose this information so we can't use app specific passwords for this.

This issue really should be closed and marked as unfixable for the time being

issue-triage-workflows[bot] commented 9 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

magicus commented 9 months ago

@balloob @frenck Sorry for the ping, but I don't know how to get visibility for this otherwise. The icloud integration is basically abandoned and broken. It needs either to have an active developer assigned as owner, or it should be removed from the core distribution.

It is listed as owned by @Quentame and @nzapponi, but neither of them have responded to any of the open bugs. @nzapponi said in https://github.com/home-assistant/core/issues/101816#issuecomment-1817365713 that he only contributed a few changes 3 years ago, and is not maintaining the component.

There are about a dozen open bugs on the icloud integration, all of them basically boiling down to the fact that authentication with Apple does not work properly anymore. The failure modes are a bit different, but typically you can install the integration and it works for a while, but then the login with Apple expires, and the problems begin.

issue-triage-workflows[bot] commented 6 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

magicus commented 6 months ago

Hi bot, this is in no way resolved. The lack of activity is due to the lack of anyone responsible for fixing it. :-/

issue-triage-workflows[bot] commented 3 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

magicus commented 3 months ago

Hi bot, this is in no way resolved. The lack of activity is due to the lack of anyone responsible for fixing it. :-/

caseyjmorton commented 3 months ago

OK. I'm done waiting on someone else more qualified to fix this. I will jump in and see what I can to to fix it myself. For those of you familiar with Major League, just call me Pedro Serrano. I will start working on PRs for all of this. If anyone has any tribal knowledge on this that they can share to help me get started in the right direction, it would be much appreciated. I cant promise a quick or complete solution, but apparently I'm all we got :)

wolph commented 3 months ago

OK. I'm done waiting on someone else more qualified to fix this. I will jump in and see what I can to to fix it myself. For those of you familiar with Major League, just call me Pedro Serrano. I will start working on PRs for all of this. If anyone has any tribal knowledge on this that they can share to help me get started in the right direction, it would be much appreciated. I cant promise a quick or complete solution, but apparently I'm all we got :)

I've got no clue who major League is but I wish you a lot of luck!

The issue is unfortunately a very difficult one to solve. Apple has no public API available so your only option to do this without the emails is using (or emulating) an apple device which is non-trivial to say the least. Apple really doesn't want 3rd parties to use their private APIs so it's really hard to get that path working.

As I see it you have a few options:

Having a dedicated Apple device which is remote controlled to fetch this information
Having a virtual apple device which is remote controlled. Not sure what the current state is but I know there was a time that a homebrewed apple device could access the APIs
Reverse engineer the Apple authentication to spoof having a real apple device. This one is probably prohibitively difficult to achieve, but if the homebrew option works than you could try this method.

vajonam commented 3 months ago

There is already working HACS icloud integration that works great. https://github.com/gcobb321/icloud3. This integration is dead I won't wast any time on this. Just my $0.02.

caseyjmorton commented 3 months ago

Understood. Having looked at the HACS one, consider my previous rant retracted.

caseyjmorton commented 3 months ago

@wolph It looks like someone else has done the real legwork and gone far beyond in a separate project. I guess, I'm just going to migrate to that one as I suggest everyone else who comes across this thread do as well.

I've got no clue who Major League is but I wish you a lot of luck!

Its a 1980's movie about American baseball. I'm honestly not sure how well the humor translates, but IMHO one of the funniest movies of all time. The exact line referenced is "I say F you, Jobu! I'll do it myself!" Its definitely worth a stream.

magicus commented 3 months ago

With that said, the icloud3 integration is very "heavy" and comes with a GUI that does not fit with Home Assistant, in contrast to this integration. If it would work, I would switch back from icloud3 right away.

Basically all the fixes that makes icloud3 work resides in changes to the pyicloud library, which this integration also uses. Unfortunately the original author of pyicloud has virtually abandoned the project, so it has not been updated upstream for a long time, but instead several patched version has appeared in different projects.

I started looking at what changes icloud3 had done to the pyicloud library. Unfortunately, the author of icloud3 have made some drastic refactorings that make it really hard to track what is going on. But I'm trying, from time to time, to spend a bit of time getting closer to untangle that mess, to see which changes are really made. I am pretty confident that some of these changes holds the key to getting this integration working again.

Ideally, pyicloud would get a new maintainer and all these patches could be upstreamed to it.

vajonam commented 3 months ago

@wolph It looks like someone else has done the real legwork and gone far beyond in a separate project. I guess, I'm just going to migrate to that one as I suggest everyone else who comes across this thread do as well.

Just follow the docs, and you will be good. with v3 its been made a lot simpler to setup.

traindriverrev commented 2 months ago

So I was using the legacy system with account password to get in. Latest update forced app specific password. Yay I thought hadn’t even kept up to date that this was even an option.

And suddenly I’m here.

bmcdonnell-zz commented 2 months ago

Third base. I'm a bit confused why an integration that hasn't worked in 3 years is kept in HomeAssistant year after year for new folks to step in when trying to connect to iCloud

pacsabi commented 2 months ago

It used to work for me (with the monthly deletion of iCloud folder and then again with the adding integration method), but now it doesn't. Of course, I may be doing something wrong, because the documentation is not fixed, there is no mention of an app specific password

PatrikNorrgard commented 1 month ago

Followed docs, still ended up here with App specific password:

Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

Most recent Home Assistant (2024.7.3)

Also deleted .storage/icloud - didn´t help.

jacky-coke commented 1 month ago

Same here: deleted the integration, /config/.storage/icloud-folder and started again from scratch with the same result:

Any suggestions?

Salvora commented 1 month ago

@jacky-coke switch to HACS icloud integration. This one is broken for a long time now. I wonder why this still exists. If it ain't working and no one is working on it to make it work, just remove it from the core.

pacsabi commented 1 month ago

Still Invalid authentication token

ahass-thedev commented 1 month ago

This issue was resolved before by removing app-specific passwords and using account password. This is now once again asking for app specific password and is returning the error.

xhighway commented 2 weeks ago

Using account password instead of requested "one-time password" goes through to 2fa but once 2fa code is submitted it shows "Invalid flow specified" error. But the result is working integration.

iamrellah commented 6 days ago

I don't seem to get this working either.

Error logging into iCloud service: ('Invalid authentication token.', PyiCloudAPIResponseException('Missing apple_id field'))

Core 2024.9.0 Supervisor 2024.09.0 Operating System 13.0 Frontend 20240904.0

sehangel commented 3 days ago

same here, not working.

home-assistant / core