search

picklepete / pyicloud

A Python + iCloud wrapper to access iPhone and Calendar data.
MIT License
2.55k stars 450 forks source link

'Failed to validate the credentials from cookie' #206

Open hakujintl opened 5 years ago

hakujintl commented 5 years ago

Since yesterday I have been hitting an error when attempting to authenticate and pull calendar data.

Traceback (most recent call last):
  File "chadCal.py", line 28, in <module>
    api = PyiCloudService(iCloudUser, iCloudPass)
  File "/Users/chadthurston/Documents/AMPScripts/pythvirtenv/lib/python2.7/site-packages/pyicloud/base.py", line 167, in __init__
    self.authenticate()
  File "/Users/chadthurston/Documents/AMPScripts/pythvirtenv/lib/python2.7/site-packages/pyicloud/base.py", line 190, in authenticate
    raise PyiCloudFailedLoginException(msg, error)
pyicloud.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseError(u'Failed to validate the credentials from cookie',))

I have verified that the password is correct, and I am able to log in online. I did not find the bug reported here, however the Home Assist community has a similar issue that is being tracked, it looks like it is using pyicloud API to interact as well.

https://community.home-assistant.io/t/icloud-device-tracker-error-after-update-to-94-x/121188

zefoo commented 5 years ago

I am seeing the same.

  File "/Users/foo/Code/app/venv/lib/python3.6/site-packages/pyicloud/base.py", line 193, in __init__
    self.authenticate()
  File "/Users/foo/Code/app/venv/lib/python3.6/site-packages/pyicloud/base.py", line 216, in authenticate
    raise PyiCloudFailedLoginException(msg, error)
pyicloud.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseError('Failed to validate the credentials from cookie',))

I have also verified browser login works. Looks like Apple changed something.

MrBronte commented 5 years ago

Same issue here (Version: 0.9.1)

spljaa commented 5 years ago

421 response for calls like POST https://setup.icloud.com/setup/ws/1/login? I'm trying to understand what calls are made when browser logins.

ragaskar commented 5 years ago

Me too, I happened to get logged out (power outage) and can't log back in. Browser login is fine.

arjannv commented 5 years ago

Same, unfortunately..

vipinbeni commented 5 years ago

same happened here

nicx commented 5 years ago

+1

joelmoses commented 5 years ago

FYI, looks like somebody was able to fix this issue by grafting on some code from another project:

https://github.com/PeterHedley94/pyicloud/commit/6bf11c87c784fa9b0f94318ea845e441b0937cf5

The new code seems to use a different auth service.

vipinbeni commented 5 years ago

Can any one please implement in java for https://github.com/viable-hartman/InflatableDonkey/tree/TwoFAHacky please help

PeterHedley94 commented 5 years ago

Joel, gave it a go and can authenticate but the next problem I encountered was throttling of requests from Apple. Didn't want to leave a red-herring on GitHub before leaving the office for a week so deleted it. May be worth exploring further if you can get it working.

joelmoses commented 5 years ago

No, thanks for the effort. It looks like the pyicloud code relies on an earlier mechanism for authentication; the full auth method requires a deriving a Widget key and then doing an IDMSA call with it. Your contrib pointed that out.

I've poked at it and got it partially working. The trouble is that it re-initates the 2FA flow on an irregular basis (not every time!) and I've just not had the time to figure out why.

yogendra123456 commented 5 years ago

HI All,

i am using build by @viable-harman https://github.com/viable-hartman/InflatableDonkey/tree/TwoFAHacky for ios 9 icloud backup but i am getting below error could you please help me out why this is coming

" Exception in thread "main" org.apache.http.client.HttpResponseException: Misdirected Request: {"success":false,"error":"Failed to validate the credentials from cookie"}"

HTTP/1.1 421 Misdirected Request [Server: AppleHttpServer/70a91026, Date: Wed, 12 Jun 2019 02:26:45 GMT,

Content-Type: application/json; charset=UTF-8, Connection: keep-alive, X-Apple-Jingle-Correlation-Key:

OWCHICBDUBFZHBVDE7QGNHD6VM, apple-seq: 0, apple-tk: false, Apple-Originating-System:

UnknownOriginatingSystem, X-Responding-Instance: setupservice:37000301:mr30p70ic-

hygg09033001:8001:1910B166:deb003cd1e4f, Cache-Control: no-cache, no-store, private, Access-Control-Allow-

Origin: https://www.icloud.com, Access-Control-Allow-Credentials: true, Strict-Transport-Security: max-

age=31536000; includeSubDomains, via: icloudedge:bm21p00ic-hygw01043501:7401:19RC207:Mumbai, X-Apple-

Request-UUID: 75847408-23a0-4b93-86a3-27e0669c7eab, access-control-expose-headers: X-Apple-Request-UUID,

access-control-expose-headers: Via] org.apache.http.client.entity.DecompressingEntity@34ace1}

gcobb321 commented 5 years ago

@joelmoses

No, thanks for the effort. It looks like the pyicloud code relies on an earlier mechanism for authentication; the full auth method requires a deriving a Widget key and then doing an IDMSA call with it. Your contrib pointed that out.

I've poked at it and got it partially working. The trouble is that it re-initates the 2FA flow on an irregular basis (not every time!) and I've just not had the time to figure out why.

I use pyicloud.py in the Home Assistant device_tracker icloud3.py custom component . I took the original icloud component and added a bunch of code to it (like 4000+ lines) but I'm not familar with Python libraries other than the ones that were in the original code. I want to import the updated code from joelmoses but am not sure how to structure the from pyicloud import PyiCloudService statement to pick up the new code and anything else I might need

Can you post the python code I need to use to point to the correct repository?

Thanks in advance.

Gary Cobb aka GeeksterGary

walthowd commented 5 years ago

@gcobb321 You'll need the updated code published to PyPi either under this project or a different project name to import it into your Home Assistant component.

I'd probably hold off for now though, as the random 2FA required will probably be more annoying to the users then it just not working at all.

joelmoses commented 5 years ago

Yeah, I'm with @walthowd: I'd leave my example be, as it kicks off 2FA randomly - which is highly irritating.

Understand my purpose here: I just took an earlier piece of work and threw something together to prove that once the new auth takes place and a session token is generated, nothing else is broken for HA: it still imports devices just fine.

What needs to happen is that pyicloud has to add support for full Apple 2FA. There's a path forward using the idmsa endpoint - the test code proves that - but it needs to be natively brought in, not just grafted on.

I'll play with it some more and see what I can do.

gcobb321 commented 5 years ago

@joelmoses, @walthowd Thanks for the advice. I wonder if @picklepete is going to take that on or if it will be someone else. I think the approach I may take over the next few days is to bypass iCloud completely and just use the interface with the HA ios app to get iCloud3 going again.

joelmoses commented 5 years ago

I've done a little more research and it looks like there's an additional security check at play here - something called 2SV, which is the thing that governs whether the browser client is "trusted" or not. If it's not, then it'll continually reenter the HSA login sequence. Basically, it's a GET request to the endpoint https://idmsa.apple.com/appleauth/auth/2sv/trust which gives you a header back called "X-Apple-TwoSV-Trust-Token". You then take this token alongside the standard session token and post it back to https://setup.icloud.com/setup/ws/1/accountLogin as a post body parameter called "TrustToken".

Failure to do so results in being 2FA-challenged repeatedly. I'm seeing what I can do about that.

zhiqihuang commented 5 years ago

same here. Any quick solution?

CWGSM3V0 commented 5 years ago

Don't know if this will help the author, but this method still works:

https://github.com/tylerhall/sosumi

ghost commented 5 years ago

Any solution? I have same question here.

davidohne commented 5 years ago

Here you go: https://github.com/danielfmolnar/pyicloud

gcobb321 commented 5 years ago

@derd3000 Thanks for the update. I've integrated it with iCloud3 and it logs into the iCloud account without any issues. It does give me a notification almost every time I do a Home Assistant restart when the account is reauthorized but that is not a problem and probably a good thing anyway.

I do have a couple of questions. I have forked the code and updated the python import statements In icloud3.py to use a local directory. Should I not do that but have the import statements point to your repository. Since I've never had to import from a non-python library, how should it be coded? Is from 'https://github.com/danielfmolnar/pyicloud' import PyIcloudService be the right way to do it?

Again, thanks and I'm glad to get things working again. I'll be pushing out a new release in a few days.

PeterHedleyJHA commented 5 years ago

@gcobb321 & @derd3000 the patch seems to be a merge from my bodged fix a week ago before going on a business trip. Hence the repo contains a file names hack.py so I would recommend cleaning it up before putting it into a production environment. I will try to find some time to do so now I am back.

@gcobb321 if you wish to use the repo then you can pip install from a github repo. i.e. the process is pip uninstall pyicloud pip install git+https://github.com/danielfmolnar/pyicloud

Your existing code should then work with no changes, but as I say the patch is not tested very well.

PeterHedley94 commented 5 years ago

https://github.com/picklepete/pyicloud/pull/207 - Just opened a pull request

jeffh0821 commented 5 years ago

I recently uninstalled and reinstalled the repo version of pyicloud - but HA is throwing errors when trying to initialize the component:

Error doing job: Future exception was never retrieved Traceback (most recent call last): File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/app/homeassistant/components/icloud/device_tracker.py", line 299, in keep_alive if self.api.requires_2fa: AttributeError: 'PyiCloudService' object has no attribute 'requires_2fa'

Thoughts? Any way I can assist with additional testing?

gcobb321 commented 5 years ago

@jeffh0821 I don't think the current mods that let icloud3 successfully access the icloud location servers have been pushed into the pyicloud repository. I am working off of the code changes PeterHedley has done to allow access. It does allow access but Apple sends the "Your icloud acct was accessed by a device in cityname" and gives the 6-digit authorization code windows constantly. To the point of all the time so it's not ready for actual usage yet. The 2fa/2sa error deals with a variable name change a while ago that did not make it into the pyicloud libraries installed with HA. I changed the name in iCloud3. Will send you a link to a prerelease version of iCloud3 when I figure out how to get the windows to stop popping up all the time.

jeffh0821 commented 5 years ago

@gcobb321 Makes sense. I was scrubbing the HA component looking for the 'requires_2fa' reference to no avail. A custom_component and the https://github.com/PeterHedley94/pyicloud.git repo would be a fine solution until everything gets sorted out and pushed up to master branches.

And then Apple changes it all again....

PeterHedleyJHA commented 5 years ago

@gcobb321 we have a number of old phones and only one is set up for 2FA which is where the issue is with the patch. Maybe try to store the cookies, appleSessionToken etc. so you only need to authenticate once per device until the token expires.

gcobb321 commented 5 years ago

@PeterHedley94 I have copied your base.py code to my HA/custom_components/icloud3 directory, added in a lot of logging debug code and am getting ready to start looking at the data being passed back and forth. Your thought was the approach I was going to look at first to NOT reauthorize all the time but only when necessary.

I found a javascript iCloud access api written by Maurice Conrad and I'm hoping the way he uses 2FA will provide a good approach. It can be found here.. Let me know if you have a chance to look at it and have some inspiration.

Mradr commented 5 years ago

File "c:\python27\lib\site-packages\pyicloud\base.py", line 194, in init self.authenticate() File "c:\python27\lib\site-packages\pyicloud\base.py", line 208, in authenticate myICloud = hack.PyiCloudService() File "c:\python27\lib\site-packages\pyicloud\hack.py", line 232, in init super(PyiCloudService, self).init(self) TypeError: super() argument 1 must be type, not classobj

Seems to be an issue running this in python2.7

Add metaclass = type to the top should fix it.

gcobb321 commented 5 years ago

@PeterHedley94 I've made some progress and have an idea on what has to be done to pyicloud to add the 2SV token code but have run into a road block. I have no experience communicating with apple servers, doing gets & posts, etc. other than what I have learned over the last 2-days digging through code and now I am stuck. I can't seem to get the code to do the actual get for the token and not crash or give me a 401 error code. It's probably a simple matter of putting the correct parameters into the headers or other fields which will give authorization to the data. Can you take a look at it and give me some pointers (code) that will get me going again.

I've put the pyicloud base.py on my repository, along with a file containing some notes about the changes here. My email address is geekstergary@gmail.com if you want to communicate directly.

jeffh0821 commented 5 years ago

So some good testing news...

I was able to make the variable name changes in the HA icloud integration (requires_2fa to requires_2sa) and install the new pyicloud package from @PeterHedley94 github. The "standard" device_tracker component is now working well and without constant notification e-mails from Apple.

Now bear in mind - my iCloud account does not have 2SV setup - so I don't know what impact that might have. However, for my needs it's working again. I'm able to enumerate and track the location of my Apple devices with the iCloud Find My Phone functionality.

Hopefully the pyicloud PR will get merged with a version bump. I'll then file a PR with HA for the icloud integration variable name changes and updates to the manifest.json.

loopy321 commented 5 years ago

@gcobb321 & @derd3000 the patch seems to be a merge from my bodged fix a week ago before going on a business trip. Hence the repo contains a file names hack.py so I would recommend cleaning it up before putting it into a production environment. I will try to find some time to do so now I am back.

@gcobb321 if you wish to use the repo then you can pip install from a github repo. i.e. the process is pip uninstall pyicloud pip install git+https://github.com/danielfmolnar/pyicloud

Your existing code should then work with no changes, but as I say the patch is not tested very well.

I tried this, but still getting the following errors:

$ pip install git+https://github.com/danielfmolnar/pyicloud
Collecting git+https://github.com/danielfmolnar/pyicloud
  Cloning https://github.com/danielfmolnar/pyicloud to /tmp/pip-FCdiuC-build
Requirement already satisfied: requests>=1.2 in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: keyring<9.0,>=8.0 in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: keyrings.alt<2.0,>=1.0 in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: click<7.0,>=6.0 in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: tzlocal in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: pytz in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: certifi in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: future in /usr/lib/python2.7/site-packages (from pyicloud==0.9.1)
Requirement already satisfied: idna<2.6,>=2.5 in /usr/lib/python2.7/site-packages (from requests>=1.2->pyicloud==0.9.1)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/lib/python2.7/site-packages (from requests>=1.2->pyicloud==0.9.1)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests>=1.2->pyicloud==0.9.1)
Installing collected packages: pyicloud
  Running setup.py install for pyicloud ... done
Successfully installed pyicloud-0.9.1
You are using pip version 9.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ icloud --username=XXX@gmail.com
Enter iCloud password for XXX@gmail.com: 
Traceback (most recent call last):
  File "/usr/bin/icloud", line 11, in <module>
    load_entry_point('pyicloud==0.9.1', 'console_scripts', 'icloud')()
  File "/usr/lib/python2.7/site-packages/pyicloud/cmdline.py", line 202, in main
    password.strip()
  File "/usr/lib/python2.7/site-packages/pyicloud/base.py", line 194, in __init__
    self.authenticate()
  File "/usr/lib/python2.7/site-packages/pyicloud/base.py", line 208, in authenticate
    myICloud = hack.PyiCloudService()
  File "/usr/lib/python2.7/site-packages/pyicloud/hack.py", line 232, in __init__
    super(PyiCloudService, self).__init__(self)
TypeError: super() argument 1 must be type, not classobj
ORevelat commented 5 years ago

@bobhavlin - update to python 3

(venv) λ icloud --username=xxx.xxx@gmail.com Enter iCloud password for xxx.xxx@gmail.com: Save password in keyring? [y/N]: N

commentator8 commented 5 years ago

Sorry i seem to have lost the flow - If i understand correctly - it was updated and works but is now dependent on python3? Or does it still repeatedly prompt the user if using 2fa?

Thanks

zhiqihuang commented 5 years ago

I kind lost here. My problem is I didn't turn on the 2fa and still have the problem. Does this occur no matter 2fa on/off?

gcobb321 commented 5 years ago

@zeeqy Constant notifications onLy happen with 2fa on.

@commentator8 Pyicloud.py has code that checks for Python v2 & v3. So probably works with both but I have only used it with v3. Constant notifications dependent on 2fa, not python version.

zhiqihuang commented 5 years ago

@gcobb321 Thanks! I got my code work using danielfmolnar/pyicloud hurrah~

gcobb321 commented 5 years ago

@zeeqy I have a customized version of pyicloud supporting the Home Assistant icloud3 custom component that works with the original find-my-phone and the find-my-friends code. Additional credits for the code go to:

Right now is on my beta repository here but will get promoted to the custom_components/icloud3 soon.

stevenmcastano commented 5 years ago

I'm working on getting this running as well... I can't update to python 3 right now as there are a ton of dependancies in my application that needs to be fixed, but I'm getting a similar error to those above:

Traceback (most recent call last):
  File "iphonelocation.py", line 875, in api_login
    api = PyiCloudService(icloudapi_conf['username'], icloudapi_conf['password'])
  File "pyicloud_ic3.py", line 285, in __init__
    self.setupiCloud = SetupiCloudService(self)
  File "pyicloud_ic3.py", line 583, in __init__
    super(SetupiCloudService, self).__init__(session)
TypeError: must be type, not classobj

I saw someone mention above about adding a line for metaclass = type but that doesn't seem to work for me either. Any suggestions?

JavierMartinz commented 5 years ago

I'm also having this error and I can't disable 2FA in my Apple account as Apple doesn't allow to do it anymore. Is there any solution?

I need the lost_iphone feature, as I use it through an automation if my house alarm got fired, so that I wouldn't ignore it even having the iPhone/iPad/Mac with Do Not Disturb mode enabled.

heggink commented 5 years ago

@gcobb321 Thanks! I got my code work using danielfmolnar/pyicloud hurrah~

Did you do anything specific to get this to work? I just did the uninstall and pip install from github but I get a login error claiming the credentials are wrong (which they are not). I don't use 2FA and use python 3.5.3.

knorrie commented 4 years ago

FYI, the old version of the code suddenly started working again. I have a place where the code was not updated, and it started downloading all new photos since June right now.

bickyb commented 4 years ago

I was able to get things to work once, but after my session dies, if I try again I get the following error: api_error pyicloud.exceptions.PyiCloudAPIResponseError: Missing X-APPLE-WEBAUTH-HSA-LOGIN cookie

this happens after I try the line devices=api.trusted_devices

Any ideas on how I can fix this?