search

fishbigger / TapoP100

A module for controlling the TP-Link Tapo P100 Plugs
MIT License
567 stars 139 forks source link

Unable to use the login() function since the latest Tapo P110 firmware update #76

Closed RvP2001 closed 1 year ago

RvP2001 commented 1 year ago

When calling the login() function, the following error occurs: "requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)" This error happens in line 159 of the file PyP100.py and is caused by r.json(). My assumption is that the latest firmware version 1.0.16 Build 220624 Rel. 171733 changed something in the response message but I cannot figure out what it is exactly

NyquistFX commented 1 year ago

Same issue.. I have a bunch of P110's that I pull energy usage from and collate data. Nothing else has changed, but I did go ahead and update firmware on all plugs yesterday. Since then no data is logged and if I dig into the problem I get the same error when p110.login() is called. My expertise is SQL and PHP dealing with the collated data, not Python :|

The full error generated from my Python script..

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Program Files\Python39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Python39\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\Python39\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Scripts\EnergyTest.py", line 38, in <module>
    p1103.login()
  File "C:\Program Files\Python39\lib\site-packages\PyP100\PyP100.py", line 160, in login
    decryptedResponse = self.tpLinkCipher.decrypt(r.json()["result"]["response"])
  File "C:\Program Files\Python39\lib\site-packages\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Whether that is any use to what has already been provided. Any input very much appreciated :)

vladak commented 1 year ago

Happens to me to. I have a P110 with older firmware that works fine and bought a new P110 for which the firmware update was performed which no longer works.

The trouble is that the P110 with the newer firmware (1.0.16 in my case) returns plain HTML to the HTTP POST request done in login():

<html><body><center>200 OK</center></body></html>

instead of JSON payload like it used to be in the 1.0.7 version:

{"error_code":0,"result":{"response":"... BASE64 encoded encrypted data located here ..."}}
vladak commented 1 year ago

To experiment a bit with firmware 1.0.7, I changed the value of https://github.com/fishbigger/TapoP100/blob/4e0458ff300102f630d953ebb463bfd7dff05f7a/PyP100/PyP100.py#L146 (so that self.cookie was ".") and also https://github.com/fishbigger/TapoP100/blob/4e0458ff300102f630d953ebb463bfd7dff05f7a/PyP100/PyP100.py#L149 (so that encrypt() was called with ".") and in both cases it resulted in the 200 OK HTML output.

My guess is that this is a reaction of the firmware to invalid input. It seems that something in the login process has changed. This probably needs to be re-reverse engineered.

Because the Tapo Android app works with both firmware versions, I wonder if it switches the payload based on the firmware version (it can gather the version before starting the handshake/login in the same way PyP100.getDeviceList() does).

pcollinson commented 1 year ago

This is happening with P100s too. The response to the POST in login() seems to be: <html><body><center>200 OK</center><body></html>__

pcollinson commented 1 year ago

Well a bit of investigation. The plugs are saying SHIP 2.0 in the response field. I'm suspecting that they are now using the Smart Home IP protocol - which is part of the EEBUS environment. This guess is somewhat reinforced by a packet dump I took on my network with the IOS client running. It shows conversations between my devices and an Amazon client to make the plugs change state. So the app is not talking to the plug, but is requesting a change from their server which in turn is changing the plug state down a webconnect link. SHIP is also used by my heating system.

SHIP is pretty well hidden, although eebus.org does allow you to download a spec of the protocol, it's xml embedded in JSON.

I think that the '200 OK' response is a 'you aren't talking my protocol' message.

However, this is all deeply frustrating.

tking2 commented 1 year ago

@pcollinson From inspecting traffic on my own plugs, the Server header with SHIP 2.0 is present prior to the firmware update, so I don't think anything has changed here.

It does appear to solely be related to login, as handshake works OK, and captured traffic implies switching power is the same, although I haven't been able to capture the handshake from the TAPO app to grab the AES key to decrypt. But I expect this is likely a payload structure change rather than a protocol change.

If anyone has the ability to dump the latest fimrware, I can take a look to identify how the login protocol or payload structure has changed.

tking2 commented 1 year ago

Turns out nothing had changed! Kicked myself when I figure out the mistake.

It appears that new firmware versions perform stricter validation on the session cookie. In a previously harmless mistake, a semicolon was kept on the end of the session cookie. P100 v1.1.0 and P110 v1.0.16 now treat this as invalid.

Tested and can now control plugs again after applying PR!

pcollinson commented 1 year ago

Yay! Back in business.

vladak commented 1 year ago

Confirmed - the change in the PR fixes the problem.

NyquistFX commented 1 year ago

Many thanks to all! Applied the update and getting data instead of errors.. Happy days!

lemoustachiste commented 1 year ago

So I'm pretty unlucky, I received the tp110 2 days ago, set up the raspberry pi and tplink account yesterday (updating the firmware), today I'm working on my python script when I get this error...

Any idea when the fix will be released?

Thanks

vladak commented 1 year ago

So I'm pretty unlucky, I received the tp110 2 days ago, set up the raspberry pi and tplink account yesterday (updating the firmware), today I'm working on my python script when I get this error...

Any idea when the fix will be released?

Thanks

depends on @fishbigger , maybe it will help if you support him via the http://buymeacoffee.com/fishbigger on his profile page.

RvP2001 commented 1 year ago

Thanks guys! I changed self.cookie = r.headers["Set-Cookie"][:-13] in line 128 of PyP100.py to self.cookie = r.headers["Set-Cookie"][:-14] and now it works again! @lemoustachiste maybe you can try this as well?