HS100 Hardware V4.1 Firmware 1.1.0 - No Detection Even When IP Manually Specified

ghostseven commented 3 years ago

Maybe a duplication or similar to https://github.com/plasticrake/homebridge-tplink-smarthome/issues/153

All HS100 Hardware V4.1 devices suddenly stopped working (work fine from Kassa App). Have reset them, have tried different versions of plug-in and config settings. Even manually specified devices. I have also reset homebridge. They now do not discover or show up.

I think this is possibly due to a firmware update from TP Link. I have a selection of HS100 devices some on hardware 2.0 and hardware 2.1 (these are all working).

Expected Behavior

The device appear in discovery or when manually specified

Current Behavior

Device does not appear

Steps to Reproduce (for bugs)

Setup a HS100 with hardware V4.1 and Firmware 1.1.0 device will not appear.

Versions

Node: node --version v12.19.0
Homebridge: homebridge --version 1.2.3
OS: Linux nash 5.4.60-1-pve #1 SMP PVE 5.4.60-1 (Mon, 31 Aug 2020 10:36:22 +0200) x86_64 x86_64 x86_64 GNU/Linux

Configuration

        {
            "name": "TplinkSmarthome",
            "addCustomCharacteristics": false,
            "deviceTypes": [
                "plug",
                "bulb"
            ],
            "devices": [
                {
                    "host": "192.168.0.155"
                },
                {
                    "host": "192.168.0.154"
                },
                {
                    "host": "192.168.0.153"
                },
                {
                    "host": "192.168.0.150"
                }
            ],
            "platform": "TplinkSmarthome"
        }

Homebridge Log / Command Output

Justanotherhomosapien commented 3 years ago

Same here. Took quite some time to understand what was happening - half my devices had automatically updated their firmware (and were no longer working) and the others were running as normal. Updated the others and now nothing can be switched on/off, however, at least its all consistent!

ghostseven commented 3 years ago

My older hardware version devices do not have firmware updates so hopefully they will continue to function.

Something has changed under the hood, hopefully it is resolvable. Suspect some packet sniffing in the future. If I get time I will try and what is sent between the different devices.

At least it looks like we know where the problem is located!

plasticrake commented 3 years ago

Can you confirm you're using v6.1.0 of the plugin? Where did you purchase your plugs? I may need to get one...

ghostseven commented 3 years ago

Can you confirm you're using v6.1.0 of the plugin? Where did you purchase your plugs? I may need to get one...

Yep confirmed 6.1.0. They were purchased from Amazon UK a while back. If needs be I can pop one in the post to you but will need to check postage costs to the USA.

If it is not rude to ask how did you reverse engineer the protocol, wireshark packet watching? I can see if I can capture data for you at the very least.

plasticrake commented 3 years ago

@ghostseven, it may be helpful if you can install tplink-smarthome-api and run some of the commands to see if your devices respond to them. If I can get some basic output from your device I can use my tplink-smarthome-simulator project to mimic a real device and see how the kasa app communicates with it.

For example:

Install:

npm --global tplink-smarthome-api

Discovery: (this sends out a UDP broadcast that devices are supposed to respond to)

tplink-smarthome-api search

Do you see your devices responding? If not, try setting a more specific broadcast address (perhaps 192.168.0.255 based on the IPs I see in your comments)

tplink-smarthome-api search --broadcast 192.168.0.255

If you get any results from getSysInfo that would be most helpful, please post your results!

tplink-smarthome-api getSysInfo 192.168.0.150

plasticrake commented 3 years ago

@ghostseven It's been years since I used wireshark for this, but someone made a dissector to decrypt the (very simple) encryption. https://github.com/softScheck/tplink-smartplug

Hopefully TP-Link didn't change it in the new firmware!

Justanotherhomosapien commented 3 years ago

Can you confirm you're using v6.1.0 of the plugin? Where did you purchase your plugs? I may need to get one...

6.1.0 confirmed for me also

Justanotherhomosapien commented 3 years ago

Looks like port 9999 is no longer open

/ # tplink-smarthome-api getSysInfo 192.168.0.83 Sending getSysInfo command to 192.168.0.83: via tcp... TCP 192.168.0.83:9999 Error: connect ECONNREFUSED 192.168.0.83:9999 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) { errno: 'ECONNREFUSED', code: 'ECONNREFUSED', syscall: 'connect', address: '192.168.0.83', port: 9999 } Error: Error: connect ECONNREFUSED 192.168.0.83:9999 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) { errno: 'ECONNREFUSED', code: 'ECONNREFUSED', syscall: 'connect', address: '192.168.0.83', port: 9999 }

Justanotherhomosapien commented 3 years ago

And next bit of investigation, only port 80 appears open

/ # nmap -p0-65535 192.168.0.83 Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-11 22:40 UTC Nmap scan report for 192.168.0.83 Host is up (0.0025s latency). Not shown: 65535 closed ports PORT STATE SERVICE 80/tcp open http

plasticrake commented 3 years ago

@Justanotherhomosapien Try it on port 80:

tplink-smarthome-api getSysInfo 192.168.0.83:80

plasticrake commented 3 years ago

@Justanotherhomosapien And try nmap with UDP -sU

ghostseven commented 3 years ago

@Justanotherhomosapien Try it on port 80:
tplink-smarthome-api getSysInfo 192.168.0.83:80

I get timeouts when trying this against any of the plugs in question.

ghostseven commented 3 years ago

@Justanotherhomosapien And try nmap with UDP -sU

Result for UDP nmap scan

blake@nash:~$ sudo nmap -p0-65535 192.168.0.154 -sU Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:46 UTC Nmap scan report for BM-TVConsole.ghost7.com (192.168.0.154) Host is up (0.0058s latency). Not shown: 65535 closed ports PORT STATE SERVICE 20002/udp open|filtered commtact-http MAC Address: CC:32:E5:A6:E5:64 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 35.37 seconds blake@nash:~$

Result for TCP nmap scan

blake@nash:~$ sudo nmap -p0-65535 192.168.0.154 -sT Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:48 UTC Nmap scan report for BM-TVConsole.ghost7.com (192.168.0.154) Host is up (0.0044s latency). Not shown: 65535 closed ports PORT STATE SERVICE 80/tcp open http MAC Address: CC:32:E5:A6:E5:64 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 30.08 seconds blake@nash:~$

So we do have 20002 on UDP open as well

Justanotherhomosapien commented 3 years ago

@Justanotherhomosapien Try it on port 80:
tplink-smarthome-api getSysInfo 192.168.0.83:80

Timeouts here also I'm afraid.

/ # tplink-smarthome-api getSysInfo 192.168.0.83:80
Sending getSysInfo command to 192.168.0.83:80  via tcp...
TCP 192.168.0.83:80 Error: TCP Timeout after 10000ms
192.168.0.83:80 {"system":{"get_sysinfo":{}}}
    at Timeout._onTimeout (/usr/lib/node_modules/tplink-smarthome-api/lib/network/tcp-socket.js:59:36)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)
Error:
Error: TCP Timeout after 10000ms
192.168.0.83:80 {"system":{"get_sysinfo":{}}}
    at Timeout._onTimeout (/usr/lib/node_modules/tplink-smarthome-api/lib/network/tcp-socket.js:59:36)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)

Nmap UDP results

/ # nmap -sU -p0-65535 192.168.0.83
Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:47 UTC
Nmap scan report for 192.168.0.83
Host is up (0.0048s latency).
Not shown: 65535 closed ports
PORT      STATE         SERVICE
20002/udp open|filtered commtact-http

Justanotherhomosapien commented 3 years ago

So some Wireshark analysis (packet capture attached). When using the Kasa app, a UDP packet is sent to the broadcast address for the network on port 20002. This packet must contain some form of identifier for the smart switch in question, which receives the broadcast and then turns on/off.

A pretty massive change to the way that the switches have previously operated!

hs100 capture.pcapng.txt

ghostseven commented 3 years ago

I am going to sit down today with wireshark and see what I can find out. May not be until later though

ghostseven commented 3 years ago

I am getting nonsensical wireshark traffic from both the old (working) and newer plugs, I am fairly sure it is my setup as even with old working plugs I get nothing on port 9999. I am going to try and setup a raspberry pi as a wireless AP and direct cap from there. This will take a little longer though.

ghostseven commented 3 years ago

I have opened a ticket here with wireshark PCAP data, still does not make a lot of sense to me.

https://github.com/softScheck/tplink-smartplug/issues/81

Justanotherhomosapien commented 3 years ago

I have opened a ticket here with wireshark PCAP data, still does not make a lot of sense to me.

softScheck/tplink-smartplug#81

Thank you!

plasticrake commented 3 years ago

@Justanotherhomosapien not sure if that file you attached it right, its only 300bytes

ghostseven commented 3 years ago

@Justanotherhomosapien not sure if that file you attached it right, its only 300bytes

There is some wireshark data on the post I linked if that helps.

plasticrake commented 3 years ago

This info may be useful: https://www.thezdi.com/blog/2020/4/6/exploiting-the-tp-link-archer-c7-at-pwn2own-tokyo https://labs.f-secure.com/advisories/tp-link-ac1750-pwn2own-2019/

ghostseven commented 3 years ago

Thats is really interesting, I will read in more depth, I do wonder why I did not see any UDP traffic in my capture though. I wonder if I am doing something wrong.

Justanotherhomosapien commented 3 years ago

@Justanotherhomosapien not sure if that file you attached it right, its only 300bytes

That really was it - a single UDP broadcast was all that I could find happening.

I see that @ghostseven has uploaded what is probably a better capture file - I was going to download that to compare with my results.

nbargate commented 3 years ago

Just wanted to report the same issue, but didn't see the point of opening a new thread. I have 4 devices, three are on V1.1.0, one remains on V1.0.3.

The plug on V1.0.3 continues to work on HomeBridge. The rest are undetectable.

nowcodingaway commented 3 years ago

Commenting to confirm - same here.

plasticrake commented 3 years ago

Is anybody in in North America having this issue? None of my devices have a firmware upgrade, would like to see where I can acquire a device with newer firmware.

Justanotherhomosapien commented 3 years ago

Is anybody in in North America having this issue? None of my devices have a firmware upgrade, would like to see where I can acquire a device with newer firmware.

Not sure how to help with that part - even if I send you a UK model, I'd also need to send a UK power outlet & you'd have to find 240V to hook it up to :(

nbargate commented 3 years ago

Is anybody in in North America having this issue? None of my devices have a firmware upgrade, would like to see where I can acquire a device with newer firmware.

I can't offer much, other then remote diagnostics if you need anything. I have 3 on the new firmware and 1 on the old.

plasticrake commented 3 years ago

Related: home-assistant/core#43088

simonsaysjake commented 3 years ago

Is anybody in in North America having this issue? None of my devices have a firmware upgrade, would like to see where I can acquire a device with newer firmware.

Yes, same issue in NA and happy to help as I can (not too technical)

si-eds commented 3 years ago

Also having this issue since one of my plugs updated... no errors in Home or Homebridge (shows as on and off)... but the plug does not physically turn on or off.

brendonrogers commented 3 years ago

Similar discussion over at Home Assistant https://github.com/home-assistant/core/issues/43088

tombassrosenfeld commented 3 years ago

I’m seeing the same. Hardware v1 firmware 1.2.5 working fine. Hardware 4.1 on 1.1.... nothing. No response to getSysInfo on the ip’s of the v4.1 models. Also in the UK.

If there are any tests I can run I’m very happy to help.

neilheyes commented 3 years ago

Same here too. Just stopped working and the HW is 4.1 and Firmware 1.1.0. My other 4 plugs are fine

smoke007 commented 3 years ago

My three model HS100, hardware version 1.0, with firmware 1.2.6 were all not working yesterday with Home Assistant, but this morning, all started working again. Port 9999 (abyss) was open yesterday on each and is the same still today. I made no changes over night, power cycled the HS100, and had tried rebooting Home Assistant several times yesterday with no change. They each worked normally yesterday from the Kasa app.

andy-dinger commented 3 years ago

Apparently TP Link closed the local port 9999 and now force all traffic to go via the encrypted webserver on port 80. There seems to be some work going on here to find a workaround if that helps: https://github.com/python-kasa/python-kasa/pull/117

andy-dinger commented 3 years ago

Is anybody in in North America having this issue? None of my devices have a firmware upgrade, would like to see where I can acquire a device with newer firmware.

I’m in the UK with a (now almost useless) 1.1.0 plug. I’d be happy to help with any testing or debugging you need

mathowie commented 3 years ago

FWIW, I'm getting the same failure errors in my US-based HS200 and HS210 wall switches. All updated to to 1.5.7 (2.0 hardware) and nothing shows up in homebridge or HOOBs using any version of the tplink plugin.

MattHardwick34 commented 3 years ago

Are we any closer on sorting these? I've got a bunch of lights and devices that aren't usable now! Hope we can get somewhere soon! <3

ghostseven commented 3 years ago

Yes progress is being made with https://github.com/python-kasa/python-kasa

There is a proof of concept working version that is being tested and will be integrated into the main library. When that is complete and functional I suspect that this will be used as a reference for this project.

Edit: I originally incorrectly stated that python-kasa was what this project was based on and that is totally incorrect. My apologies.

MattHardwick34 commented 3 years ago

Thanks for the update !

plasticrake commented 3 years ago

This plugin is not related to and does not use python-kasa. However I will be using what people have learned to try to add support for the new firmware since I am unable to get a hold of a device with the new firmware myself.

MattHardwick34 commented 3 years ago

@plasticrake where are you based? Also if you have HW version 4.1 HS110 the latest firmware from the KASA app will give you what you want.

ghostseven commented 3 years ago

@plasticrake my apologies, sorry I made a bad assumption! Sorry to state your plugin was based on this!

throwingspaghetti commented 3 years ago

@plasticrake I have a spare plug that I can send your way if it helps with the integration...

mankittens commented 3 years ago

Issue is affecting me as well.

It's a damn shame that Kasa made breaking changes to their API. Should've come out with a new SKU... I'm gonna sell all my Kasa devices--already purchased some Meross ones to replace them.

borez commented 3 years ago

Got hit with this issue, after my plug auto updated to firmware 1.1.

Do note that Kasa can retroactively flash back older firmware, on request.

https://community.tp-link.com/en/home/forum/topic/237614

throwingspaghetti commented 3 years ago

@plasticrake let me know if you need any help. Happy to ship a hs2xxx switch or perform diagnostic steps on my end if you need. I have about 50 Kasa switches installed.

nowcodingaway commented 3 years ago

Is this a lost cause now? Time to jump ship from the TP Links?

plasticrake / homebridge-tplink-smarthome