TP-Link Integration Not Pulling In HS103 Switches

JZ-SmartThings commented 4 years ago

The problem

Added and removed the TP-Link integration multiple times but only getting 6 out of 8 switches to show up in HA.

Environment

Home Assistant Core release with the issue: 0.115.2
Last working Home Assistant Core release (if known):
Operating environment (OS/Container/Supervised/Core): Ubuntu 20.04 running Supervised
Integration causing this issue: TP-Link
Link to integration documentation on our website: https://www.home-assistant.io/integrations/tplink/

Problem-relevant `configuration.yaml`

I removed all YAML config that I had for TP-Link and used the front-end UI for all configuration. HA has been reset and rebooted multiple time.

Traceback/Error logs

Not sure what to enable for further logging from TP-Link. I have however hit the API site to show the JSON that's coming back. My two switches are the "lite" switches all the way at the top. The JSON is below... the "Smart Wi-Fi Plug Lite" are the ones not making their way into HA.

Additional information

{"error_code":0,"result":{"deviceList":[
{"deviceType":"IOT.SMARTPLUGSWITCH",  "role":0,"fwVer":"1.1.2 Build 191113 Rel.095623","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Smart Wi-Fi Plug Lite","deviceHwVer":"2.1","alias":"Garage Volcano","deviceMac":"XXXXXXXXXXX","oemId":"24B27E003E98D053CF8B51CE8086FDA8","deviceModel":" ","hwId":"18967AEED1C89BED1BBC7F62FD06468A","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTPLUGSWITCH",  "role":0,"fwVer":"1.1.2 Build 191113 Rel.095623","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Smart Wi-Fi Plug Lite","deviceHwVer":"2.1","alias":"Garage Parent Switch","deviceMac":"XXXXXXXXXXX","oemId":"24B27E003E98D053CF8B51CE8086FDA8","deviceModel":"HS103(US)","hwId":"18967AEED1C89BED1BBC7F62FD06468A","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTPLUGSWITCH",  "role":0,"fwVer":"1.2.6 Build 200727 Rel.121701","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Wi-Fi Smart Plug With Energy Monitoring","deviceHwVer":"1.0","alias":"TPLink Power","deviceMac":"XXXXXXXXXXX","oemId":"FFF22CFF774A0B89F7624BFC6F50D5DE","deviceModel":"HS110(US)","hwId":"60FF6B258734EA6880E186F8C96DDC61","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTPLUGSWITCH",  "role":0,"fwVer":"1.5.6 Build 191114 Rel.104204","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Smart Wi-Fi Plug Mini","deviceHwVer":"1.0","alias":"Soldering Iron","deviceMac":"XXXXXXXXXXX","oemId":"003E098AF0D44D4BAB796B3F6A7A830E","deviceModel":"HS105(US)","hwId":"E5D7E6089B060EF662783C23AE110522","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTPLUGSWITCH",  "role":0,"fwVer":"1.5.6 Build 191114 Rel.104204","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Smart Wi-Fi Plug Mini","deviceHwVer":"1.0","alias":"Powerline Ethernet Switch","deviceMac":"XXXXXXXXXXX","oemId":"003E098AF0D44D4BAB796B3F6A7A830E","deviceModel":"HS105(US)","hwId":"E5D7E6089B060EF662783C23AE110522","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTBULB"     ,  "role":0,"fwVer":"1.8.11 Build 191113 Rel.105336","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Smart Wi-Fi LED Bulb with Tunable White Light","deviceHwVer":"1.0","alias":"Nightstand Light","deviceMac":"XXXXXXXXXXX","oemId":"3444835A20788C514CB24B9F869A0A5E","deviceModel":"LB120(US)","hwId":"111E35908497A05512E259BB76801E10","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTPLUGSWITCH",  "role":0,"fwVer":"1.2.6 Build 200727 Rel.121701","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Wi-Fi Smart Plug With Energy Monitoring","deviceHwVer":"1.0","alias":"Office AC Plug","deviceMac":"XXXXXXXXXXX","oemId":"FFF22CFF774A0B89F7624BFC6F50D5DE","deviceModel":"HS110(US)","hwId":"60FF6B258734EA6880E186F8C96DDC61","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1},
{"deviceType":"IOT.SMARTPLUGSWITCH", "role":0,"fwVer":"1.5.11 Build 200214 Rel.152651","appServerUrl":"https://use1-wap.tplinkcloud.com","deviceRegion":"us-east-1","deviceId":"XXXXXXXXXXXXXXX","deviceName":"Smart Wi-Fi Dimmer","deviceHwVer":"1.0","alias":"Hallway Bathroom Light","deviceMac":"XXXXXXXXXXX","oemId":"3B13224B2807E0D48A9DD06EBD344CD6","deviceModel":"HS220(US)","hwId":"84DCCF37225C9E55319617F7D5C095BD","fwId":"00000000000000000000000000000000","isSameRegion":true,"status":1}]}}

probot-home-assistant[bot] commented 4 years ago

Hey there @rytilahti, @thegardenmonkey, mind taking a look at this issue as its been labeled with an integration (tplink) you are listed as a codeowner for? Thanks! _{^{(message by CodeOwnersMention)}}

rytilahti commented 4 years ago

Related potentially to https://github.com/python-kasa/python-kasa/issues/42 - how to fix this, or whether it is even fixable, remains a mystery with no updates from anyone affected by the problem.

The json you included in "additional information" is related to the cloud connectivity, pyhs100 (and python-kasa, which is where the development is now being done) is only aimed for local control.

JZ-SmartThings commented 4 years ago

@rytilahti thanks for getting back to me. I see what the options are at this point... "local only" config may work and obviously I should have ignored the firmware upgrade :) oh well. This should probably warrant a bit of documentation change as the model I'm using was theoretically supported but stuff changed. All the best!

JZ-SmartThings commented 4 years ago

Hey @rytilahti here's a thought... please let me know if feasible. Using the TP-Link API and token, I'm easily able to trigger the plug. It would be great if this integration allowed for a MANUAL token to be provided for the devices that are not able to work locally, etc. Let me know what you think...

One more thing: the switches are on a different WIFI and subnet --- for IoT devices. They only have access to the web and my stuff can access that subnet but it cannot access back to my secure/home network.

Thanks!

rytilahti commented 4 years ago

Hi @JZ-SmartThings, did you already try to make it use local only mode (as described in https://www.tp-link.com/us/support/faq/2707/)?

re: documentation. PRs are more than welcome!

re: usage over cloud API. This is not something that's planned, and it would require changes on several layers to make it function. I don't know how the cloud api works, is it even using the same protocol? If the protocol is the same, just with a different transport, we can discuss if it makes sense to integrate that to python-kasa. Otherwise, I think it the cloud implementation should be a separate integration.

On your last thing, I'm not sure what do you mean with that? The only functionality that requires devices to be in the same subnet is the discovery, so as long as communication is allowed across subnets, static configuration can be used.

JZ-SmartThings commented 4 years ago

@rytilahti thanks again for everything! I resolved it... Few points and one last question:

All switches areon the latest US firmware on these HS103's as of 2020-10-26
My switches are located on my IoT wifi network --- IoT network cannot access my home wifi network, but home can access IoT. This is probably my problem. Don't feel like breaking stuff down for testing other models, it may not just be HS103 related.
The solution was to throw it into the config YAML file with the specific IP addresses of each device. After a restart of HA, they showed up. That also means that your "local only" recommendation would have worked as well.

Question for you: how do clear the TP-Link integration's token/credential information which must be either stored on disk or the DB? To clarify: I'm never able to re-type my credentials when removing then re-adding the TP-Link integration. Just removed and re-added it using Edge chromium which I cleared fully (cache, etc.) and adding the integration back didn't prompt for credentials. Seems that they're getting saved on the back-end...

rytilahti commented 4 years ago

@JZ-SmartThings you're welcome, glad you got it working!

The discovery works only inside the broadcast domain (i.e., the same subnet), so that could be a reason why they were not detected before manual configuration. But it still leaves me wondering how some of the devices were detected?! There was a PR to add configurable subnets for use-cases like yours (#29505), but it never got merged.

Unfortunately I'm not sure what to answer to your question, as there are no token/credentials anywhere in this integration (it's all local communication using merely the ip address of the device). Are you sure you are using the integration shipped with homeassistant? Where did you get that log you posted under additional information in the initial issue description? :-)

JZ-SmartThings commented 4 years ago

@rytilahti here is the source of my info: https://gist.github.com/MichaelWhi/89fe680a233927a6aa368574d71627f7

Stuff like this may also be of interest... it's the REST API for geting device info, turning switch on and off, etc.: https://itnerd.space/2017/01/29/get-the-state-of-your-hs100-smart-plug/

This capability of accessing the cloud API is what I was talking about earlier... but your LOCAL way is better :)

I also see this subject came up before, I just didn't realize that this was the crux of the issue: https://community.home-assistant.io/t/tp-link-devices-on-different-networks/103952

Take care!

rytilahti commented 4 years ago

Ah, I see. The integration (and the backend lib) is only for local communication, so the cloud API is not really interesting here. I'm again glad you got it working, feel free to create a PR to improve the documentation if you think that could be helpful for others :-)

Belval commented 4 years ago

Related potentially to python-kasa/python-kasa#42 - how to fix this, or whether it is even fixable, remains a mystery with no updates from anyone affected by the problem.

The json you included in "additional information" is related to the cloud connectivity, pyhs100 (and python-kasa, which is where the development is now being done) is only aimed for local control.

As someone who just bought HS103 because they were in the compatible list. I think the documentation should be updated to reflect that python-kasa and by extension home-assistant does not in fact support the HS103.

JZ-SmartThings commented 4 years ago

@Belval I think it will work but you have to define the IP in the configuration.yaml file... I just tested again before writing this and indeed turning it on in HA does reflect in the Kasa app and vice-versa.

Belval commented 4 years ago

Just tried and you are right! So the discovery parameter simply stopped working or is this a problem only on some networks? In the former case I can make a PR to add the info to the official docs.

rytilahti commented 4 years ago

Hm, I forgot that pyhs100 (the library homeassistant integration is using) already also used to query several modules for discovery. Please feel free to create a doc PR, I think this is related https://github.com/python-kasa/python-kasa/issues/105 – seemingly some newer firmwares fail when one tries to query the emeter information alongside the status.

home-assistant / core