Home Assistant support - Githubissues

clach04 commented 6 years ago

Looks like there are a few people interested in HA support, and at least one implementation https://github.com/home-assistant/home-assistant/pull/11000 in the works. Opening ticket as a place to jot notes down for sharing.

Custom component:

https://github.com/sean6541/tuya-homeassistant

clach04 commented 6 years ago

CC @nijave, @ImNightwing

fastrax commented 6 years ago

Hi, Apologies if this isn't the right place for this. I have also posted on the Home Assistant Forums.

I have just bought a 2 gang light switch from Aliexpress (https://bit.ly/2qirAM8 1). It’s model number is SM-SW102U.

https://fccid.io/2AJ5F-SM-PW701U/Letter/Model-Differences-Description-3200194/ is documentation from Xenon to the FCC declaring “models SM-PW701U, SM-PW702, SM-SW801, SM-SW102, SM-S0301, SM-PZ701, SM-PZ702, SW-SZ801, SM-SZ202, and SM-SZ301 contain identical circuitry”.

I thought that I would easily get this up and running using the existing Tuya work that has been done. https://github.com/sean6541/tuya-homeassistant / https://github.com/clach04/python-tuya 1

Unfortunately, the tuya protocol on this switch seems to be slightly different. I have used WireShark whilst registering the device and the packets are different. See below:

POST /gw.json?a=s.gw.token.get&gwId=01200950ecfabc795eb3&other={“token”:“7wBxo260”,“region”:“AZ”,“tlinkStat”:{“configure”:“smartconfig”,“time”:6,“source”:“station”,“path”:“multicast”}}&t=7&v=3.0&sign=cd5c887908064c32c8f8185ac32ffbf5 HTTP/1.1

POST /gw.json?a=s.gw.dev.pk.active&gwId=01200950ecfabc795eb3&other={“token”:“7wBxo260”}&t=1523597921&v=3.0&sign=84da2c27ddbf551ec2f0938c0790c7d6 HTTP/1.1 (application/x-www-form-urlencoded)

POST /gw.json?a=s.gw.update&gwId=01200950ecfabc795eb3&t=1523598924&v=2.0&sign=01468796fe75f6b66f2e5a8e67533349 HTTP/1.1 (application/x-www-form-urlencoded)

POST /gw.json?a=atop.online.debug.log&gwId=01200950ecfabc795eb3&t=1523578928&sign=56d0cf5a79efac8fc4fde30f7a4740e5 HTTP/1.1 (application/x-www-form-urlencoded)strong text

POST /gw.json?a=s.gw.dev.timer.count&gwId=01200950ecfabc795eb3&t=1523578939&sign=dc95e227bbd7bd6ca5036e1c68cd99f4 HTTP/1.1 (application/x-www-form-urlencoded)

I now know the host, device_id, but cannot seem to identify to the local_key from what I have from WireShark.

I have also used adb logcat and my phone, it has provided the above. Still none the closer…

05-19 20:46:39.707 17272 28787 I ReactNativeJS: Running application “TYRCTApp” with appParams: {“initialProps”:{“devInfo”:{“isShare”:false,“panelConfig”:{“bic”:"[{“code”:“timer”,“selected”:true}]"},“isLocalOnline”:true,“bv”:“5.12”,“verSw”:“1.2.0”,“isOnline”:true,“uiPhase”:“release”,“isVDevice”:false,“dps”:{“1”:false,“2”:false},“schema”:{“1”:{“type”:“obj”,“name”:“Wall Switch 1”,“mode”:“rw”,“code”:“switch_1”,“id”:“1”,“schemaType”:“bool”,“iconname”:null,“property”:"{“type”:“bool”}"},“2”:{“type”:“obj”,“name”:“Õ╝ÇÕà│2”,“mode”:“rw”,“code”:“switch_2”,“id”:“2”,“schemaType”:“bool”,“iconname”:null,“property”:"{“type”:“bool”}"}},“productId”:“At6C3S6JYzp4ez1T”,“nodeId”:"",“ability”:0,“ui”:“00000001xx_1.2.3”,“uiConfig”:{},“gwId”:“01200950ecfabc875eb3”,“networkType”:“WIFI”,“t”:1526719598,“icon”:“https://images.tuyacn.com/smart/icon/1478860503_0.jpeg",“name”:"Wall Switch “,“uiId”:“00000001xx”,“pcc”:””,“isUniversalPanel”:false,“devId”:“01200950ecfabc875eb3”}},“rootTag”:1}. DEV === false, development-level warning are OFF, performance optimizations are ON

Any help or guidance would be hugely appreciated.

clach04 commented 6 years ago

@fastrax this is not the best issue to discuss registration (either open a new ticket here or https://github.com/codetheweb/tuyapi/ to avoid fragmentation).

https://github.com/clach04/python-tuya/wiki is probably your best bet for how to get the key, if you can figure out how to use it with Wireshark that would be great (it would be great if you could record how you do it) as its not a known technique. The wiki (and links to the readme in the upstream javascript project) covers the techniques that are known to work. It was discovered recently that using the latest version of the vendors app no longer revealed the key :(

Exilit commented 5 years ago

In order to be able to create homeassistant support following their best practice advices python-tuya needs to be available on pypi.

@clach04 Do you have any plans to create a pypi package. Or the other way around: Do you see any reasons not to do so?

clach04 commented 5 years ago

No reasons not to do this other than time :(

I just sent you a collaborators invite. ALso created Issue #34

nijave commented 5 years ago

sean6451 packaged the code and uploaded to pypi a while back and we had some progress on getting this working with home assistant but I kept getting hard to reproduce connection issues (especially those that would crop up on HA but not testing on my laptop with Wireshark open). I suspect there's still something subtle missing. When the app streams requests it uses a different method than constantly polling (it keeps a socket open) and I think that might be related

https://github.com/home-assistant/home-assistant/pull/11000

Edit looks like you own the pypi project now

nijave commented 5 years ago

Also I'd be interesting in helping again--I ended up getting Sonoff switches but still have some Voion and those and the other related brands are usually pretty cheap. I also have a pytuya+HA fork on Github somewhere slightly different than sean6541

I'm guessing there's a way to set this up with Travis and hook into pypi for auto publishing but I've never tried to get that setup before

Exilit commented 5 years ago

I saw this package and it looks up to date which is maintained by @sean6541. I'd still like to add the setup.py to the repository.

caffeinatedMike commented 5 years ago

@clach04 A couple questions

After reviewing the code a bit more, am I right to assume that this repo & yours work on a local level and do not interact with Tuya's official api. In short, will these implementations allow completely local control over the devices without needing internet access (in the event of say an internet outage)?
Looking at your repo it isn't quite clear what devices are covered (maybe the readme should be updated to specifically state it). If I had to venture a guess, it seems to only support switches? I would like to see a version that also implements the door/window sensors @NorthernMan54 has been working to incorporate as those are the main two classes of tuya's products that i'll be using.
Do you have plans to submit this as a replacement of the current Tuya component of HA, so that we can have a truly local implementation with no cloud interaction?

caffeinatedMike commented 5 years ago

And @NorthernMan54 would you be willing/able to port over your implementation (once finished) from your nodejs version to python or do you not have the necessary knowledge of python? Unfortunately, I don't have Home Assistant setup yet & don't have the devices to be able to test myself. I more so jumped in when I did to make sure they're supported before I selected them for my (planned) setup.

nijave commented 5 years ago

@caffeinatedMike yep, this version is local (uses a sort of serial protocol over TCP). I tried to integrate with HA but it wasn't reliable enough (there would be random socket/hard disconnect issues). Looking at the Tuya Android App, it uses a different command to stream status from the device instead of repeatedly hammering for single updates (HA polls every 30 seconds for a status update in case it's toggled manually) which I think might be related

The device was hard closing the TCP connection (either requests were too fast, something was invalid, etc)

nijave commented 5 years ago

If you look at https://github.com/clach04/python-tuya/blob/master/pytuya/__init__.py it has BulbDevice and Switch extending the base device so it should work with those two although I think some devices have minor variations

caffeinatedMike commented 5 years ago

@nijave That's good to hear (about the local aspect). Any ideas on possible solutions for the disconnects/unreliability at this point? Have you tried letting Packet Sniffer on android run for maybe 5 minutes with the Smart Life app open to see the activity? Probably best to do so with an Android on 5.0 (marshmallow) because from what I heard the app is able to even sniff ssl on that OS, but can't with the more recent OSes.

nijave commented 5 years ago

Yeah, the device randomly sends TCP RSTs for no apparent reason

caffeinatedMike commented 5 years ago

My guess is those are heartbeat requests (to maintain the consistent connection to their api/servers). That's how they're able to react so fast to state changes/requests. @NorthernMan54's update mimics this activity using a sort of what he calls an EventEmitter. Have a look here. This is how he also managed to get door/window sensors working, by monitoring the events with constant local polling.

nijave commented 5 years ago

HA is sending "heart beat" requests, but I have no idea why these will randomly start failing (the device will kill the TCP connection forcibly mid-request). The app uses a different command to stream updates instead of constant single polls like this module. The device uses MQTT for connecting to the Tuya cloud service which is how it responses immediately

I'm sure there's a workaround to get them working, but I stopped messing around and got some Sonoff switches that work much better with custom firmware

caffeinatedMike commented 5 years ago

So, I guess it's safe to say that local HA integration has been abandoned until someone else wants to take it upon themselves? That sucks :frowning_face:

NorthernMan54 commented 5 years ago

Nick,

In my investigation with wifi dimmers, I found that I needed to send a heartbeat every 15 seconds to hold the socket open. Then any events where sent in real time. I haven’t looked at your code yet, so don’t know if you are doing this. I also added retry logic to handle the odd rst packet etc.

Michael,

I think somewhere the wires got crossed on what I was working with. I have wifi dimmer switches, that I’m looking to pass local events ie button pushes up to HomeKit with. Theoretically this pattern should also work with contact and motion sensors as well, but I don’t have any from Tuya so I can’t say for sure.

On Sep 28, 2018, at 10:22 AM, Michael Hill notifications@github.com wrote:

So, I guess it's safe to say that local HA integration has been abandoned until someone else wants to take it upon themselves? That sucks ☹️

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

nijave commented 5 years ago

Yeah, this doesn't hold the socket open. The app uses a different command which basically does that. Not sure if you can use the same command while trying to hold the socket open

By command see hexByte around line 117 in payload dict https://github.com/clach04/python-tuya/blob/master/pytuya/__init__.py

On Fri, Sep 28, 2018 at 12:56 PM Northern Man notifications@github.com wrote:

Nick,

In my investigation with wifi dimmers, I found that I needed to send a heartbeat every 15 seconds to hold the socket open. Then any events where sent in real time. I haven’t looked at your code yet, so don’t know if you are doing this. I also added retry logic to handle the odd rst packet etc.

Michael,

I think somewhere the wires got crossed on what I was working with. I have wifi dimmer switches, that I’m looking to pass local events ie button pushes up to HomeKit with. Theoretically this pattern should also work with contact and motion sensors as well, but I don’t have any from Tuya so I can’t say for sure.

On Sep 28, 2018, at 10:22 AM, Michael Hill notifications@github.com wrote:

So, I guess it's safe to say that local HA integration has been abandoned until someone else wants to take it upon themselves? That sucks ☹️

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/clach04/python-tuya/issues/24#issuecomment-425499659, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXd2vK0W5hGJ8F1CiQyvtNKKjvn0BWjks5uflS9gaJpZM4Tepaa .

clach04 commented 5 years ago

nijave That's good to hear (about the local aspect). Any ideas on possible solutions for the disconnects/unreliability at this point? Have you tried letting Packet Sniffer on android run for maybe 5 minutes with the Smart Life app open to see the activity? Probably best to do so with an Android on 5.0 (marshmallow) because from what I heard the app is able to even sniff ssl on that OS, but can't with the more recent OSes.

@caffeinatedMike are you seeing disconnects/unreliability? For the devices I have, the only unreliability I've seen so far has been with the devices themselves. I've a few devices and whilst they are mostly reliable they occasionally do not respond to the official app (and they've been deliberately sequestered away from this and other 3rd party libraries), sometimes requiring a power cycle (maybe once every 3 months). If you do, take a look at https://github.com/codetheweb/tuyapi/issues/84#issuecomment-423820935 and tixi's experimental fork.

caffeinatedMike commented 5 years ago

@clach04 No, I've not had a problem yet. My switches are working well so far. Any chance we could incorporate door/window sensors into the custom HA component? 🙏

clach04 commented 5 years ago

@caffeinatedMike pytuya would need that support first (it sounds like those need long open sockets support based on some of the posts I've seen). I don't know how feasible that is. Its not on my todo list but patches are always welcome.

One other option would be to flash the sensor devices you have with a new (open) firmware that has HA support. I've not yet looked into this in much detail. I have a couple on my research list to check out:

https://github.com/xoseperez/espurna
https://github.com/arendst/Sonoff-Tasmota (I'm thinking about using this on a custom hardware project as it supports lots of devices out of the box, and needing to write custom gpio code as well as comms)

Related to HA. I did add a version string the other day and created a tag (https://github.com/clach04/python-tuya/archive/7.0.1.zip) so it should be possible to document the github dependency (without having a package on pypi). See https://developers.home-assistant.io/docs/en/creating_component_deps_and_reqs.html for more details

aleqx commented 4 years ago

@clach04 @nijave Sincere apologies for necromancing, but I could use your expertise and experience. I've wasted a lot of time with some Tuya smart plugs (haven't we all?) and HA using pytuya (i'm familiar with tuyapi). I just posted about it here: https://github.com/mileperhour/localtuya-homeassistant/issues/8 ... both posts are of interest.

I'd be grateful for some input given you obviously have more experience than me with the code (and also Tuya devices)

p.s. I might just fork the dosh and buy more TP-Link HS110 ... expensive and very bulky, but work well and the api is a breeze (clear json-rpc over tcp without any auth whatsoever).

TRSx80 commented 4 years ago

Another apology for necro! :)

@aleqx,

expensive and very bulky, but work well and the api is a breeze

No idea where your thoughts are these days, but there are (luckily) quite a few options nowadays:

433mhz - Very cheap, mostly reliable, pretty open, not clogging up your Wi-Fi network
Sonoff/Tasmota - Or other Open Source implementations on mostly ESP8266 with lots of relatively inexpensive hardware available
MySensors - Making your own custom devices out of Arduino (usually Pro Mini) + your choice nRF (or other) radios; also pretty inexpensive, not on Wi-Fi at all but their own network (could be pro or con, depending on your outlook :wink: ), but need a little knowledge (or be willing to learn) some basic electronics and radio propagation info...

clach04 commented 4 years ago

@aleqx just saw the update from @TRSx80 and I would also recommend Sonoff/Tasmota combo. I even flashed my Wuudi SM-S0301-US unit (which I had used with pytuya) with Tasmota, its such a more useful device with that firmware (and it doesn't lock up like the original firmware).

There are other firmwares but Tasmota worked out of the box for me so I've not dug much further into those. Re-flashing existing tuya devices varies in complexity.

Only issue with the Sonoff is the form factor does require some customization in many use cases (I have a plan to hack a sonoff into window fan and control it over GPIO - which shows this as a positive rather than a negative as its offers more options)

clach04 / python-tuya

Home Assistant support #24