SDNick484 / rectec_status

Scripts to talk to RecTec pellet smokers with WiFi controllers
Apache License 2.0
29 stars 4 forks source link

Support for Hassio & Home Assistant #3

Open chansearrington opened 4 years ago

chansearrington commented 4 years ago

Hey there,

I'm another RecTec lover (Bull) and really excited to come across this and wondering if you're interested in making this work for Home Assistant.

There are a few people over in the community that are looking for this.

I think it could help you achieve:

SDNick484 commented 4 years ago

Hi @chansearrington, the issue I was referencing on Reddit is this one. Please see if you can execute the script in this comment.

At this point what we know is my RT-680 with a WiFi controller (the one they released in spring/summer 2018 that needed to be manually swapped in) uses the older Tuya firmware that doesn't involve TLS. The Stampede (RT-590) that @impala454 has is definitely using the newer firmware; it's not clear which your Bull (RT-700) uses. It's also possible that it may depend on when you got your model; it's my understanding that the new TLS firmware controllers started showing up around Jan 2019 and it's certainly possible that the same Rectec line might have both types of controllers depending on purchase time.

I'm in the process of updating my laptop and just finished moving to current Node & Python. I'm also checking to see if any of my other Tuya products (primarily plugable smart outlets) are on the newer firmware (thus far they all seem to be on the older non-encrypted one).

chansearrington commented 4 years ago

@SDNick484 just did a quick port check and my RT-700 has an open port on 6668.

I'm looking into this script stuff now.

chansearrington commented 4 years ago

@SDNick484 sorry, my "coding" abilities are non-existant.

Can you confirm the below?

I assume that I need to save this as a file (ex "script.py") and then execute the file using Python 2.6 from a terminal (I'll probably use WSL)

# discover devices
# NOTE for my devices this only works with Python 2.6
# py3.6.1 runs but never reports packets

import socket
import struct

host_port = 'port_open_on_my_device'
host_ip = 'local_ip_address_of_my_device'

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
sock.bind(('', host_port))

mreq = struct.pack('4sl', socket.inet_aton(host_ip), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

try:
    while True:
        print('listening')
        data = sock.recvfrom(1024)
        raw_bytes, peer_info = data
        print(data)
finally:
    sock.close()
chansearrington commented 4 years ago

@SDNick484 I'm getting an error when I try to run this in Python 2.7 or 3

chansearrington@Surface-Studio-Home:~/rectec$ sudo python rectec_discover_device.py Traceback (most recent call last): File "rectec_discover_device.py", line 18, in <module> sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 22] Invalid argument

SDNick484 commented 4 years ago

Yep, that's the process, save it and execute. I don't have WSL experience, but I would suggest running: sudo apt install python3 python3-pip

Then try calling the script from Python 3

impala454 commented 4 years ago

@chansearrington can you link back to specific community posts you're referring to? I wouldn't mind following along there either. I have been going back and forth with @SDNick484 in the issue linked above, but curious where on Reddit or Home Assistant forums y'all are also communicating.

chansearrington commented 4 years ago

@SDNick Same error:

chansearrington@Surface-Studio-Home:~/rectec$ sudo python3 rectec_discover_device.py Traceback (most recent call last): File "rectec_discover_device.py", line 18, in <module> sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) OSError: [Errno 22] Invalid argument

chansearrington commented 4 years ago

@impala454 we've moved our conversation to here. You haven't missed anything except me talking him through the perks of Home Assistant :-)

SDNick484 commented 4 years ago

@chansearrington, it looks like that may be compatibility issue with how WSL presents sockets to Python. You may want to try the native Python for Windows instead. The port 6668 is good though, as that implies it's the same firmware version as mine.

@impala454, you haven't missed any developments yet. An idea I am toying around is registering my RT with Smart Life or Jinvoo instead of the native app (I've done that before and going back and forth shouldn't be an issue) and seeing if I can get the various cloud APIs for Tuya to work instead of going local.

impala454 commented 4 years ago

Some interesting reads I've found today: https://github.com/ct-Open-Source/tuya-convert - might be nice to get the firmware off the grill https://github.com/unparagoned/cloudtuya - allows control via user/pass, but haven't tried it yet.

impala454 commented 4 years ago

I spoke to RecTec tech support today and while they said currently they can't expose the APIs or anything, they did listen to my concerns that the only way to remotely control the grill is via a Chinese cloud server, and also that there's no real way to get the data from a cook besides looking at the graph. I think they will take these as requests moving forward and did say they have a dedicated person for working on the software.

SDNick484 commented 4 years ago

Yeah, I'm aware of tuya-convert and alternative firmware like Tasmota & ESPHome, but I am scared to try it on the RT (although if there's a RO way to get the firmware, that's interesting). Their wiki does warn that devices with secondary controller integrations (which would apply here) tend to need some tweaking, and I just don't have a spare RT to experiment on (or a wife that would approve of one). Now with that said, one of the main drivers for those alternative firmwares is to bypass foreign servers although personally I don't really care if they know my RT data (but yeah, there's obviously downsides to having stuff like that on my home network).

Yeah, cloudtuya is part of the approach I plan to look into. Most of the cloud client implementations (including HA) require you to pass some sort of indicator as to which cloud you're using (i.e. Smart Life, Jinvoo, etc.) and I'm not sure what to pass for RT, but I will be playing around with that.

That's great to hear they're receptive to your concerns, and yeah, I think a RFE to allow exporting the data to be a pretty fair request. I know a few folks on FB tried to pull Ray Carnes into the original video I posted demonstrating Alexa intergration but nothing came of it obviously.

SDNick484 commented 4 years ago

So I'm working through the setup instructions, have my Cloud API Authz and will try linking the device tomorrow evening. Once that's set up, I'll see what I can pull. The good news is if this works, it should be valid steps on either type of controller.

SDNick484 commented 4 years ago

Good news: I was able to register the grill using the setup instructions referenced previously. As I expected, this takes the device out of the Rectec app control however I also confirmed I can re-add it (but that will presumably break TuyaAPI control). Once I linked the device using the instructions, I was able to use TuyaAPI/cli to pull the same information I get via the python-tuya. I confirmed the information was being pulled locally an from the cloud. Next step will be trying the cloud client.

@impala454, it will be very interesting to see if this method works with the new firmware (it should). I suspect you'll need to pass the newer protocol.

impala454 commented 4 years ago

Awesome. I see where I went wrong in the steps previously, I hadn't "applied for cloud API Authorization". Will try it once my auth goes through.

SDNick484 commented 4 years ago

Cool, I think it only took like a day or so. I didn't see an email that I got my key, but I logged in and sure enough it was there. I haven't been able to get tuyapi-cloud to work yet, but I did validate that I can turn on and off the smoker via the tuya-cli. Likewise, if you wanted to pull specific temp values, it should be pretty trivial. Main issue will be figuring out how to move from local to the cloud/MQTT approach so we can avoid the issue of only one active connection.

SDNick484 commented 4 years ago

Couple updates: I confirmed I'm able to set target temperature via tuyapi/cli however I also confirmed it only works locally. I'm still researching the cloud approach however I did stumble upon this project and it gives me some good hope for HA integration. Since we already know the mapping of what the various DPS entries are for (i.e. 102 == Target Temp, 103 == Actual Temp, etc.) it should be easy to extend what @mileperhour is doing.

impala454 commented 4 years ago

I got my cloud API auth and attempted again. It does seem to actually be trying this time, however times out with Error: Timed out waiting for devices to connect.. I may not be 100% clear on the instructions but not following how the device is actually connecting. I'm running the command from a VM on my desktop. Does the CLI need to be run from a laptop or other device with a wifi card? The instructions specify (the device you're running this on can be connected to a different network than the one you want the Tuya device to join, as long as the connected network has at least one access point that will broadcast packets and the target network has a 2.4Ghz band), but I don't see how this could possibly work, as the grill in pairing mode generates its own wifi for the linking process. @SDNick484 did you do this from a device with a wifi card, and if so, did you notice it disconnecting/reconnecting to the grill during the linking process?

impala454 commented 4 years ago

Scratch all that, looks like the CLI link does require a host with wifi access. Should probably be clarified in their instructions. I verified that this works with the returned localKey on my RT-590!

Here's the script I'm running:

const TuyAPI = require('tuyapi');

const device = new TuyAPI({
  ip: '<redacted>',
  key: '<redacted>',
  version: 3.3});

(async () => {
  await device.find();

  await device.connect();

  let status = await device.get({
    schema: true
  });

  console.log('Current status:');
  console.log(status);

  device.disconnect();
})();

which outputs:

Current status:
{ devId: '<redacted>',
  dps:
   { '1': false,
     '102': 250,
     '103': 0,
     '104': 65,
     '105': 0,
     '106': 0,
     '107': 0,
     '109': false,
     '110': false,
     '111': false } }

For the next step I will block all outbound internet access to the grill and make sure this works 100% locally. Stoked so far that this is working!

SDNick484 commented 4 years ago

@impala454, that's awesome. It looks like it's following the same schema as what I posted in the wiki which i great as that means we already know what everything does. If all you're interested in doing is extracting temperatures to visualize in something like grafana or for records, it should be simple enough to poll and just look for dps == 102 (Target Temp), 103 (Actual), 105 (Probe A), & 106 (Probe B).

Just for your awareness, you can poll with just the id and not the key (meaning you can re-link the grill with your Rectec app which changes the key but keeps the ID) and still get those outputs. You won't be able to turn the grill on or set temperatures or alarms from the CLI without the key (but the app would work), but that might not be a big deal. I suspect you can even use the latest version of pytuya now that you have the ID (just be sure to set the protocol to 3.3).

For reference, you can poll for specific values via the CLI, here's an example: tuya-cli get --ip xxx.xxx.xxx.xxx --id 123456789abcdefghijk --dps 103 NOTE: You don't even need to specify the IP, but responses with IP are much quicker (<1s vs 3s).

It seems with the current tuyapi/cli implementation, you can only get one dps value at a time unless you use the -a flag. Should be easy to either fix in code (if I have time, I may submit a PR to let you specify multiple dps flags although I'm really not a NodeJS coder). As a quick hack, this should get you a CSV of those four values: tuya-cli get --ip xxx.xxx.xxx.xxx --id 123456789abcdefghijk -a | egrep '103|104|105|106' | awk ' {print $NF}' | sed -e :a -e '$!N; s/\n//; ta;s/.$//g' >> temp.csv

SDNick484 commented 4 years ago

@chansearrington, I made a little progress on the HA side last night, but need some help and will be hitting Reddit. What I've done so far is link the grill to the Jinvoo Smart app and connect Jinvoo to HA. Interestingly, within the Jinvoo app I'm able to control the grill (i.e. start it, set temps, etc.). All the characters are in Chinese, but that's not a big deal as it's pretty obvious what everything does (I actually originally used the Jinvoo app in 2018 to extract the schema). However the app doesn't appear to be a "smart" device from Jinvoo's perspective meaning I can't create scenes or automations. I set up Tuya in my configuration.xml and can see my other devices (some switches), but the grill doesn't show up. I tried using the localtuya-homeassistant project I linked to yesterday, but it's not working thus far (or at the very least the grill isn't showing up).

Basically at this point I need help trying to figure out how to get HA to recognize non-standard devices (i.e. those not in the supported list) in either the default tuya integration or the localtuya-homeassistant one (there seems to be some clones of that project out there that I plan to try.

impala454 commented 4 years ago

I confirmed that the script above still does work with outbound internet blocked so it is indeed 100% local. Yay. Interesting note is that the tuya-cli get command does not work for me though. No matter what arguments I give it just prints the help. Not sure what the deal is.

I am more of a Python guy than JS but my home automation is almost 100% NodeRED, so I'll likely try and build a node-red-contib-rectec or something for that. I don't mind testing if you need someone to test your HA stuff on a newer grill. I can spin up a HA instance pretty quickly as I dockerize everything anyways.

chansearrington commented 4 years ago

@impala454 are you using Home Assistant with the Node Red Add-on?

chansearrington commented 4 years ago

@SDNick484 I'm confused on the role of Jinvoo. Why is it involved in getting to HA? If you can give me a little detail I might be able to provide some pointers or find someone that can help.

I'm hoping the existing home assistant work around tuya climate and the work on custom devices can get us there since the grill is basically a Thermostat with two additional room temperature sensors.

The dev for the custom tuya stuff is user "boblatino" in the home assistant forum.

impala454 commented 4 years ago

@chansearrington I'm not using Home Assistant, just Node-RED.

impala454 commented 4 years ago

@SDNick484 can you tell me which version of TuyAPI your grill is responding that it has? I noticed in testing that unless I specify 3.3 it does not respond for me. If I build generic RecTec node I want to make sure it selects the proper version for the grill.

rdelcorro commented 4 years ago

HI, I am @boblatino from the home assistant forum and adding my 2 cents:

I have looked at the grill board and it seems like the hardware is 100% accessible, so if you flash it wrong, you can upload the old firmware fairly easy by using a usb to UART converter ($1) and a soldering iron. https://www.youtube.com/watch?time_continue=44&v=TGtOGIF7HjI&feature=emb_title

SDNick484 commented 4 years ago

@impala454 I'd love to see what you eventually come up with. I don't have any Node Red experience, but it definitely sounds like a cool use case. My RT-680 is still using the older 3.1 protocol; I suspect that also applies to @chansearrington who has RT-700 however I'm not clear if they changed chips for newer RT-700s. You'd definitely want the ability to specify the version if you make a generic module.

@chansearrington So with the generic Tuya HA implementation, it seems to work with only three platforms (Tuya Smart which is the default, Smart Life, or Jinvoo Smart). There's actually tons more platforms besides those three (Rectec, Feit Electric, Globe Suite, etc.) which are probably just re-brands of the same cloud, but I'm not clear if they'd work with HA out of the box (I certainly don't know what to specify as the "platform").

What I do know is that I can pair my Rectec with Jinvoo Smart and it shows up in the app with full features (albeit in Chinese). I also have a couple other power switches under Jinvoo control so I linked that with HA to verify I'm connecting to Jinvoo (I am) and to see if Rectec would show up (it didn't). I'm now trying some alternative paths like the custom_component localtuya. I am pretty confident we can make it work, just a matter of time/experimentation.

impala454 commented 4 years ago

I got all fired up creating my own node-red-contrib-rectec then while searching some errors found this node-red-contrib-tuya-local which works with my RT-590 perfectly. I set up the request injection node to query every 30 seconds or so, but then noticed the node would automatically update every time the temperatures changed. Should work great!

SDNick484 commented 4 years ago

@impala454 Awesome, just so I'm clear, are you linked via Tuyapi or do you still have access through the Rectec app (and if so, how'd you grab the key)?

@rdelcorro Thank you for the feedback. I'm completely new to HA, and I'm not clear on how to use scripts, etc. from it (although I figured it was possible). Good to know about the devices stop working after not being able phone home when on stock firmware. I may give flashing a go, but right now it's a lower priority as I have other mitigating controls (segmented network).

impala454 commented 4 years ago

I'm only using the tuyapi, I don't really have a use case for the RecTec app now that I have this all working. I have my cook data in a database now and pretty graphs and alarms with Grafana.

chansearrington commented 4 years ago

@impala454 what's your frontend for the TuyaAPI?

You should really check out Home Assistant. It runs and integrates with NodeRed, Grafana and a ton of other services.

image

impala454 commented 4 years ago

I have dabbled into Home Assistant a couple of times, and realized Node-RED does everything I need for the automation and hooking up different brands of things together. I like using Google Home as the front end. This node-red-contrib-nora node lets me add arbitrary things to my Google Home setup. For the grill I'll probably just activate the grill directly from Node-RED, but may try adding it to Google Home as a thermostat or something. Honestly remote control of the grill isn't all that important to me so much as getting at the data.

impala454 commented 4 years ago

@rdelcorro I would love to get away from the stock firmware, but this is much more complex than a light switch or thermostat. It's got a PID based motor controller feeding wood pellets into a fire chamber, multiple temperature probes, etc. Not to mention these grills are upwards of $1,000 or more, I don't think many of us would risk bricking the grill completely (or worse, burning down a house).

rdelcorro commented 4 years ago

@impala454 I do understand the cost and risk involved on a bad firmware but at least visually I can see on your board that it has the PID, probes, etc in a different board and a simple ESP attached to it (see the YouTube video I pasted above). I believe that the tuya software is just doing simple gpio / reading analog or digital values via gpio, so it should't be different than a light. Worst case scenario they sell the boards but its not cheap. If you see the manufacturer page, they sell "plug and play" "drag and drop" solutions. I doubt they have a "grill" mode

impala454 commented 4 years ago

Right. I'm sure it's not that complicated, but I'm not going to be the guinea pig 😁

chansearrington commented 4 years ago

@rdelcorro I actually have an extra brand new board and I'm willing to be the guinea pig here :-) But wouldn't this be most similar to a Tuya thermostat?

Set Temp Actual Temp Room temp 1 (Probe 1) Room temp 2 (Probe 2)

What steps would I need to follow to try? I need the dummy proof version of instructions :-)

impala454 commented 4 years ago

For completion's sake here's a write up I created to document my "final" setup.
https://gitlab.com/cclaunch/rectec-node-red

SDNick484 commented 4 years ago

Very cool, appreciate you posting the steps as I was thinking of putting together something similar.

SDNick484 commented 4 years ago

Hey folks, sorry for the lack of updates, I hope everyone is well. I recently picked up a new laptop and am working from home full time so should have more time to focus on this.

Today I made a very cool discovery - you can get your device key from the Android Rectec app. This is great because it means you no longer need to choose between using the RT app or using tuyapi (via tuya-cli link and using your cloud credentials). The downside is that you'll need a rooted device to reach the file.

SDNick484 commented 4 years ago

Steps to extract the key are here: https://github.com/SDNick484/rectec_status/wiki/Device-ID-&-Key-Extraction

impala454 commented 4 years ago

Very nice find! Also a follow up to @rdelcorro 's comment about the device not responding after two weeks of being outbound blocked. I have had mine working for over three weeks now and have not seen this happen.

SDNick484 commented 4 years ago

@chansearrington Good news, I got an initial integration working: image

I'm leveraging localtuya-homeassistant; my modified switch.py is here. My configuration.yaml is below; you do seem to need the correct localkey (which is what I extracted from the Rectec app). They key you get if you go the Tuya cloud API authorization route should also work although that would break the native Rectec app.

switch:
  - platform: localtuya
    host: <my_IP>
    local_key: <my_key>
    device_id: <my_gw_ID>
    name: rectec

I have verified HA detects state changes and now need to figure out how to get the info being reported into something like InfluxDB as well as figure out how to set states (i.e. turn on/off, set alarms, etc.) via localtuya and add GUI switches to do so.

SDNick484 commented 4 years ago

More progress: image

The relevant configuration for polling the temperatures is:

sensor:
 - platform: template
   sensors:
     rectec_target:
       value_template: >-
         {{ states.switch.rectec.attributes.target }}
       unit_of_measurement: 'F' 
     rectec_current:
       value_template: >-     
         {{ states.switch.rectec.attributes.current }}
       unit_of_measurement: 'F'      
     rectec_probea:
       value_template: >-
         {{ states.switch.rectec.attributes.probea }}
       unit_of_measurement: 'F'
     rectec_probeb:
       value_template: >-
         {{ states.switch.rectec.attributes.probeb }}
       unit_of_measurement: 'F'
SDNick484 commented 4 years ago

I'll be smoking spare ribs today this afternoon and will try to do it solely from HA. I confirmed the switch allows for turning the smoker on/off, and need to figure out the proper way to set the target temperature. Current temp is showing 0 because the grill is off but shows actual once it's turned on; the probes always show temp even when grill is off. This is typical behavior (I see it with the native app and in tuya-cli/pytuya), I suspect it's because the current temp uses the PID thermometer.

Besides setting temp, I'd also like to figure out ways to set alarms in HA and to getting that temp data into something like InfluxDB so that I can retain records of the cooks as well as combine in a single visualization with something like Graphana. I know there's community add-ons for both and will start taking a look there.

Longer term I'd still like to move to the MQTT model. Last night I discovered @TradeFace has a couple relevant repos: Tuya which is a re-write of pytuya (although limited to 3.3 devices at the moment so I can't test my smoker on it) and tuyamqtt which relies on that re-write.

SDNick484 commented 4 years ago

Influx DB is configured: image

I'm using the following query: SELECT mean("current") AS "mean_current", mean("probea") AS "mean_probea", mean("probeb") AS "mean_probeb", mean("target") AS "mean_target" FROM "homeassistant"."autogen"."state" WHERE time > :dashboardTime: AND time < :upperDashboardTime: AND "entity_id"='rectec' GROUP BY time(:interval:) FILL(previous)

SDNick484 commented 4 years ago

Here's a shot of Sunday's cook (spare ribs). I did no Texas crutch, around the 3h mark I pulled the probes, sauced the ribs and bumped the temp target to 325 (hence that jump at the end). image

I may break this out as a separate Github project from the main app. I'm hoping there's a better way to use the main Tuya integration in HA and as mentioned before, want to move to MQTT at some point.

chansearrington commented 4 years ago

@SDNick484 This looks amazing!!! I'm going to take a stab at getting this working today.

chansearrington commented 4 years ago

@SDNick484 a couple questions....

  1. Do I need to extract the device & key on an android device? OR use the cloud? I'm confused by the conversation thread above. I'd prefer that it NOT break my RecTec Apps if possible.

  2. Is the ry.py code supposed to replace the code in switch.py or am I supposed to have both files in custom_components/localtuya

SDNick484 commented 4 years ago

@chansearrington If you want to retain a working Rectec app, then you'll need to extract your device key from the Android app. The key is unique and is generated when you pair your grill with one of the clouds; it doesn't seem to be sent in the clear so I'm not aware of other ways to fetch it yet. The alternative method involves registering with a different cloud using the Tuyapi instructions and pairing your grill to that which causes a new key to be generated and breaks your Rectec app.

Once you have the key, install localtuya on your Hassio install in config/custom_components. Replace switch.py that comes with custom_components with rt.py from my project page (I'm not sure if it's important, but I would keep the switch.py file name).

Your configuration.yaml should then be updated as follows:

switch:
  - platform: localtuya
    host: <your_grills_IP>
    local_key: <your_grills_key>
    device_id: <your_grills_devID>
    name: rectec

sensor:
 - platform: template
   sensors:
     rectec_target:
       value_template: >-
         {{ states.switch.rectec.attributes.target }}
       unit_of_measurement: 'F' 
     rectec_current:
       value_template: >-     
         {{ states.switch.rectec.attributes.current }}
       unit_of_measurement: 'F'      
     rectec_probea:
       value_template: >-
         {{ states.switch.rectec.attributes.probea }}
       unit_of_measurement: 'F'
     rectec_probeb:
       value_template: >-
         {{ states.switch.rectec.attributes.probeb }}
       unit_of_measurement: 'F'