Additional device support

[ptd006]

Hi all,

thanks for your work so far on this.

I want to use it with a Broadlink based heating controller ("Hysen HY02B05H", photo here: http://peter.windridge.org.uk/playing-with-cheap-iot-devices/20171120_163450.jpg ).

I can .discover() and .auth() the device. The devtype is: 0x4ead.

(1) Has anyone started working on this device? (2) If not, I need to capture packets from the Android app (which calls a libBLNetwork.so binary). I tried wireshark on Ubuntu but setting promiscuous mode on my laptop doesn't seem to work. I had considered install the Android SDK emulator to run the app on my PC but this seems like a lot of faff. Is there any easier way? (Sorry this question is not directly related to python-broadlink so I don't mind being told to take it elsewhere :))

My goals are:

• read room temperature
• put control in manual mode
• change temperature

I have not tried sending the device codes from check_temperature() etc for fear of bricking it...

Peter

[ptd006]

For anyone interested I finally spent some time with this:

• I rooted an old phone (HTC Wildfire).
• The phone only had Android 2.x, which wouldn't run the heating controller app. So I installed Cyanogenmod (Android 4.0.4 base). If you are using a newer phone then this shouldn't be necessary, of course. Reference for rooting and installing Cyanogenmod: http://bernaerts.dyndns.org/android/275-android-wildfire-unlock-bootloader-flash-cyanogenmod (however I used a more recent ROM from xda-developers).
• I installed tcpdump from https://www.androidtcpdump.com/ .

  adb root
  adb remount
  adb push ~/Downloads/tcpdump  /system/xbin/tcpdump
  adb shell chmod 6755 /system/xbin/tcpdump

• I did some stuff with the controller while capturing network traffic to the sdcard: adb shell tcpdump -i any -p -s 0 -w /sdcard/capture.pcap
• Retrieve the dump back to my laptop: adb pull /sdcard/capture.pcap .
• Open with wireshark: wireshark capture.pcap

This allowed me to see the network traffic from the app. It all looks good. So far I double checked the discovery packets and will start looking at the command packets soon.

I also installed QPython on the phone and will use it for a basic web interface to my various IoT toys.

[ptd006]

I got a little further and can now read the current temperature.

I basically copy-pasted the whole payload for refreshing the temperature from the app so there's still more work to decode the commands properly.

The code is in my github fork.

I suggest splitting the device classes into separate files as the main module is becoming bulky.

Also, why not use the correct device type ID? (For now I hard coded my 0x4EAD at 0x24-0x25 in packet as initially I struggled to get useful responses, not sure if it is required)

[ptd006]

I can now set the temperature manually. It took a while because there were two mysterious bytes which I just realised are simply the (modbus) CRC16 of control code and temperature.

So the basic functionality is done:

• get_temp() : get current room temperature
• set_temp() : switch to manual control and set thermostat temperature
• switch_to_auto() : switch to automatic (predefined) schedule/temperatures

[fermentfan]

Hey Peter, thanks for your work on the thermostat. I am in the same situation as you and would like to control it with my home automation software. Do you mind sharing your script which uses your fork of the python broadlink API? I am not experienced in Python and this would save me a ton of time I think.

Thanks!

P.S: Any reason you closed this issue?

[ptd006]

Hi @DennisVonDerBey!

which home automation software are you using? I think @ralphm2004 has written an MQTT link for openhab (see https://github.com/ralphm2004/broadlink-thermostat ) using my fork.

I have also attached an early web interface I wrote. hysen.zip It is a CGI script you can run with the python CGIHTTPServer module. I don't want to share my live version until I have improved the security features. Anyway, the simple demo uses most of the current functionality of my python-broadlink fork- it has a slider control to set the temperature manually and a button to switch to auto/scheduled mode. It also prints the device info. The blob of CSS in the header is to make it better on my phone (screenshot 2nd picture at http://peter.windridge.org.uk/playing-with-cheap-iot-devices/ ).

I closed the issue because I finished adding all the features I wanted (get/set the temp, switch to auto mode) and I think my code is ready to be merged with mjg59's master. That said, recently I have been working on setting the timer schedule but have not tested it properly.

Cheers, hope you find the code useful! Peter

[ptd006]

Hi @DennisVonDerBey,

I guess if you are really not experienced in Python then the CGI example I attached isn't that helpful. Here is some simpler code. Note it takes the first device .discover()'d so may not work if you have multiple Broadlink controllers!

import broadlink
dev = broadlink.discover()
dev.auth()
print('Device: ' + dev.type)
print('Current temperature: '+ str(dev.get_temp()) + 'C')
print('Set temperature: ' + str(dev.set_temp(22.0)) )
#print('Test switch to auto mode: ' + str(dev.switch_to_auto()) )

[fermentfan]

Wow you guys really made my day. Didn't think anyone really worked on this device already and now I find this and an out of the box working connection to my Mosquitto. I love you! <3

Gonna look at the code now. I see the MQTT fork is also capable to control the thermostat, but I find it confusing what it listens to. It looks like it waits for a message from MQTT. Do you happen to know the topic and the message format it takes?

//edit:

Nvm I found it. The structure is like that: /[topic]/[mac]/cmd/[command] Thanks again!

[ptd006]

I haven't used it but skimmed the code and it looks like default topic is 'broadlink' and the format is like /broadlink/[mac address]/cmd/[whatever]

where [mac address] is your thermostat's mac and [whatever] gets passed as cmd in:

                            if cmd=='set_temp' and float(opts)>0:
                                self.device.set_temp(float(opts))
                            elif cmd=='switch_to_auto':
                                self.device.switch_to_auto()
                            elif cmd=='switch_to_manual':
                                self.device.switch_to_manual()

The code also publishes the results from device.get_full_status() as follows:

                            data = self.device.get_full_status()
...
                        for key in data:
                            ...
                                mqttc.publish('%s/%s/%s'%(self.conf.get('mqtt_topic_prefix', '/broadlink'), self.divicemac, key), data[key], qos=self.conf.get('mqtt_qos', 0), retain=self.conf.get('mqtt_retain', False))

So if your mac is deadbeef then room temperature is published to /broadlink/deadbeef/room_temp

You can refer to the code for get_full_status in my fork for what else is returned. I think all the important ones are present and correct but no guarantees!

See also https://github.com/eschava/broadlink-mqtt

Note that setting the temperature requires PyCRC but I haven't added that to the dependencies list in setup.py yet

[Giermann]

Hi Peter, this is really awesome! I just started to use a thermostat I bought in China, found it's name to be 'Broadlink-OEM...', searched for this python library and then came across your issue here after reading the devtype 4EAD! And finally all my work is already done - THANKS. :-)

But still one thing that is not clear until now: You return a field sensor (1 in my case). Does this one reflect the switching between internal and external temp sensor? If so, did you already find the command to actually switch between the sensors? I still hoped to be able to access both sensors via the API, but at least I could switch the sensor, wait some seconds and the read the temp again.

Or might there be further command to explicitly read the internal and external sensor? Is your device equipped with 2 sensors? Otherwise I'll have to follow your guide to capture the packets of my app switching the sensors.

Sven

[ptd006]

Hi Sven! thanks, glad you find it useful :)

My device has the capability to read an external sensor (it came in the box) but I did not connect it. I only guessed the sensor field based on what I saw the app doing. My sensor field is 0. From what I've seen the codes follow the manual so I assume you have the external sensor active?

I believe the sensor setting is the last byte of the mode code, i.e. do

  def set_mode(self, auto_mode, loop_mode,sensor=0):
    mode_byte = ( (loop_mode + 1) << 4) + auto_mode
    # print 'Mode byte: 0x'+ format(mode_byte, '02x')
    input_payload=bytearray([0x01,0x06,0x00,0x02,mode_byte,sensor])
    self.send_request(input_payload)

(I made sensor optional because I currently have functions that call set_mode with only 2 args!)

Of course, my thermostat just gives me an "Err" message (on the screen) when I set sensor_mode = 1, but the sensor field is =1 when I call get_full_status.

If you get chance, please can you check?

Maybe one day I'll connect the external thermostat and see if it's possible to specify which to read.

Cheers, Peter

[ptd006]

The manual mentions "2:internal control temperature, external limit temperature", which is helpfully explained as follows:

When internal control temperature and external limit temperature(high temperature protection),under power on state,press time key not move first,then press on/off key together to switch and view external temperature (this time measure temperature display zone displays OUT TEMP temperature value),press time key to display room temperature;

If you can shed light on what that is please submit a patch :)

[Giermann]

Peter, I'm working on it! ;-) I already found the temperature of the external sensor and the meaning of most of the remaining returned payload. Just to explain to users, you have 3 different operating/sensor modes:

0: use internal sensor for display and switching according the set point 1: use external sensor for both 2: use internal sensor for switching at set point, but additionally check external temperatur to not exceed OSV setting (advanced setting no. 2), using dIF (adv. setting 3) as hysteresis.

If you use sensor_mode 2, you'll see both values in the app and use the provided procedure to switch displayed temperature at the display.

BTW: Do you still have your packet sniffing setup ready? Did you already try to find the commands to set the time (highly appreciated) and the advanced settings including power, remote-lock and anti-freezing?

[Giermann]

Well, I followed your guide to sniff the packets sent by the app. Found that these are also sent to the Broadlink gateway server, if one is in another network. But: How do I encrypt these within Wireshark?

[ptd006]

Hey Sven,

thanks for the explanation.

I pushed a new function set_time(self, hour, minute, second, day), e.g.

import datetime
now=datetime.datetime.now()
dev.set_time(now.hour, now.minute, now.second, now.weekday()+1)

I haven't used any of those extra features and not tried to implement them.

Yeah I also saw traffic to Broadlink and it's one of the reasons I didn't install the app on my main phone (am quite paranoid ;)).

Now about decrypting packets you extract in wireshark- the thermostat uses the same key for everything that auths to it. So, I just copy the payloads into my (interactive) Python session and run dev.decrypt() on them. (Wireshark has a handy feature of letting you copy the packet minus header as a C-style array that you can copy-paste directly). The interesting part begins at 0x38 as described in the python-broadlink protocol page . In more detail, I do:

input_packet = bytearray([
 0x5a, 0xa5, 0xaa, 0x55, 0x5a, 0xa5, 0xaa, 0x55, 
 ... other bunch of stuff copy-pasted from Wireshark omitted ...
 0xf2, 0x13, 0x80, 0x02, 0x63, 0xa1, 0x4b, 0xb1
 ])
input_payload_enc = input_packet[0x38:]
input_payload = dev.decrypt(bytes(input_payload_enc))

# print it
print ' '.join(format(ord(x), '02x') for x in input_payload)
# send to device
response = dev.send_packet(0x6a, bytearray(input_payload))

Oh, and the response can be decrypted with

print (response[0x22] | (response[0x23] << 8) ) # should be 0
# Decrypt payload
payload = dev.decrypt(bytes(response[0x38:]))
print ' '.join(format(ord(x), '02x') for x in payload)

Here, dev refers to the python-broadlink .auth()'d device.

AFAIK all commands use the serial pass through (0x6a) and the first 2 bytes are the length and the last 2 are the modbus crc (see send_request(), hopefully that explains clearly).

[ptd006]

Hi again Sven,

I had another bash and added set_advanced for (nearly) all the settings in the advanced settings of the app. I don't know exactly what each setting is but I confirmed each in the app and on screen where possible (e.g. for the "Anti-freezing" mode).

For anyone reading who has lost their manual, here are the settings. The last value is the default.

1 | SEN | Sensor control option | 0:internal sensor 1:external sensor 2:internal control temperature, external limit temperature | 0:internal sensor 2 | OSV | Limit temperature value of external sensor | 5-99℃ | 42℃ 3 | dIF | Return difference of limit temperature value of external sensor | 1-9℃ | 2℃ 4 | SVH | Set upper limit temperature value | 5-99℃ | 35℃ 5 | SVL | Set lower limit temperature value | 5-99℃ | 5℃ 6 | AdJ | Measure temperature | Measure temperature,check and calibration | 0.1℃ precision Calibration (actual temperature) 7 | FrE | Anti-freezing function | 00:anti-freezing function shut down 01:anti-freezing function open | 00:anti-freezing function shut down 8 | POn | Power on memory | 00:Power on no need memory 01:Power on need memory | 00:Power on no need memory

[Giermann]

Well done, Peter! So you identified all the fields, I also found yesterday. Regarding your "burn first two bytes, not sure what they are": I noticed that the first byte seems to always be the size of the totally used payload data (remember the whole payload is always a multiple of 16). The next 3 bytes seem to always be 0x0 0x1 0x3, followed by the length of the actual data, which is always 5 less than the first byte. But I only checked the two messages get_temp() and get_full_status(). So I'd suggest, if you strip the first bytes, you should strip the first 5 bytes... as you start with payload[3{+2}] already. Further, just to mention, the 16(+5) bytes inside get_temp() are equal to the ones in get_full_status() - so one could retrieve all status except time and schedules with the request in get_temp().

Regarding the remaining unknowns, I'll sumbit a patch to your branch!

[ptd006]

Thanks Sven. Shortly after commenting on the first 2 bytes I realised they were simply the length and the CRC follows. I added the CRC check but forgot to remove the comment! :)

Totally agree with your other comments. If I get chance I'll refactor the get_* codes later.

Cheers, Peter

[Giermann]

Really cool - for anyone who's interested: I expanded the Python script for locating your Broadlink devices and added the ability to paste Wireshark's output of 'copy as hex stream' to the end of the discovery loop:

  length = 1
  while length > 0:
    hex_stream = raw_input('Paste some HexStream to decode or [Enter] to continue: ')
    length = len(hex_stream)
    if length % 2 > 0:
      hex_stream = hex_stream[:-1]
    payload = bytearray.fromhex(hex_stream)
    # TODO: check for start to be '5aa5aa555aa5aa55'
    # we strip off the first 0x38 bytes and need multiples of 16
    if len(payload) >= 0x48:
      payload = payload[0x38:]
      if len(payload) % 16 > 0:
        payload = payload[:-(len(payload) % 16)]
      payload_dec = mydevices[index].decrypt(bytes(payload))
      print ''.join('0x{:02x} '.format(ord(x)) for x in payload_dec)

[ptd006]

Nice, should be useful for working on other devices. Thanks for your commit on the power setting, we have basically full support now.

[fermentfan]

we have basically full support now.

Is there already a function to set the power state of the thermostat?

//edit: Ahh found it in the commit list. (https://github.com/mjg59/python-broadlink/pull/138/commits/73b68ba24df29d6495fd5885ad4dd88869945a43)

Thanks again for all the work!

[fermentfan]

There is a new redesigned version of the thermostat being sold on the internet. It still works with the original app like the old one, but it doesn't seem to work with this project.

Does anybody have one?

[ahmaddxb]

There is a new redesigned version of the thermostat being sold on the internet. It still works with the original app like the old one, but it doesn't seem to work with this project.

Does anybody have one?

I recently got the new version of that thermostat and would like to integrate it into openhab. I have tried some of guides from the openhab forum but was not able to discover the device. Any Suggestions?

[ptd006]

Hi @DennisVonDerBey @ahmaddxb , sorry I don't have the new device.

If the original app works the issue should be minor but identifying it might require delving into the Python code or packet sniffing (which involves more work).

@ahmaddxb - when you say it's not discovered do you mean no device is returned by discover()? If the "generic" broadlink device is returned it probably means only the device ID is changed.

Peter

[ahmaddxb]

Hi @ptd006 I mean there is no devices discovered? Maybe you could give me i guide to follow and I'll try do some packet sniffing and post results.

Here are my results.

[ptd006]

@ahmaddxb -

0) does discover() just time out? 1) Can you definitely ping the device from your computer? (to exclude weird network config problems). 2) By the "original" app do you mean from the previous device? If it's a completely new app it's possible (but unlikely) they're not using a Broadlink chip anymore. Is the device hostname still like BroadLink_OEM-blah-blah? (The Broadlink discovery stuff is fairly generic and I'm not aware of them releasing a new version)

I can't write a packet sniffing tutorial for now. However, there are plenty online.

[ahmaddxb]

0. It will stay like in the picture for a long time never seen it timeout.
1. I can ping the device and it replies.
2. The original app is called Beca Smart on the play store. The hostname of the device is ESP_C887A9.

[ptd006]

@ahmaddxb your answers strongly suggest your device is not Broadlink based but rather ESP (Espressif), probably 8266 running Arduino code (based on the hostname). I actually decided to switch to this chip for my own DIY thermostat project. Anyway if it's not broadlink I'm afraid you're wasting time with python-broadlink and you'll need to get that packet sniffing setup sorted. Depending on the hardware you have available it might be as easy as running https://www.wireshark.org/ . Probably you need another solution though. I ended up routing an old phone and using tcpdump (see my post above). I then imported the saved dumps into wireshark for analysis.

[ptd006]

@ahmaddxb as an alternative to packet sniffing- depending on where you live, it might be legal to decompile the APK. If you're lucky all the communication code is in plain Java rather than being a closed source binary like the Broadlink blob.

[ahmaddxb]

ok. I am currently sniffing and i found that the app is communicating with there mqtt server and have been able to connect to it and send some message to turn on the AC and change fan speed and mode. Im not really sure what i can do from here?

[ahmaddxb]

I have not been able to subscribe to anything to see responses though.

Edit: I have no been able to subscribe to the topic and can see all the replies for my setting changes of the app

[ahmaddxb]

I can decompile the app but i don't know Java, is there anyone that would know what I'm looking for in the decompiled APK?

[melka]

1. It will stay like in the picture for a long time never seen it timeout.
2. I can ping the device and it replies.
3. The original app is called Beca Smart on the play store. The hostname of the device is ESP_C887A9.

I was just gonna order this same model. Happy to hear it might have an ESP8266 / ESP32 inside, reverse engineering the firmware might be difficult (and I don't think I'd like to do that, since it's wired into mains and a boiler, a coding error might make things ugly) but I have hope that someone more adventurous than me will try to update the firmware, maybe using Tasmota

[dpeddi]

About the new device esp based... There is already a project opened?

Perhaps: https://github.com/clach04/python-tuya

[Bajoras]

New API for heating wifi thermostat.txt esp model api

[tonic48]

Came to this thread while searching the info about BHT-002-GBLW floor heating thermostat or something https://www.aliexpress.com/item/WiFi-Thermostat-Temperature-Controller-LCD-Touch-Screen-Backlight-for-Electric-Heating-Works-with-Alexa-Google-Home/32896579061.html?spm=2114.10010108.1000023.5.7c2041aeMm5tSL It looks like mentioned above Beca thermostat however it's not connecting to bestbeca.cn domain as described in API document and it controls by My Smart Thermostat app. Does anyone get lucky to sniff communication from that device? Regards

[TheItschi]

Yes, it works with TuyAPI. You need to extract the local device key (follow instructions at https://github.com/codetheweb/tuyapi/blob/master/docs/SETUP.md), lookup the device ID in the app comming with the thermostat and determin it's ip address in your LAN.

After that you can run tuya-cli with the following parameters:

tuya-cli get --ip 192.168.6.105 --id 65363xxx --key ff87122xxx --dps 2

That gives you the current temperature setting *2 (you need to devide that value by 2).

More Infos can be found here: https://www.domoticz.com/forum/viewtopic.php?f=34&t=25965&p=205818#p205818

[tonic48]

@TheItschi Thanx a lot! It works like a charm now.

[betyar]

HI,

I have been using the script for a while now but since May 31st it suddenly stopped working properly. Discover can still find the device and identify it as a Hysen controller, but it stops after that. There is a Reconnect / Connect, reason: 0 and then waits and doesn't timeout. Could it be that the protocol or encryption has changed somehow?