search

ct-Open-Source / tuya-convert

A collection of scripts to flash Tuya IoT devices to alternative firmwares
MIT License
4.66k stars 500 forks source link

New PSK format #483

Open countcobolt opened 4 years ago

countcobolt commented 4 years ago

Hi all

I am trying to flash a BSD34 smart socket. I have downloaded the latest tuya master built using git. Working on an RPI 3B. I can connect the socket using the normal smart life app. I did this after I absolutely could not manage to flash it at first (still cant). Decided to dive a bit deeper into it today and found this:

In smarthack-psk.log I have a ton of entries as following

new client on port 443 from 10.42.42.20:53000
ID: 0242416f68626d64366147393149465231509241f729c9f0af3aa41e355b7cbeb1ece63da6ff54b74f271af0ef044466e6
PSK: d9488a1b4524ae3e31acd0342e6d0b2eedbb5d55e957a9a51073b95e36ab1c5c
('could not establish sslpsk socket:', SSLError(1, u'[SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:727)'))
new client on port 443 from 10.42.42.20:5371
ID: 0242416f68626d64366147393149465231509241f729c9f0af3aa41e355b7cbeb1ece63da6ff54b74f271af0ef044466e6
PSK: d9488a1b4524ae3e31acd0342e6d0b2eedbb5d55e957a9a51073b95e36ab1c5c
('could not establish sslpsk socket:', SSLError(1, u'[SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:727)'))

My smartphone and device are actually connecting to the acces point. In smarthack-wifi.log you see

wlan0: AP-STA-DISCONNECTED ac:57:75:99:f2:9f
wlan0: AP-STA-CONNECTED ac:57:75:99:f2:9f
wlan0: AP-STA-CONNECTED c4:4f:33:bc:10:c6
wlan0: AP-STA-DISCONNECTED ac:57:75:99:f2:9f
wlan0: AP-STA-CONNECTED ac:57:75:99:f2:9f
wlan0: AP-STA-DISCONNECTED ac:57:75:99:f2:9f
wlan0: AP-STA-CONNECTED ac:57:75:99:f2:9f

The AC mac is my smartphone, while the C4 is the actual device I am trying to flash.

The mqtt log remains quite virgin

1578041479: mosquitto version 1.5.7 starting
1578041479: Using default config.
1578041479: Opening ipv4 listen socket on port 1883.
1578041479: Opening ipv6 listen socket on port 1883.

as does the UDP log

Listening for Tuya broadcast on UDP 6666
Listening for encrypted Tuya broadcast on UDP 6667

Smarthack-web.log does not give me more either Listening on 10.42.42.1:80

I have browsed and google for hours now and mostly I find that this might be due to not using the ESP8x chip. Yet funnily enough, when I simply configure them using the smart life app, I see them in my network as ESP_some_ref_number in it. And it does get an IP address.

The socket looks like this

Any clues on why the SSL error shows up? Using openssl 1.1.1d / Python 2.7.16 and 3.7.3

Kind regards Steve

rsbob commented 4 years ago

Runs on fw 1.1.4 in smarlife app IMG_20191225_200509 IMG_20191225_200542 Have the same issue.ssl 1.1.1d and python 2 and 3 .have to flash with soldering. Esp chip is mounted. For to open the smart plug you have to eat first Southtirolean Speckknödl😂 IMG_20191225_200044

countcobolt commented 4 years ago

Runs on fw 1.1.4 in smarlife app IMG_20191225_200509 IMG_20191225_200542 Have the same issue.ssl 1.1.1d and python 2 and 3 .have to flash with soldering. Esp chip is mounted. For to open the smart plug you have to eat first Southtirolean Speckknödl IMG_20191225_200044

Just wondering did you just pull really hard on it or what is the trick to open it? Turn? Worried to break them

rsbob commented 4 years ago

moved a screwdriver into one hole and then leveraged.nothing breaks

countcobolt commented 4 years ago

A few more stupid questions : 1: did you solder on the bottom or on the dots? 2: did you connect the gnd to reset? 3: which tasmota did you put on it?

Thanks a lot in advance :)

rsbob commented 4 years ago

2 screws in total.hav no usb serial programmer here.its on the way ... only ioo to ground .the rest on grenn pcb .from right to left.....3.3 gnd......

kueblc commented 4 years ago

Hi @countcobolt and @rsbob, it looks like you may have discovered a new version of the Tuya firmware. The PSK identity in your psk log does not have the expected prefix, which explains why the calculation is failing.

If you are flashing by wire, could you please make a backup of the stock firmware? We can use this to further debug and possibly develop a new workaround.

countcobolt commented 4 years ago

@kueblc Ok I have it open and will try tomorrow morning. A few stupid questions (I am on Linux or Mac) do you have some tutorials on how to download the existing firmware?

Kind regards

rsbob commented 4 years ago

https://github.com/espressif/esptool https://github.com/espressif/esptool/blob/master/README.md

kueblc commented 4 years ago

Once you have it installed you can use something like:

esptool.py read_flash 0 0x100000 firmware-backup.bin
rsbob commented 4 years ago

flashed now ... esptool.py --port /dev/ttyS0 flash_id esptool.py --port /dev/ttyS0 read_flash 0x00000 0x100000 image1M.bin esptool.py --port /dev/ttyS0 erase_flash esptool.py --port /dev/ttyS0 write_flash -fs 1MB -fm dout 0x0 tasmota-DE.bin

IMG_20200104_084608

IMG_20200104_084613

IMG_20200104_090057 @kueblc

rsbob commented 4 years ago

Here the orig.FW BSD34--image1M.zip

countcobolt commented 4 years ago

@rsbob which. pin did you connect the GPIO0 on? I have CW, R and then 00 in one line (or similar (can't really read it properly on them)) Thanks

Steve

rsbob commented 4 years ago

gpi0 (esp pcb side soldered to ground ,then give voltage and run commans )Green pcb . 3,3 volt Gnd rx tx on the main platine side . yellow cable to the the blue ones.see foto. i think its printed IO

countcobolt commented 4 years ago

@rsbob hey, I know I need to connect GPIO0 to the ground, but that is the problem: which one is GPIO0 on the PCB? The printing on my PCB is really bad, so cannot read it (literally :) )

rsbob commented 4 years ago

😂You se IO on blue board near 3,3 volt and grount lines?but if you can wait wait a hour i add a foto.

countcobolt commented 4 years ago

15781358451174391883751543533112 Wondering which of these is gpio0 :)

rsbob commented 4 years ago

the left from Rx .the one without trace market OO

countcobolt commented 4 years ago

@rsbob thanks;, got it :) now got it flashed and working :) @kueblc let me know if you need testing, I have 2 more here I can test with tuya-convert

countcobolt commented 4 years ago

just one thing: using the template on https://github.com/blakadder/templates/blob/master/_templates/BSD34 the led is not working, toggle etc is working as expected

secondly , wondering how to enable the power metering ==> Sorry noticed it doesn't have that feature... Stupid me

Farfar commented 4 years ago

I get the same sslpsk errors in smarthack-psk.log though slightliy different ID: ID: 0242416f68626d643661473931494652315ee70ad6ee67338a4f486a0750cfc89d52ee6a3be1727dcaa5eef3054e2165f5 PSK: 2b25c3013c2186c23697c1cc752161c039d1672c36b3d7e6df51c4f43b87586a ('could not establish sslpsk socket:', SSLError(1, u'[SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:727)'))

The device is a Deltaco SH-P01E socket with power monitoring.

smarthack-wifi.log with my Huawei phone and smart plug connected: ... wlx90f652e46df6: AP-ENABLED wlx90f652e46df6: AP-STA-CONNECTED 30:45:96:1d:86:56 wlx90f652e46df6: AP-STA-CONNECTED d8:f1:5b:8b:34:59

Would be nice to be able to use tuya-convert since I don't know how to take the plug apart without damaging it.

rsbob commented 4 years ago

any backup of old smartlife fw for me😂bsd34

Farfar commented 4 years ago

If anyone is interested, here's the firmware from my Deltaco SH-P01E plug. Deltaco_SH-P01E_20200110_image1M.zip The firmware is the latest up until Jan 10th 2020.

kueblc commented 4 years ago

Thank you @rsbob and @Farfar for collecting this data. Apologies for the late response as we have had a loss in the family. I will try to work on the issue this week, please feel free to ping me for a follow up.

kueblc commented 4 years ago

Took some time on this today. Unfortunately these latest firmware builds seem to have changed a number of things making this a non-trivial fix.

Thus the cat and mouse game with Tuya continues.

kueblc commented 4 years ago

If anyone is able to capture network traffic from one of these devices pairing with the cloud, along with the corresponding firmware backup, this would be instrumental in reverse engineering efforts

Tollbringer commented 4 years ago

If anyone finds a US variant of this I would be happy to get a couple sacrificial units and give it a go.

AIexBV commented 4 years ago

@kueblc If you are interested to analyse the traffic, you could probably use the bulb sold in a big online store as "bakibo smart wlan led". I have the same issue with it. It shows could not establish sslpsk socket: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:720).

eku commented 4 years ago

@AlexBV mine is a Bakibo TB95 with the same problem.

Tollbringer commented 4 years ago

@AIexBV It would be great for anyone wishing to contribute, who has the ability to disassemble/backup the original FW before doing anything and then another backup after trying TC.

Logs are good too!! We're going to need to sacrifice (anyone in the community who is willing/able) to sacrifice some modules to play this game of Cat & Mouse

Tollbringer

AIexBV commented 4 years ago

@Tollbringer I already tried to open the bulb. But I do not have access to good tools, so unfortunately I damaged the chips. But maybe the log entries will help. I got many lines like this:

new client on port 443 from 10.42.42.11:63510 ID: 0242416f68626d643661473931494652318f1af4a8c59aa2df416796cf6c3509959021a9fc70c1265ab969a0c21765060a PSK: cae5783da388990f138996dc22b9b50e3c155cc50fdd80401e8e27db0a2d58c5 could not establish sslpsk socket: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:720)

eku commented 4 years ago

@kueblc please find attached a tcpdump of the communication: bakibo_bulp_pcap.zip

aviogit commented 4 years ago

Hey guys, I think I'm in an even worse situation, when trying tuya-convert from within docker I get a wall of these:

new client on port 443 from 10.42.42.31:37666 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37670 could not establish sslpsk socket: [SSL: WRONG_SSL_VERSION] wrong ssl version (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37674 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37676 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37680 could not establish sslpsk socket: [SSL: WRONG_SSL_VERSION] wrong ssl version (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37686 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37694 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37730 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37738 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37748 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37756 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37760 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37772 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37784 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37796 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37806 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37810 could not establish sslpsk socket: [SSL: NO_SHARED_CIPHER] no shared cipher (_ssl.c:852) don't panic this is probably just your phone! new client on port 443 from 10.42.42.31:37820 could not establish sslpsk socket: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:852) don't panic this is probably just your phone!

The device is a Nooie Led Smart Bulb, the FW is at version 1.0.5, so it should be old enough, but it talks protocol version 3.3 (I think) and I had some trouble fetching its key.

nooie-fw-1 0 5

I say it talks a 3.3 version protocol because looking at the traffic with it's official app, I see a lot of 3.3 in each packet (see attach) and the already known 0000 0000 55aa sequence (maybe I'm wrong, it's just a little bit more of 24h that I'm studying this topic :)

tcpdump-host-and-port-X.txt

If you need some reverse engineering help, let me know, I'm ready to mostly everything except cracking open the device.

kueblc commented 4 years ago

Thank you for the data @eku I'll take a look

@aviogit

don't panic this is probably just your phone!

aviogit commented 4 years ago

I've tried both with my phone and my tablet, given the problems I had with the bulb I bet it's the bulb.

kueblc commented 4 years ago

@aviogit I'm not saying your bulb does not have problems, but that specific error you are reading is unrelated. You can check which device is assigned that IP address.

aviogit commented 4 years ago

Mmm, running everything outside of docker, I get this:

new client on port 443 from 10.42.42.10:36974 argument 2 must be str, not bytes new client on port 443 from 10.42.42.10:36975 argument 2 must be str, not bytes

And you're right, 10.42.42.10 is my tablet:

> arp -n

Address HWtype HWaddress Flags Mask Iface 10.42.42.10 ether 10:7b:44:bd:f2:b7 C wlan1

I confess I'm not understanding the pairing procedure and why another device is needed, so if you have any advice on how to solve this, I'm all ears.

kueblc commented 4 years ago

argument 2 must be str, not bytes

You must run install_prereq.sh again after updating tuya-convert

aviogit commented 4 years ago

I don't know, the device completes the pairing procedure, but the message is always the same:

Device did not appear with the intermediate firmware Check the *.log files in the scripts folder

I've re-run install_prereq.sh, now I get again the NO_SHARED_CIPHER messages, as it was inside docker. I'll try again.

Using interface wlan1 with hwaddr 00:02:6f:8d:e7:e3 and ssid "vtrust-flash" wlan1: interface state UNINITIALIZED->ENABLED wlan1: AP-ENABLED wlan1: AP-STA-CONNECTED 68:57:2d:6d:f1:d7 wlan1: AP-STA-CONNECTED 10:7b:44:bd:f2:b7 wlan1: AP-STA-CONNECTED 68:57:2d:6d:f1:d7

> arp -n

Address HWtype HWaddress Flags Mask Iface 10.42.42.27 ether 68:57:2d:6d:f1:d7 C wlan1 10.42.42.10 ether 10:7b:44:bd:f2:b7 C wlan1

kueblc commented 4 years ago

I don't know, the device completes the pairing procedure, but the message is always the same:

Device did not appear with the intermediate firmware Check the *.log files in the scripts folder

Yes, the message is always the same on failure. You'll need to check your log files. Since your issue is not related to this one, if you would like further assistance, please open your own issue with your log files attached and I would be happy to continue the discussion with you there.

pixomanie commented 4 years ago

@AlexBV mine is a Bakibo TB95 with the same problem.

Bakibo TP22Y the same problem Version: WI-FI Modul: 1.1.4 MCU-Modul 1.1.4

bakiboT22Y.pcap.zip

eku commented 4 years ago

@kueblc Colin, is there anything else I can contribute to support for the new PSK format beyond a network recording before returning the smart bulbs? Or should I better keep them because it is foreseeable that they can be converted soon?

andreashergert1984 commented 4 years ago

unfortunatly i discoverd another "thing" with the same error: it´s a tuya smart shutter . grafik

iracigt commented 4 years ago

After spending some time digging through the firmware, it's pretty clear Tuya has upped their game here. It appears a unique preshared key is programmed into each device at the factory. Unlike before where the key could be derived from the TLS identity, now the identity is a hash of the device's id and MAC address. For those following along, the new identity derivation is:

'\x02' + 'BAohbmd6aG91IFR1' + sha256(prod_idx + mac_addr)

where prod_idx is the ASCII of the 8 digit product ID, and mac_addr is the MAC address of the device in hex, all lowercase, no punctuation. These parameters are all stored in the flash in a JSON blob at 0xfb000.

kueblc commented 4 years ago

Great findings @iracigt, thank you for sharing.

@eku I really appreciate the data you have collected, unfortunately it is not possible to say how long it will take to find a workaround, but I am optimistic someone in the community will.

connoleg commented 4 years ago

@kueblc @iracigt thanks for your help with this. Much appreciated.

brott8 commented 4 years ago

Appreciate your work @kueblc @iracigt . I just received a tuya wifi smart led bulb and connected to Smart Life. Then tried to flash the next day and received similar errors to those reported here. Let me know if there's anything I can do to help with my device.

leifclaesson commented 4 years ago

Here's the firmware dump from a Moeshouse Smart Downlight with the same issue. New right out of the box, never paired with the tuya app.

moeshouse_downlight_1m.zip

Tallboy22 commented 4 years ago

Think I'm seeing the same issue with a BSD29 - getting the following error in the smarthack-psk log:

('could not establish sslpsk socket:', SSLError(1, u'[SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:727)'))

Anyone worked through a solution yet?

Woodpeckercz commented 4 years ago

Here's the original firmware for BSD29 smart plug. Hope it helps.

BSD29_firmware_1M.zip

Tallboy22 commented 4 years ago

How did you manage to open the devices up to flash them? They look pretty impenetrable!