search

jasonacox / tinytuya

Python API for Tuya WiFi smart devices using a direct local area network (LAN) connection or the cloud (TuyaCloud API).
MIT License
937 stars 167 forks source link

Tuya Protocol 3.5 - Cannot get (decode) response from Temperature & Humidity sensor on Zigbee #248

Closed romicaiarca closed 1 year ago

romicaiarca commented 1 year ago

Cannot get (decode) response from Temperature & Humidity sensor on Zigbee

Hi, I have 3 temperature humidity sensor that works on Zigbee (https://www.aliexpress.com/item/1005004854419532.html?spm=a2g0o.order_list.order_list_main.5.2c4d1802OiqBRH) and I cannot get data from them. They are connected to a SilverCrest gateway (something like this https://paulbanks.org/projects/lidl-zigbee/).

We already discussed here about it: https://github.com/jasonacox/tinytuya/issues/69

https://github.com/jasonacox/tinytuya/issues/69#issuecomment-1373870806 - execution and response DEBUG trace code.

https://github.com/jasonacox/tinytuya/issues/69#issuecomment-1374113821 - execution and response DEBUG trace code: a Gateway parent and sensor as children

I have opened this topic in order to help you to find a solution for this. For the moment here you have few pictures of the hardware:

Temperature & Humidity sensor: https://user-images.githubusercontent.com/1266110/211324718-bea4aaba-5917-4645-8c09-9369ec95fa89.jpg | Temperature Humidity Sensor_01 https://user-images.githubusercontent.com/1266110/211325022-04bf819e-8647-481d-bff1-e08f890f4696.jpg | Temperature Humidity Sensor_02 Note: The other side of the board we have only the notations for the pins TTL connection.

SilverCrest gateway:

https://user-images.githubusercontent.com/1266110/211325291-4b67c2a7-80bc-40c9-ad57-765cdc9e766c.jpg | SylverCrest Zigbee Gateway_01 https://user-images.githubusercontent.com/1266110/211330706-fce6cbaa-5d49-40b6-9dc0-f890de469af8.jpg | SylverCrest Zigbee Gateway_02 https://user-images.githubusercontent.com/1266110/211330712-84aac2b8-eba0-4a87-9973-6e9968d302a4.jpg | SylverCrest Zigbee Gateway_03 https://user-images.githubusercontent.com/1266110/211330716-a0e5d092-f01f-4748-82ad-ec1788435012.jpg | SylverCrest Zigbee Gateway_04 https://user-images.githubusercontent.com/1266110/211330719-1f12f8ef-fe89-44d1-84a3-d80c2fcc6f8a.jpg | SylverCrest Zigbee Gateway_05

Now the support that I need is regarding TTL to USB dongle that I need in order to send you dumps for them. Are these plug and play for this purpose (connect the dongle to the boards and direct connect to the USB port of my computer: https://www.aliexpress.com/item/4000120687489.html .

If they are what I need, until they arrive, is there any other way that I can help make these sensors/gateway to be added in support for tinytuya?

romicaiarca commented 1 year ago

@jasonacox @uzlonewolf , just a memento comment! Can you please give me some advice about TTL to USB dongle version that I need to get dump files for the Zigbee Gateway and the sensor?

jasonacox commented 1 year ago

I've never done this, though I've programmed many attiny and esp chips. I'll let @uzlonewolf comment how to get a firmware dump and if it would help. He is the guru for that. 🙇

As mentioned, we are starting to see new devices that seem to be using a new protocol or key that do not work with TinyTuya. It may be good to see the output as mentioned in #247 :

import socket

clients = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
clients.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
clients.bind(("", 6667))
clients.settimeout(7)

for i in range( 8 ):
    data, addr = clients.recvfrom(4048)
    print( addr, '=', data )
romicaiarca commented 1 year ago

I've never done this, though I've programmed many attiny and esp chips. I'll let @uzlonewolf comment how to get a firmware dump and if it would help. He is the guru for that. 🙇

As mentioned, we are starting to see new devices that seem to be using a new protocol or key that do not work with TinyTuya. It may be good to see the output as mentioned in #247 :

import socket

clients = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
clients.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
clients.bind(("", 6667))
clients.settimeout(7)

for i in range( 8 ):
    data, addr = clients.recvfrom(4048)
    print( addr, '=', data )

@jasonacox , here you have the output for socket scan:

python -m sensor_local
('192.168.100.7', 48923) = b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb92e0c0c3b1b87\xb6\x06\xaa\xafX\x83\xdd\xc36~S\xc2\xb7^\x8cr\x9a{\x0eo\x84a`\xf6\xdd\x99\xeeU\x88x\xbf\xabNu\xad/1\xf9\x9dP\xf1*S\x94\r\xe0\x8c\xf5p\x84\xfeq\xec\x88}g\xd8\x9f8`\xfcm\xc4\xa9b\xb5\xd6\x94\x92\xe70\x7f\xa8\x90\xd7\xda\xb1\xf2>\xe2\x02\xfe.\xcf\xc9\xb6\x9b#\x00\x91\x9b#X\x11T\x99\xdd\xa4JU\x9b\xdcE\t\xc0\xed\xa8T\xa0\xcb\xcez\xcb]*\x02\n\xb5\x01\xbbo=\x00\xc9e\xebd%qw\xa7\x11\xec\x91Im\xe9\xb60w\x19\xa8\xba\xb4\x1a\x90\x18C[\x82\xea9`\xc7dP\xc5\xebx\xda)m\xce`_\xf4\xa0\x00\x81\x94\xcf^\xe6\x00\x00\x99f'
('192.168.100.7', 48923) = b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb95d5270eda863\x8d\xbc\xfb\\\xf3$\x8cuz\xbd\x85\x9f\x98\xa0\x80;\xadv\xf4\xb0\xef\xb7\x9e=\xf3<m\xbc\xa6q\x06\x01Sg,\xca\x8d>/a\xdd\x91~\xd5\x14S\x19\xf6h\xbf\xab\xda\xe6\x9d*\xa91^\xde\xde\t\xbdC\xfbx\xe8\xbb_J\xb8#x\x86\xbd\x86\xaaXVAh\xdaX\x0b\xe9H\xb3+\xc2\xc5\xf3 \x84\x96\xda\xf4"\xe4p\x06d\xe8\x90\xfbI\xcb\x0f\xe2\xba\x84\xcb\x97\xee$\xd4LJ>F\xc4\xb8\xff\xa7\x80LNb\x87\x01dJ\x00pb\xd1\xaf\xf5\xc3W\x15\x02\x07J\xce\x80|\xa9l\xec/\xc2S\xb9G\xa1\xcb+\xbdLbUD\xd5.\xd1\xa1\x1b\xea(\xf8r\xff\xe9\xb5\x00\x00\x99f'
('192.168.100.7', 48923) = b"\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb9ad41b933a215#\xed?\xb8\xadp&XwmJ c\xce\xaa\xaa\xbe\xb2\x06u\xa3\xf0b\x9d_\xd3$\xa1sl\xfc;\xfaSS\x1c\x1c\x1d\xd6\x05\x87b\xa0W\xb4\xae\x12\x8bG\t}P\x85\x86\x88\xff\xfa\x85\xa6\xf4Quj\x1f\x83\xf3Y\x05\xfd\x07\xde\x17U\xfb\xc9;\xa4\x16\x05\xffO\x0e5\x8f\n]\xe7\xd34!\xc6\xb5W\x8cOIR\xdd#\xc6\xd1\xb7*\xcf\xadv\xc8MS\xb2\xf0HM\xb0\xd0\xf8\xc3\x87]\xbf\xafK\xf7O\rR\xa8\x03\x94\xa0\xbdD-p\x8bD\xf2\xae\xef\xdd4\xca\xb9r\xe8\x86\xf7y\xa8\xff<\xe7N\xfbht\xd188\x1aK\xb8j 'v\xaa.@5\x81\x1d\xb4\x00\x00\x99f"
('192.168.100.7', 48923) = b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb9a88c8914513e\x0fTY\xf7\x0b7\x94\x92\xb6z\xdf\x7f\x0c9\xdf\xa2\n\\\xc3\xaaW\x81\x86\x1bIi~U\x99k\x89\x81\x0c\x8e\x9e\xe5%\xd0x\xd0\x1c\x15z\xafM\xb6\x98\xa9Y\x83\xf0\x90a:I;\xc3\xa5\x00\xacK6v\x94\xbc4r\x1d=\x8c\xf6\x1c^\xeff\x9aNP\xb8;\tV\xe5\xffG\xc6@\x02\x00\xe5\x84\xa9!1\x0c\x83o\xf2\x95\xfc&\xfb\xbca\x08\x88;\x959\x91D\x16E\x96]\xf4\x0f\x1f\xc6\x98\xdaL]5\xa6\xc3\xb9,\xcb\xd8\x8b\xce\x12\xaf\x9a\xd8\x80\x937|\x07\xe1\xcd\x84e\xb4\xf2\x80\xbbQ\xfdC\xb0r\xe2\x9bq\x7f\x99\x9d\x13n\xb3\x84o\xde\xb7P\xc5\x8e`,u\x00\x00\x99f'
('192.168.100.7', 48923) = b"\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb9ff8cfe8abdb7'\x90\xf2\x17}\x9b\x88\x91\xd3\x16\xcb\\\xbe\xd5#\xc5\x84\xe15\xe6`*\x86\x9f\xe5\x87\x92\xd5\x04<~ei\x11\xc6\xef\x0eK\xffE\xd9\xec\x92 u\xfe\\\xa4s55\xc1\xd3\t\xe3;#=\x0c\x0c\x02.\x8f\x98\xb3\xbfrP\xe8=\x02\xe4\x0e\xa1\xca\x84\x9b\x0e\xee\xc0\xeeZ\xc4}\xfd\xf5;\xfe+\xe7w\x90\x0c\xcc\xd0s\xa8\xd2\xb7dCkG\x94\xce\xb4\xf5\xeakl\n\xd4\xe9\x11;\xeb8\x97\xe87\xf9\xc6\r\xf6\xb5\xed\x11O\xdfd\x1d\xaf%\xb8!\x99\xc0\x14?u\xe9\xd4\x10\x98\xe4\xf9C\x99\xedO\xef\x08\xa4\xad\xa4\x9d\xe9\xea)\\\xbfI;+\xe3\x04\xa5\xc3\x00{|\x92?\x00\x00\x99f"
('192.168.100.7', 48923) = b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb96fa11c6b4e9d\x81\x8cq\xee\xaf\x02\xb0E.\xf6d\xbd\xe5\x10\x97\x17\xdc%\x96}\xae\x80\xc5M\xdb\xa2@\xae\xc5 \xd6\xdd_\x0e\x88GB\xbfBQ$("\xb9\xdf\x084\x8f\\`y\xd7Mpu\xac\xa1~z\xbeS\x0b\x1b\xb3\x13\xeb\x1fk\x06|H\xf27\xf2>A\xcb@\xe1\x98\x98\x12\x1a\xf7X\x1b\xcd\x92\xe5\xf9_\xfa"\xe3\xe3\xdb\xae\x086\xe5?\x86\x99\x02.\xe8T[&\xfbh\x87u\x1c\x8c\xc9e\xffK\x05\xf5V\x04G\x9fSh\xf1\xf3\x00y\\N\xb7p{[Z\x1dv\x1d\x84\xd1\xf7\xba\x14\x0e\x17\x85\xcd\xfd\xeak\xd9\xad\xe5\xe2\x01/q\xdfKPA+OV\xf0\xd6XH\xbcT\x00\x00\x99f'
('192.168.100.7', 48923) = b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb98a2db6db6674\x99\xd6!)\xb6\xa1\xdf\xf4\x04\x9a\x01\\\xe74#\xb6q\x82\xcdY\xb0\xa2\xc9RU\xad\xfc\xaf%\x01\xc5\x07\xedM\xe4\xdb#Fl\xd3\x91\nQ\x0f#\x90y\xbdc9\x1e\x031.\xf3%\xbf\x8b\xa3\xe6\xf3\x9b\x93\x7f\x85\xbd\x94\xdc4yJ\x938Kt\xefA\x80\rX-\xaaf\xfc\x88\xe3um[ \xd5j\x85\x1eT\xf6\xfcC\x84~\x97\x82\x9cd\x9an1\x1d\x88.\xaa:\xac\x8eM\xea\x8c\xbc\xfa\x8d\x06`2\xa9\x00\x18B\xf9~;\x89\xe0\x1e\x11*P\x15\x0b\xcb\xdd\xfe\xf0\r\xbb-s@\x7f?\xfdF\xd0\\\x17Sq)\xd1\xe5\xfbWf\xe5\x9b\xa2k@\xae\xc6B@~\x89\x00\x00\x99f'
('192.168.100.7', 48923) = b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb940f0db75a26ah\x07jp\xbd\xc4\x0f\xe1\xf6$\xb7\xd2\x1e5;\x1a\xc2\xf6\x05>\xaf\\\x8c\xdclN\x10\xf7\xaeRL\x8fL\xd3\xb8\x8a\x15d<o\xee\xc3\x15\xbe\xe8q\x94R\x16\xf5\xec\x1a\xf05\x0b\x81\xed\xd0\x01w\x83\x19G\x1e\x80\xf1K\r\x01\xb9\xac\xee\xd9K\xd3(\x88#;\xec\x0f)\xf6$T\xb3\xe8\x9c\xe5hs\x85m\x87\n4\x1a\xc7\x019\x9e\xa11Z\x01\xe2\x18M\xb3\x17\x82.N\x8e`\\\x01\xf1\n\x87\xc8\xcd\x00z\x01c\x12\x11\x88fTFA\\N\x80\x1f\xe2\xd4[I\xed\x1d\xdf\xc0a\x8b~\x98\xfb}\n\xe3BX*g9\xc3\x96e\xa8o\x85`D\x86\xa4\xe0c\x05\x8c\xc8\x00\x00\x99f'

I just wonder if in the data returned by the script, there are ok the duble or triple \ or chars like `, !, ), >, ?, @, ", %, $, ., &, \n, # etc.

uzlonewolf commented 1 year ago

Unfortunately the specs for those USB-to-serial adapters are not clear about what voltage they use for RX/TX I/O, and for this you will almost certainly need 3.3v.

Are you sure you'll be able to dump the firmware from it? It's basically a mini computer running Linux and you can't just dump the firmware like you can an ESP32 or something; you'll need to root it and use something like netcat (nc) to transfer it to your PC.

uzlonewolf commented 1 year ago

I just wonder if in the data returned by the script, there are ok the duble or triple \ or chars like

Yeah, that's normal for Python byte strings. You can make it display in hex by replacing print( addr, '=', data ) with print( addr, '=', data.hex() )

romicaiarca commented 1 year ago

Unfortunately the specs for those USB-to-serial adapters are not clear about what voltage they use for RX/TX I/O, and for this you will almost certainly need 3.3v.

Are you sure you'll be able to dump the firmware from it? It's basically a mini computer running Linux and you can't just dump the firmware like you can an ESP32 or something; you'll need to root it and use something like netcat (nc) to transfer it to your PC.

I never did this, but the internet is big enought to find the way to intercept the comunication with those ttl to usb and/or get a dump.

I just wonder if in the data returned by the script, there are ok the duble or triple \ or chars like

Yeah, that's normal for Python byte strings. You can make it display in hex by replacing print( addr, '=', data ) with print( addr, '=', data.hex() )

ah, ok, I understand. I am asking because I'm a PHP dev and I have only a week of python (just running the example tinytuya and other example plus some debugging on scanner from tinytuya - I've tried to change encoding from UTF-8 to some other formats, even latin-1, where I've obtained a lot of chinese chars).

Also I've looked over the internet to see if there is any other known key for dectypt. Also I've made some changes on remove the envelope of the udp package (data[20:-8])

uzlonewolf commented 1 year ago

Also I've made some changes on remove the envelope of the udp package (data[20:-8])

data[20:-8] is only valid for the 55AA header, this new 6699 header uses a different format and those byte counts are wrong for it. data[18:-4] would be a good starting point, but I suspect the next 12+ bytes form part of the key and there's also going to be either a 4-byte or 16-byte checksum at the end.

What's really throwing me for a loop is the message contains an odd number of bytes. All previous encryption schemes used AES in ECB mode (which requires a multiple of 16), so either we need to cut even more off or they're not using ECB mode.

uzlonewolf commented 1 year ago

Got it!

It's AES in GCM mode with the first 12 bytes being the iv/nonce. Key is the same md5(yGAdlopoPVldABfn). The checksum is 16 bytes just like v3.4, so it's probably a SHA256 HMAC.

b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb92e0c0c3b1b87\xb6\x06\xaa\xafX\x83\xdd\xc36~S\xc2\xb7^\x8cr\x9a{\x0eo\x84a`\xf6\xdd\x99\xeeU\x88x\xbf\xabNu\xad/1\xf9\x9dP\xf1*S\x94\r\xe0\x8c\xf5p\x84\xfeq\xec\x88}g\xd8\x9f8`\xfcm\xc4\xa9b\xb5\xd6\x94\x92\xe70\x7f\xa8\x90\xd7\xda\xb1\xf2>\xe2\x02\xfe.\xcf\xc9\xb6\x9b#\x00\x91\x9b#X\x11T\x99\xdd\xa4JU\x9b\xdcE\t\xc0\xed\xa8T\xa0\xcb\xcez\xcb]*\x02\n\xb5\x01\xbbo=\x00\xc9e\xebd%qw\xa7\x11\xec\x91Im\xe9\xb60w\x19\xa8\xba\xb4\x1a\x90\x18C[\x82\xea9`\xc7dP\xc5\xebx\xda)m\xce`_\xf4\xa0\x00\x81\x94\xcf^\xe6\x00\x00\x99f'

-> 

b'\x00\x00\x00\x00{"ip":"192.168.100.7","gwId":"bf583<snip>","active":2,"ablilty":0,"encrypt":true,"productKey":"keyyj3fy8x98arty","version":"3.5","token":true}L\xcf\xbdW\xb3]\xa7\xc2\xfe\xff\xb1\x83p\xe3\xa4y'

@jasonacox I'll start getting this added.

On a side note the broadcasts from the app on port 7000 decrypt the same way b'{"from":"app","ip":"172.20.10.64"}P\x87\x61\xa9\x8d\xcf\xa5\x1a\x92\x1fsl\xb9\xa3f\xfeI\x09D\xaa'

uzlonewolf commented 1 year ago 
from Crypto.Cipher import AES
from hashlib import md5,sha256

import socket

clients = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
clients.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
clients.bind(("", 6667))
clients.settimeout(7)

amode = AES.MODE_GCM
key = md5( b'yGAdlopoPVldABfn' ).digest()

for i in range( 8 ):
    data, addr = clients.recvfrom(4048)

    header = data[:18]
    iv = data[18:30]
    payload = data[30:-4]

    print( addr, '=', payload )

    decrypted = AES.new(key, amode, nonce=iv).decrypt( payload )

    print( decrypted )
    print('')
romicaiarca commented 1 year ago

Nice! So you are pretty close to get these devices work? :)

uzlonewolf commented 1 year ago

No, there's still a long way to go for that.

At this point I need to see some actual data changing. If they're still doing the 3-way handshake they did in 3.4 then I need a packet capture from between the app and a device while in local-offline (no internet) mode. If they are now using a nonce in every packet instead of that 3-way handshake then a passively listening socket may work.

Can you try running this and, while it's running, use the app to turn something off or on? Or warm up a temperature sensor or something?

import socket
import time

ip = '192.168.100.7'

for i in range( 3 ):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout( 5 )
    sock.connect( (ip, 6668) )
    print( 'connected!' )
    stime = time.time()
    sock.settimeout( 300 )
    while True:
        try:
            data = sock.recv( 4096 )
        except:
            print( 'connection closed after ', (time.time() - stime) )
            break
        print( 'data:', data )
    sock.close()
    sock = None

If that does return a data: ... line then I'm also going to need the private key for the device. You can generate a new key if you want by removing the device from the app and re-adding it.

romicaiarca commented 1 year ago

@uzlonewolf , here you have the output for the code:

❯ python -m sensor_local
connected!
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
data: b''
. . . 
data: b''
data: b''
^Cdata: b''
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/romica.iarca/code/tinytuya/sensor_local.py", line 192, in <module>
    socketsScanConnect()
  File "/Users/romica.iarca/code/tinytuya/sensor_local.py", line 145, in socketsScanConnect
    print( 'data:', data )
KeyboardInterrupt
jasonacox commented 1 year ago

"version":"3.5"

So we are looking at v3.5 devices?!

@uzlonewolf you are a rockstar! How do we get some of these devices? If any one is willing to donate one to the cause for TinyTuya support, please let us know. If I happen to get one that turns up 3.5, I'll send one to you, LW.

romicaiarca commented 1 year ago

"version":"3.5"

So we are looking at v3.5 devices?!

@uzlonewolf you are a rockstar! How do we get some of these devices? If any one is willing to donate one to the cause for TinyTuya support, please let us know. If I happen to get one that turns up 3.5, I'll send one to you, LW.

Not sure if version3.5 is because of the Gateway or sensors. If the problem is Gateway, I only have one, if the problem are sensors, I can donate one motion sensor (I have more of those and I can donate one), because those have same unrecognised issues. I'm from Romania, so not sure if they will get to you fast enough. Just let me know!

romicaiarca commented 1 year ago

@uzlonewolf, not sure if this may help, but if I am putting the sensor in pairing mode, and the script from https://github.com/jasonacox/tinytuya/issues/248#issuecomment-1379828709 is running, Tuya Android App cannot find the sensor.

uzlonewolf commented 1 year ago

So we are looking at v3.5 devices?!

Yep. It seems like it was only yesterday we were working on v3.4.

I'd feel a lot better about it if I could figure out what checksum/signature hash it uses. Unlike the 4-byte CRC32 from <=3.3 and 32-byte HMAC-SHA256 from 3.4, this one is 16 bytes long. A MD5 hash is 16 bytes, but I just can't figure out what data is hashed to generate the signature.

I dug through my giant box of random Tuya devices and managed to pull out a Moes brand wired Zigbee gateway which appears to have the same board as the SilverCrest, so if @romicaiarca can dump the firmware from theirs then I should be able to load it onto mine. Using dump.py from https://paulbanks.org/projects/lidl-zigbee/ I was able to dump mine, but the serial port is sloooooooow and it took almost 4 hours. If you instead just grab the root password and use the serial port to simply launch dropbear then you can pull it in like 38 seconds with ssh -p 2333 root@192.168.x.x cat /dev/mtd?dro >mtd-dro.bin.

I've been poking around it for a while now and discovered it seems to make connections to all my other devices in order to listen for scene triggers, so if I can figure out where it's saved then this might be a way to get a complete list of local keys without a developer account.

uzlonewolf commented 1 year ago

Not sure if version3.5 is because of the Gateway or sensors.

It's the Gateway.

uzlonewolf commented 1 year ago

Something else of interest with this gateway is it is listening on TCP ports 6668 and 6682, and UDP ports 6608 6667 6681 12121 12122 40604 and 56735. No idea what any of that does.

romicaiarca commented 1 year ago

Something else of interest with this gateway is it is listening on TCP ports 6668 and 6682, and UDP ports 6608 6667 6681 12121 12122 40604 and 56735. No idea what any of that does.

I will try tomorro to check your code from https://github.com/jasonacox/tinytuya/issues/248#issuecomment-1379828709 to see if I get informations from the sensors. I want to try to see if each port is listening for a specific device (motion sensors on 6682, lights on 6608 etc.).

Have you tried to contact https://paulbanks.org/projects/lidl-zigbee/ maybe he still have the original dump of his silvercrest gateway?

My question and propose still remains regardig getting the dump from the gateway/devices with https://www.aliexpress.com/item/4000120687489.html

Screenshot_20230113-232246_Chrome. From the description looks like they support 5v and 3.3v. But I will be new in this think and I will need to know if buying this convertor ttl to usb, will be the only hardware that I need to get the dump from the gateway and/or sensors.

uzlonewolf commented 1 year ago

The gateway has been out since 2020, so even if he has an original firmware dump it won't be the correct version.

I believe that "3.3v and 5v" is talking about power pins. That converter supplies both 3.3v and 5v power, but it does not say which it uses for I/O. The CH340G version has a jumper you can move to select between 3.3v and 5v I/O, but I'm not sure how well supported the driver for that chip is.

uzlonewolf commented 1 year ago

Ok, figured out a few more things. First, you can, in fact, pull some of the local keys for other devices out of this gateway. I haven't been able to figure out where they're stored, but scraping the memory of the tyZ3Gw process reveals them. I'm not sure how it decides which devices to take an interest in and which to ignore.

Next, the firmware upgrade file for Moes v1.19.2 is downloaded from http://s3-us-west-2.amazonaws.com/airtake-public-data/smart/firmware/upgrade/20220902/1662107489-tuya_rtl8196e_gw_tar_UG_1.19.2.bin . Rename to .tar.gz to unpack. Unfortunately due to the date/time embedded in the URL I have no idea what the URL for the SilverCrest gateway is.

Finally, I finally got the signature figured out. Those 16 bytes are the "tag" which gets passed to .decrypt_and_verify() after you .update( result[4:18] ).

from Crypto.Cipher import AES
from hashlib import md5

amode = AES.MODE_GCM
key = md5( b'yGAdlopoPVldABfn' ).digest()

data = b'\x00\x00f\x99\x00\x00\x00\x00\x00\x00\x00\x00\x00\x13\x00\x00\x00\xef\\\x86\xc0v$y]\x96\xdd#Y)\x02\xa2"\\\xc2\xe0\xcf\xcd}\x89\xd4@\xcd\xcc6\xc7\xc1S\\\xe1\xdc,\xef\xfcu\x13\xae\x02\xff\xe0\xc6\xfa\xd7r"Y\x953\x1a\xc0\x10\xdc\x9cZe\x08\x1e\x97\xc8\xb1\xe7\x89j\xb6\xb56\x10u\x0e\xcd\xfdf\xf9\xcbA\xcf\x04\xff9\xa02\xe3\xe9\xc0\xe2V=\xc8\x17\xa8"\xaa[\xee\x84\x8aN\xf9\xd5\xcb:\xf4|\x0b\xcdT\x04\x03\x03\x82^\xd1\x1b\x7f\xb57\xaa,\x05{O\x04\x86\xd8\x0e\x8a0\x9e\xd6/\xd2\x0cE\xe9\x97\xfa\x92t\xfa\xa1}\x80\x9f\x01\x98\xf3\x07\xa9\xeb\xce\xb1 !`w\x92\x06\x14\xffT^\xe8\x19>~\xf4\x0b\x0f\x0c\xb6,\x11\xb9\x05^\xdc@\x97a\xfdOa\xb7\x9d\xd0M\xb4P\xdb+\xe3\x87\x9b\xd8d\xb0=\xa4\xb2Z\x04\xc2s\xa4_v\x9d#\x7f \xc1\x99\x97\xadF6\xf8U\x8aT\x1b\xac\xa2\x85\xf7\xb9 \x8d\xb4M\x02\xe2\xd7\x94\xd7\xa97\x00\x00\x99f'

header = data[:18]
iv = data[18:30]
payload = data[30:-20]
tag = data[-20:-4]

c = AES.new(key, amode, nonce=iv)

# the header is part of the signature but is not encrypted
c.update( header[4:] )

# will throw "ValueError: MAC check failed" if verify fails
decrypted = c.decrypt_and_verify( payload, tag )
print( 'decrypted:', decrypted )
print( '' )

# re-encrypt to verify
c = AES.new(key, amode, nonce=iv)
# add the header to the signature
c.update( header[4:] )
encrypted, tag = c.encrypt_and_digest( decrypted )

print( 'Encrypted:', encrypted )
print( 'Tag:', tag)
print( '' )
print( 'Final:', header + iv + encrypted + tag + b'\x00\x00\x99\x66' )
romicaiarca commented 1 year ago

The gateway has been out since 2020, so even if he has an original firmware dump it won't be the correct version.

I believe that "3.3v and 5v" is talking about power pins. That converter supplies both 3.3v and 5v power, but it does not say which it uses for I/O. The CH340G version has a jumper you can move to select between 3.3v and 5v I/O, but I'm not sure how well supported the driver for that chip is.

Then can you suggest sumething for me to get that dump that you need in order to finish the investigation for this?

uzlonewolf commented 1 year ago

I'll see if I can dig one up.

In the meantime, can you run this (after adding your ip and local key) and let me know if it displays anything?

import socket
import time
import struct
from hashlib import md5,sha256
from Crypto.Cipher import AES

ip = '192.168..'
key = b'YOUR LOCAL KEY'

for i in range( 2 ):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout( 5 )
    sock.connect( (ip, 6668) )
    print( 'connected!' )
    stime = time.time()
    local_nonce = b'0123456789abcdef' # not-so-random random key
    local_iv = local_nonce[:12]       # not-so-random random iv

    pkt = struct.pack( ">IHIII", 0x6699, 0, 1, 3, len(local_nonce)+len(local_iv)+16 )

    c = AES.new(key, AES.MODE_GCM, nonce=local_iv)
    c.update( pkt[4:] )
    encrypted, tag = c.encrypt_and_digest( local_nonce )

    pkt += local_iv + encrypted + tag + struct.pack( ">I", 0x9966 )

    #print( pkt )
    #print( pkt.hex() )

    sock.sendall( pkt )

    try:
        data = sock.recv( 4096 )
    except:
        data = None

    print( 'data:', data, 'in', (time.time() - stime) )

    sock.close()
    sock = None
romicaiarca commented 1 year ago

I'll see if I can dig one up.

In the meantime, can you run this (after adding your ip and local key) and let me know if it displays anything?

import socket
import time
import struct
from hashlib import md5,sha256
from Crypto.Cipher import AES

ip = '192.168..'
key = b'YOUR LOCAL KEY'

for i in range( 2 ):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.settimeout( 5 )
    sock.connect( (ip, 6668) )
    print( 'connected!' )
    stime = time.time()
    local_nonce = b'0123456789abcdef' # not-so-random random key
    local_iv = local_nonce[:12]       # not-so-random random iv

    pkt = struct.pack( ">IBIII", 0x6699, 0, 1, 3, len(local_nonce)+len(local_iv)+16 )

    c = AES.new(key, AES.MODE_GCM, nonce=local_iv)
    c.update( pkt[4:] )
    encrypted, tag = c.encrypt_and_digest( local_nonce )

    pkt += local_iv + encrypted + tag + struct.pack( ">I", 0x9966 )

    #print( pkt )
    #print( pkt.hex() )

    sock.sendall( pkt )

    try:
        data = sock.recv( 4096 )
    except:
        data = None

    print( 'data:', data, 'in', (time.time() - stime) )

    sock.close()
    sock = None

Hi, sorry for the delay, here you have the result for local IP of the gateway and local key of the gateway

python -m sensor_local
connected!
data: None in 5.0029120445251465
connected!
data: None in 5.0005292892456055
❯ python -m sensor_local
connected!
data: None in 5.002928018569946
connected!
data: None in 5.00040602684021
uzlonewolf commented 1 year ago

Sorry, I had a typo in it, can you replace >IBIII with >IHIII and try it again? Thanks!

romicaiarca commented 1 year ago

Sorry, I had a typo in it, can you replace >IBIII with >IHIII and try it again? Thanks!

Yep, now we have output:

❯ python -m sensor_local
connected!
data: b'\x00\x00f\x99\x00\x00\x00\x00\xb68\x00\x00\x00\x04\x00\x00\x00P6df8fe7b1aaf\xf1O\xc1\xa3\x85\xc1\xc2\xb0(m\xdf\xb6\xd5\xb0N\xc9\x91\x90\n)\xd7\x08\xfb\xdb\xf8\xef\xd4/\x0f\x12\xba\x8e\x0c\x9e\x83\xb2\xb0\x8ba\x0f\x10\xac\x18\xfd\xbf\xd8I\xad\x84Ru\xe1\xe7\xf4\x05\x8aC" \xb4\x151\xeb\xa6A8\x8eg\x00\x00\x99f' in 0.026500940322875977
connected!
data: b'\x00\x00f\x99\x00\x00\x00\x00^\x9d\x00\x00\x00\x04\x00\x00\x00P099e73d7a984\x08\x84f\x99\x07y\xdc\xff\xfb\x83E\xc4\x9f\x97\x9e\xb1Mu\xac\xf6e!\xb9\xb3\xef\x1c:\xcc\x03\xa7T\'&\x80\xb0!A\xa7\xd3\x02\xac \xe6\x02\x95r\x02=1\xba\x9d\xbb^\n*\xa5F\xea\xb0\x0fkTS":\xf744\x00\x00\x99f' in 0.03119516372680664
uzlonewolf commented 1 year ago

Can you checkout https://github.com/jasonacox/tinytuya/pull/256 and let me know if it works?

romicaiarca commented 1 year ago

Can you checkout #256 and let me know if it works?

git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   tinytuya/core.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    output.txt
    sensor_local.py
    socket_listen.txt

no changes added to commit (use "git add" and/or "git commit -a")
❯ git diff
❯ python -m tinytuya scan

TinyTuya (Tuya device scanner) [1.9.2]

Scanning on UDP ports 6666 and 6667 for devices (15 retries)...

*  Unexpected payload=%r
 b'f4a6803010\xca\xf1p\xf7\xfc?;\xf3\x90\x9a\x0f\xf5\xa6\x06\x11C_\xce/\xd5\xb7\xde\xaa\x073\xda\xd4!_\xbb?Q\xe6_\xa4\xee\x87\xdc\xf9\x8f<\xc7\xfc\xce\xd4Q\xbb\x8c\nP\xc13\x96L|m[C\xb2\x1f\xbdeH\x85\xbfe\xd2f\xd2\x8a\x13\x9e\xb0\x1c\x95\x86\xc0\xbb\xad\x13\xfb\xfc\xf3\x8ag\xf6\xdbf\xc5\x1a`k\xbcm\x8d\xbc"g~\x92\x1cp\x13}\x1d\xa3\xd6\xd3bO\x88\xe6\xe5\xf7\x11I\x8cdwEe\xc5\xd0B\x85\xa9\x9f\'\xd0\x07\xf2\x9b\xa1\x8fJP\xbe\xec\xa4\xbc\x87\xe5\x9f^\xae\n"\x9c\xaf\x1c\x88\xdcw\xee\xdfg@\x9e\xff\x9d\xc1Z;\x1f%<\xb1\x1ee'
Unknown v Device   Product ID =   [Unknown payload]:
    Address = 192.168.100.7,  Device ID = , Local Key = ,  Version = , MAC = 
    No Stats for 192.168.100.7: DEVICE KEY required to poll for status
*  Unexpected payload=%r
 b"d6609426a3\x84\x80M\x03.\xcb(;\xaf%\xe2Y=\xe3\xfc\xa4'Aa\x10\xacH\xa1V\xfb\x89\xa7\xf8\x9c\xc3\xf7\xc8\xbfq\xb4\xe42A49 \x14\xc4X*\x90\xd8w\xd1\xa3Hj\xe2\xda\x1c\x7f0\\\xb2\x80<\xed\xbdC\xfbd\x9e{\xae=\x109\xb8(\xbe\xc0'\xe9\xd46\x97\t\x1dp1\xe7\t\x1c\x89c]r-\t^\xe9W\x91\xeaa\x9f\xa4\xca\xc1\x8dn\x9c\xca\xb2x\xec\xb2\xe5p.*\x84\x1e\x1a\x06d\xfb\x12\xb6Y_?\xdc\xee\x11=\xe7o\xd8`\xc5q$C\xdca\x123\xe5\x00L\xe07c\x17\xf9\xab\x8ci\xbc\x85\x91le\x1f\xd48\xfe!o|\xa6c`"
*  Unexpected payload=%r
 b"fcaa4ec537&\x14\x84\xf5\x08\xea\xcea\xf5\xc8\x80[L\xb5P\xcd\xb2\xcd\x86E\x9d-\xa1\t\x8b\xce\xd0E\x1d\xc6l}\x98\x1faU\xefa\x05j%x\xdd\xbd\x918<T\xf6R\xf7b\x84\xd8/\xa3\xfd'm\x06-\xdd\xf6[\xea\xf9\xbf\x06fa\xbez\x9b\x80+\x86L\xef\xe9\x13\xdf:\xd4gV\xb1\xaa\xc9E\x80\xe8H\x18E5\xe0\x8c\xeb\xd7U\x03\xf6\xa6oE\x84\xf2\x9c\xa0\xc26\x82\x8a\xab\xb2\x0e\x0f\xadBP\xa3TqC\xbf\xben\x1a\xbc)S\xde\xea+\xfe\x92~[\x8e!s\x91\xcf<\xf3\xcd\x1b\xeb\xf0Y\xeb\xc8\xc9J\x17;c\xbd\x10m\xe2^\xf8O\\J1\xe9l"
*  Unexpected payload=%r
 b"7eb9f11c89\xda\x00\xa7\xc5a\xd8\xedW\xea\xf2J\x92\x9fw\xb2\xdd\xa0-\xab\xf1\xa1\x0c\x1a'\xb9\x02\\\x87\xc3^\x1e\xf6\x84\xc3+\x931T\x1ad\xd1\xc2\xd5\x01\x04t\n\x01\x81\x03r\xb6\xe5\xc6\xcdd\\\r\x87\xa6\xfa\x7fW6[\xfcU\xa3i\x9c\xa3\xe4\x1c\xd1r.\xd7\xf5<\x1e\xfdyd\xd1\x14\xda\xd8F\xe9jcC\xf4\x9a\x98\x18\xd00\xbf2\x85\x14\x0b}[\xf3\xee\x8ba\x80V\xa0\x01\xcc\xaf:8+':\xde\x9f\x1d\x1f\x90\xf3g \xbc\xbdy\xf5\xb1|\x9e5\x8d\xa0\x9f\x10y\x93Ok\xb2\x0fM\x1d\xa5\xd0\x12\xf1K\xb6/\xd6_^\x12\x93,i\xa0\xb1\xff\xd9\xf9:\xad"
*  Unexpected payload=%r
 b"42eb07baa3x^\xc0>\x8b@\xf4\xc4\xff@\xf2\x10h\xf3\xbb\t!9\xeb\xfd\xc7\xe7S\xdf\x91\xc3\xd9_*\x00m\xf5\x89$ \xeb\x90'6\xcb\xf6{+\xf9k\x9f\xf5\xe9\x0b\xfa+\xdb\xd6FY\x1aY\x1c\x13\xad\xc4\xa5j\xf2F\xeeL;\xd0\x16\xd0X\x97\xb7\x05\x88\xcb\x8d\nG\xae\xe0\x0es?x0\x1e\xdc\x15\x92\t3z\xa5\x82L\x97\xa5G\xc8\xa2\xaaN\t(\xa5\x13,@\x0c\xc4O./\xba\xf3\xd5\x85\x9b\xd2;|\x0e\xf00\xe2W0(\xd7\x168j\xa5J\xd4F\xa6\xc0\x05\x83%\x18\xf7\x1bi\x92\x010\x1f\\>\x894\xc39t/\xfbi]L\xf5\x98c\xd2\xd8\xbb"
*  Unexpected payload=%r
 b'16fd2f0e9ak\x0b\xea\x9e\xb7r\x0e\x9cG-\xdb\xf6\xb8\xfb9\xd1\xcfK\x0e\xb7\xdb\xb5zj\x98p\xc7Q\xa4\xcc\x88\xc2\xdd\x83\xc9\x93\xbb\xa3\x83w\xef@\x97\xac\x88l\xa5q\xd1Z\xbf\xad\x05\xe9\x132\x8a\xfaJ\x0bK\xd7C|\xd6\xc5\x93\xe9\xf4Yr\xad>]\xbf.R\x1b\x019qY6(\xab\xcf\x99)\xf8H\xe8\x8d\xfc3\x98\x1b\xc0\x15o\x82u\xef\xcf5D\xa7v\x17\x82\xc4\x1d\x07}H\xda\x84\xe3s\xd8\xe6\xdb\x0f\xf7\x17\x97\x82\'\xfa{\x1e\x14\xd1\xd9\xbe);\xa4\x89\x9b"\xcd\xa4\xc1\x9c\xd1\xf1\xdc\xc1u\xa9&\xf9*4R\xe5CN\xee\n\xaf\x1c\xf7\xb8S\x9b\x98\xf3\xbb'
*  Unexpected payload=%r
 b'b0ae5ea737\x17l\t1R6<t\xe9\xe6\x1ct(\x9e"\x04\xba\xea\x86\x1d\xd0\xa9QX\xf7O..\xae)\x7fkbet\xca\xcc\xa92c\xe6p\x1b<\xe2\xbc\xe3\xcc+\x90\x7fgi\x13\xc7?k\x94\x07D\x962q\xe5\xcc"\xde\x16\x92\xbe\xf6\x1a\xb7\\\xa9:V\xd0\xa3\x03\xb5\x90\xcb\x91~\xd9C]{\x1a\x916\xc0\xcf\xc2\xc2\xdf\ro\xc7M\x94z\xea\x10b\x11\x9f\x06\xc2\x1d>\xb2\xc5\xa7H\xcfe\xa7\xce-\xcc\x1e\x92\xb7\xb5~\xc7\xa80\x08\xb89\xf9]\x01\xeeMQ\x92D\x01>70"\x11\x13\xe1<SI;\xe5`\xc1X!\xa9v\xb2w<\n\xecgw\x188'
*  Unexpected payload=%r
 b'cb196b715f!\xe3}\x8a\xa8z2\xfc\x8bu\x1c\xdc8\x1c\xfc\xf6)~{\xcbe\x80\xbb;\xabj\x87ka%\x99\x92.@\x88k.\xa8\xe5\x83"R\xa1\xad-\x0c@\x0f\xd3\x8ci\x9e\x17X\x89\xb7+\xa0\x1bHL\x1aK\xaf\xdc\x86$AQ\xed\xfeOSxA\xcd\xa4\x97VI\x05I\x8f\x95\x9e\x0e=\xc3\x93A\xf3\xddz\xca~\xc39\x80\xb8\x14\xcc\x1d4\x0f\xabg,\xfdV{\'\x1f4:\xb16\x0f\x99x\xf0W\xcd=r\xa8\xc4F\x02]4z\x18/C\xd7\xe0}\xc7.\x9e,\xab\xf5\xc0\x1f\r\xee\x90T\xe1\x86\xa3P\x19\xe6a\xd5P`\xdf\xed^\xc7`\xca\xf5o\t{'
*  Unexpected payload=%r
 b"3856aa06457bDz4Df(\xee\x8cj\x14\x9cqp&D<T\xb9\xa3`\x94\x97\xd3\x9a\xf2\xb0a'\xe8\xf7\xf6\xa7\xd1\x84\xb3\xaa\x15e\xf4J\xcb\xa5O\x06\x12\x07\xc1$)\xe2\x0bi\xbdWm\x0fc\xb9\x91\x9c\x08\xe3\xb9\xd3v\x91\xae\x07\xe2\xa8\x98\x95\xb6\xf0.\xfa.\n\x88\x8f\xbeRO$\xd6\x9bS\x9e\x7f\x11\xca*\xf3\xddZ\xacZhb\xa84\x90%\x01yJi\xdc\xa3t\x9d\x0enZ\xb0[\x8c#\xecq\xf4\x08`%\x85\x9e\x91\xb5ml*\xb6HP#\xbf\xa1\x97\x9b|\xd4\xfc\x95\xc7p\rB  \xa3\x19=(ZV\x06zS\x9e\xad*m\xe6S\x99\x8a\x1c"

Scan Complete!  Found 1 devices.

>> Saving device snapshot data to snapshot.json

╭─   ~/code/tinytuya on   master !1 ?3

Here you have the output for scan :).

romicaiarca commented 1 year ago

Also runnig this code:

def getStatusDeviceViaGateway():
    tinytuya.set_debug(True, False)
    # configure the parent device
    gw = tinytuya.Device( ZIGBEE_GATEWAY_ID, address=ZIGBEE_GATEWAY_IP_ADDRESS, local_key=ZIGBEE_GATEWAY_LOCAL_KEY, persist=True, version=3.5 )

    print('GW IP found:', gw.address, '- getting status')

    # configure one or more children.  Every dev_id must be unique!
    zigbee1 = tinytuya.OutletDevice( THS_OFFICE, cid=THS_OFFICE, parent=gw )
    # zigbee2 = tinytuya.OutletDevice( 'eb04...l', cid='0011223344556689', parent=gw )

    print(zigbee1.status())

I have this output:

❯ python -m sensor_local
DEBUG:TinyTuya [1.9.2]

GW IP found: 192.168.100.7 - getting status
DEBUG:status() entry (dev_type is v3.5)
DEBUG:building command 10 payload=b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673736292","cid":"bf26..2gu"}'
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000001000000030000002c31363733373336323932382e924a26e11ef49877f8448961b5aca7c0a8c13c19825a71197674ad8493a435a000009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000002000000030000002c31363733373336323937382e9b14e1863f80389911eea1d8088eb6e6cfd87679038c03b1b71b6e6d6cc8f9dc00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000003000000030000002c31363733373336333032382e797564085e00cd6e122e6223d45273fb94de7ee0277e20942e4a96afe00a0be600009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000004000000030000002c31363733373336333037382e84db5089eb66a734dc0ffbb8f9b843e2411316fbed2fecdc42775bc8130b8e1700009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000005000000030000002c31363733373336333132382ea0d29190188a79e66868a4d25bea5b85948a5ad99f566c7573142e53cb39cd0a00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:ERROR Network Error: Device Unreachable - 905 - payload: null
DEBUG:status() received data={'Error': 'Network Error: Device Unreachable', 'Err': '905', 'Payload': None}
{'Error': 'Network Error: Device Unreachable', 'Err': '905', 'Payload': None}
uzlonewolf commented 1 year ago

Ok, can you checkout https://github.com/uzlonewolf/tinytuya/tree/pv35 and let me know if it's any better? Also, can you double-check the local key to make sure it's correct?

romicaiarca commented 1 year ago

Ok, can you checkout https://github.com/uzlonewolf/tinytuya/tree/pv35 and let me know if it's any better? Also, can you double-check the local key to make sure it's correct?

On scan part, it looks a little bit better:

❯ python -m tinytuya scan

TinyTuya (Tuya device scanner) [1.9.2]

Scanning on UDP ports 6666 and 6667 and 7000 for devices for 18 seconds...

*  Unexpected payload from '192.168.100.7' to port 6667: '\x00\x00\x00\x00{"ip":"192.168.100.7","gwId":"bf58...4zgjr","active":2,"ablilty":0,"encrypt":true,"productKey":"key...arty","version":"3.5","token":true}' (b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb9924c8c20b6aaN\x05\x84\xf9a_\x96J\xadH\xa7X\x93 \x7f\xe1\xdc\x1c"x\x81m\x1eN\x07\xd3BL\xe7<\xa3W\xf1\xce\xb1iL\xfa\x13v\x93\xc4\xe2\xd4DxK\xa1\xe4.y(\x83\x89\xf9m\xcb\xa3\xb2!.\x8fi\x86\x167U\xbf\x06\xf7|\xe9\xe3\xff\xb99\xa2\xfb\x90q+\xea/r\x13\xad\xb6\x92\x80r\xb5\xe7\x142j\x8bm\xc6\xb3\x0b\n\xe4\xd5\xb9@\x1ej\x1d_\x84;R8\xff\x8e=\x9a\xec%i\x82+\xb4\xe4\xf4\x07\xf3\xe8\n\xfa[k\x89K\xbe,\x94n\x8aP\xc3s\xed\xd4\xcea\x05p\x11\xb2w\x83KC$\xa5\xc4`W\xb1\xbb\xe7\xe7Fu\xe3\xdc\x90\xc9\xae\'\x81\xed\x00\x00\x99f')

*  Unexpected payload from '192.168.100.7' to port 6667: '\x00\x00\x00\x00{"ip":"192.168.100.7","gwId":"bf5....gjr","active":2,"ablilty":0,"encrypt":true,"productKey":"key....rty","version":"3.5","token":true}' (b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb9abf9744a3c073g/MF({>\x7f\x8e\x1d\xcbv\xca\xce\xb3u\x9d\x1a\xa0\xd4\xc4\x06\x12\xba\xe7\x9f4<\x15\xd6|\x8e\x9b\xec\x8c\xc6)\xd8\xa4\xbc:\xf9\xfcw\xba\x99\xccW0?\x83r\x85\xdc\xf9\x93\x0e\xe1\xa5\x91\xf2\xb8&\x98\xad\x1b\xe2\xd6\x91W\xa6\x87Y`/\xe3e\xf9t$\xe7@)\xda\xfa\xd8\x11\xa59\xa6\xdc\xa9j\x19\xc7\x10\xa4\xb3\x04\n\xc8H X\x17\x18\x1cf\xa1\x92\xb5Y_\x94\xe9\x818\xd2\xef\xad}\xdd\xcb$\xe2l\xeb\xb0/\xdf\xdehF\x8f\xe5\xc5_\x8aMr\xec5\xd5D\xa6K\xc8\xad\\i\x81\x1bR5\x8c!\xe8\x96\xb7w\x86d\xa1O{D\x8e5\xfb\xafP\xf3\x00\x00\x99f')

*  Unexpected payload from '192.168.100.7' to port 6667: '\x00\x00\x00\x00{"ip":"192.168.100.7","gwId":"bf58...4zgjr","active":2,"ablilty":0,"encrypt":true,"productKey":"key...arty","version":"3.5","token":true}' (b'\x00\x00f\x99\x00\x00\x124Vx\x00\x00\x00\x13\x00\x00\x00\xb9d983ae731cf9T\xd0\xc9f\x03\xdb^\xcc;\xb73\xc5\xc1\xf8,\xa0\xd4\xa9\x98\xc99\x18\xd0]MA\xd8\xb2\x8d\x86bM"\x85/\xef=\xfa=\x96\x8e\xcb\xe0\x87\x9eF"xV\x93\x86\x1f\xabv\x9e\x1e\x8b\xf4\xbd\xe5\x13\x0b\xc58"I7\xe8\xc6i.\xfe1\xa3ia]l\xbf\xd6\xb8\xf0\xd8\xe0\xfa\x82\x1d\x96\xe5\xdf\xec\xc5]\xe8\xf6\x06\x14.\xb9\xee\xea\xe4C\xd5\xb1\xdf\xde\xb3\xe6-^\xdbS\xdd\xd6\xba\x8d\xaf\xec\xe2n\x17fp\x9c\x11\x9b\xb6\xe4\xb0\xf3\xe0C\x13^7\\\x01\xc5\xbe\xd9\x02\xe7.r7M\xc78\xbf\x1e!gy\x01g/o\xd9\x1b[\\2HL\xff\xa8G\x87\xb5\x87\x8d\x12\x00\x00\x99f')

Scan completed in 18.0585 seconds                 

Scan Complete!  Found 0 devices.
Broadcasted: 0
Versions: 

>> Saving device snapshot data to snapshot.json
romicaiarca commented 1 year ago

The output for get deviceViaGateway is :

❯ python -m sensor_local
DEBUG:TinyTuya [1.9.2]

GW IP found: 192.168.100.7 - getting status
DEBUG:status() entry (dev_type is v3.5)
DEBUG:building command 10 payload=b'{"gwId":"bf583defebc47442a4zgjr","devId":"bf26e524aed4d6bfe9m2gu","uid":"bf26e524aed4d6bfe9m2gu","t":"1673739150","cid":"bf26e524aed4d6bfe9m2gu"}'
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000001000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac8b7f44f3b3599e5a01532962557888a8600009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000002000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac816d7cfb5a4688a950ef014422128f09f00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000003000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac87636b0302b385079f84e6a60f348d96800009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000004000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac89690cea8878a54ff3975108ccc6804ac00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000005000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac8f671b12d08da8e13cfcb6eae1e082d5b00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:ERROR Network Error: Device Unreachable - 905 - payload: null
DEBUG:status() received data={'Error': 'Network Error: Device Unreachable', 'Err': '905', 'Payload': None}
{'Error': 'Network Error: Device Unreachable', 'Err': '905', 'Payload': None}
romicaiarca commented 1 year ago

And one more think that I just noticed:

For any subdevice (sensor) local_key is the same with the one of gateway. But... the sensors have some node_id attribute... maybe this can help you. :)

uzlonewolf commented 1 year ago

If you do a git pull the scanner should be fixed. I'm not sure why your gateway is not responding to your sensor_local script though, can you reboot it and try again?

romicaiarca commented 1 year ago

If you do a git pull the scanner should be fixed. I'm not sure why your gateway is not responding to your sensor_local script though, can you reboot it and try again?

I'll tomorrow :-). It is a bit late in Romania.

uzlonewolf commented 1 year ago

Sounds good. Please do a git pull before trying as I've made a few more updates.

romicaiarca commented 1 year ago

Sounds good. Please do a git pull before trying as I've made a few more updates.

❯ git checkout pv35
Already on 'pv35'
Your branch is up to date with 'origin/pv35'.
❯ python -m tinytuya scan

TinyTuya (Tuya device scanner) [1.9.2]

Scanning on UDP ports 6666 and 6667 and 7000 for devices for 18 seconds...

Unknown v3.5 Device   Product ID = keyyj3fy8x98arty  [Valid Broadcast]:
    Address = 192.168.100.7   Device ID = bf583defebc47442a4zgjr (len:22)  Local Key =   Version = 3.5  Type = default, MAC = 
    No Stats for 192.168.100.7: DEVICE KEY required to poll for status
Scan completed in 18.0464 seconds                  

Scan Complete!  Found 1 devices.
Broadcasted: 1
Versions: 3.5: 1
Unknown Devices: 1

>> Saving device snapshot data to snapshot.json
❯ python -m sensor_local
DEBUG:TinyTuya [1.9.2]

GW IP found: 192.168.100.7 - getting status
DEBUG:status() entry (dev_type is v3.5)
DEBUG:building command 10 payload=b'{"gwId":"bf583...4zgjr","devId":"bf26e...m2gu","uid":"bf26e...9m2gu","t":"1673781189","cid":"bf26e...m2gu"}'
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000001000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac8b7f44f3b3599e5a01532962557888a8600009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000002000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac816d7cfb5a4688a950ef014422128f09f00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000003000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac87636b0302b385079f84e6a60f348d96800009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000004000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac89690cea8878a54ff3975108ccc6804ac00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000005000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac8f671b12d08da8e13cfcb6eae1e082d5b00009966'
DEBUG:received null payload (None), fetch new one - 1 retries remaining
DEBUG:received null payload (None) but out of recv retries, giving up
DEBUG:session key negotiation failed on step 1
DEBUG:ERROR Network Error: Device Unreachable - 905 - payload: null
DEBUG:status() received data={'Error': 'Network Error: Device Unreachable', 'Err': '905', 'Payload': None}
{'Error': 'Network Error: Device Unreachable', 'Err': '905', 'Payload': None}
uzlonewolf commented 1 year ago

Can you do a git pull (not checkout), reboot the gateway, and try again?

romicaiarca commented 1 year ago

Can you do a git pull (not checkout), reboot the gateway, and try again?

❯ git pull origin pv35
From github.com:uzlonewolf/tinytuya
 * branch            pv35       -> FETCH_HEAD
Updating de107a3..11a3e2e
Fast-forward
 server/server.py    |  2 ++
 tinytuya/core.py    | 14 +++++++-------
 tinytuya/scanner.py | 26 ++++++++++++++------------
 3 files changed, 23 insertions(+), 19 deletions(-)
❯ git stash pop
On branch pv35
Your branch is up to date with 'origin/pv35'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    new file:   sensor_local.py

Dropped refs/stash@{0} (97e70f4c496a770de04626338e3ceca66f175fe3)
❯ python -m sensor_local
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/romica.iarca/tinytuya-v35/sensor_local.py", line 229, in <module>
    localScan()
  File "/Users/romica.iarca/tinytuya-v35/sensor_local.py", line 96, in localScan
    scanRes = tinytuya.scan()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 1755, in scan
    scanner.scan(maxretry=maxretry, color=color, forcescan=forcescan)
TypeError: scan() got an unexpected keyword argument 'maxretry'
jasonacox commented 1 year ago

Hi @romicaiarca - the new scanner updates from @uzlonewolf changed maxretry to scantime, but there were shortcut function in the core library that still referenced maxretry, which I suspect you are using. Thanks for the catch! I fixed the mapping and pushed the change in the main repo. Using @uzlonewolf repo, you can run the scanner directly (avoiding the maxretry error):

import tinytuya
from tinytuya import scanner

tinytuya.set_debug(True)
scanner.scan()

Or you try to pull from https://github.com/jasonacox/tinytuya.git where I made the fix (but @uzlonewolf may have other updates in his fork):

import tinytuya

tinytuya.set_debug(True)
tinytuya.scan()
romicaiarca commented 1 year ago

Hi @romicaiarca - the new scanner updates from @uzlonewolf changed maxretry to scantime, but there were shortcut function in the core library that still referenced maxretry, which I suspect you are using. Thanks for the catch! I fixed the mapping and pushed the change in the main repo. Using @uzlonewolf repo, you can run the scanner directly (avoiding the maxretry error):

import tinytuya
from tinytuya import scanner

tinytuya.set_debug(True)
scanner.scan()

Or you try to pull from https://github.com/jasonacox/tinytuya.git where I made the fix (but @uzlonewolf may have other updates in his fork):

import tinytuya

tinytuya.set_debug(True)
tinytuya.scan()

He have updates in the fork since he tries to find a way to make SilverCrest gateway to work (v 3.5). That's why O am using his forked repo. I'll make the fix that you said and test his implementation for v 3.5

romicaiarca commented 1 year ago

@uzlonewolf , here you have output for scan:

❯ python -m sensor_local

TinyTuya (Tuya device scanner) [1.10.0]

Scanning on UDP ports 6666 and 6667 and 7000 for devices for 18 seconds...

Unknown v3.5 Device   Product ID = keyyj3fy8x98arty  [Valid Broadcast]:
    Address = 192.168.100.7   Device ID = bf583defebc47442a4zgjr (len:22)  Local Key =   Version = 3.5  Type = default, MAC = 
    No Stats for 192.168.100.7: DEVICE KEY required to poll for status
Scan completed in 18.0653 seconds                  

Scan Complete!  Found 1 devices.
Broadcasted: 1
Versions: 3.5: 1
Unknown Devices: 1

>> Saving device snapshot data to snapshot.json

and here you have the output for connect to device with gateway:

❯ python -m sensor_local
DEBUG:TinyTuya [1.10.0]

GW IP found: 192.168.100.7 - getting status
DEBUG:status() entry (dev_type is v3.5)
DEBUG:building command 10 payload=b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673837820","cid":"bf26...m2gu"}'
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000001000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac8b7f44f3b3599e5a01532962557888a8600009966'
DEBUG:received data=b'00006699000000008564000000040000005030333038343530643337333382560567f25d41dc97f81d832ee646deaa0b1d1c10bf82886f1ab72904627b8121e0d12aa30991deda6aa21e42acb26cd1b9daf1d3a86f624819a528002a6bb79d648d0f00009966'
DEBUG:decrypted session key negotiation step 2 payload=b'\x00\x00\x00\x00c9f9356412cfa8cf&r\x0eK\x81\x90\xc8\xad\x88\xf3\x18\x8bzy\xe8\xc2\x13c\xd2]Q\x91\xe9\xe67\xe3\n\x1c\x02\xbf\xe6\xf0'
DEBUG:payload type = <class 'bytes'> len = 52
DEBUG:session key negotiation step 2 failed HMAC check! wanted=b'26720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c02bfe6f0' but got=b'6138636626720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c'
DEBUG:session local nonce: b'0123456789abcdef' remote nonce: b'\x00\x00\x00\x00c9f9356412cf'
DEBUG:sending payload quick
DEBUG:final payload: b'\x15\xa6?\x87\x83\xaa\xde n\xa7\xa4\xad\xd9\xa3j\x15\xed\xa6\x7f\xb8>/\xe7\x85\xa8w&Q\xeb\xcd/\x0b'
DEBUG:payload encrypted=b'00006699000000000002000000050000003c303132333435363738396162362e458f067d3049c7d97626851e45bb71f4150450be7826229896c9208fc971881dc7642b3b84b51d7e650db624db2500009966'
DEBUG:Session nonce XOR'd: b'0123W\x0cP\x0e\x0b\x0cWVRV\x06\x00'
DEBUG:Session key negotiate success! session key: b'\xb2g7T\xc6hw\xeb\xaf\xc1|\xe1M\x82#\xb8'
DEBUG:sending payload
DEBUG:final payload: b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673837820","cid":"bf26...m2gu"}'
DEBUG:payload encrypted=b'0000669900000000000300000010000000ad3031323334353637383961620a9b16e0520a1f1ceb1185355ac983fbd33d269e4648cd1eab76bf0143bb57e52c026ba827a1f07495e2bc01f8ffbf1d1e32170876ebfcdd505fc10a34c1e7e6c80a336a9a4a766ff44603133325bb381a624bbc832390d2de651b6a997ef70b7952f731f88cd0c9e572a16d7c91d7124f01f53361f9a8336099800c0e12874263e7606a3376c677ff6001b91708bdfa3b06186f0e564523bd7d890d893dff38a000009966'
DEBUG:Network connection error in _send_receive() - retry 1/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 817, in _recv_all
    newdata = self.socket.recv(length)
ConnectionResetError: [Errno 54] Connection reset by peer
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000004000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac89690cea8878a54ff3975108ccc6804ac00009966'
DEBUG:received data=b'00006699000000007e7f0000000400000050393532353464623232343563c44215cce3d5efc6182088b6f59b2009cffa63172d2fe48b21318a406307513ecb559dd7965c39dba406e0803e9957c836bb8f53bc5fc80c173badc1d4b3f2104bf3a48c00009966'
DEBUG:decrypted session key negotiation step 2 payload=b'\x00\x00\x00\x00a28206337a8db514&r\x0eK\x81\x90\xc8\xad\x88\xf3\x18\x8bzy\xe8\xc2\x13c\xd2]Q\x91\xe9\xe67\xe3\n\x1c\x02\xbf\xe6\xf0'
DEBUG:payload type = <class 'bytes'> len = 52
DEBUG:session key negotiation step 2 failed HMAC check! wanted=b'26720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c02bfe6f0' but got=b'6235313426720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c'
DEBUG:session local nonce: b'0123456789abcdef' remote nonce: b'\x00\x00\x00\x00a28206337a8d'
DEBUG:sending payload quick
DEBUG:final payload: b'0\xb1ft:\xba]#\x10\xda\xef\xbaY\xc8e50\xb7\xb2;\xf0\xa2c\xdb\x14\xd6T\xb1\xcclw\xd3'
DEBUG:payload encrypted=b'00006699000000000005000000050000003c30313233343536373839616213391c7cbf6db34ab9a43d3105754a9bace5d8879e33fc789e39e429072e91a9feea8ca163f005264b0515726a82dbea00009966'
DEBUG:Session nonce XOR'd: b'0123U\x07\x0e\x05\x08\x0fRQT\x05]\x02'
DEBUG:Session key negotiate success! session key: b"\xf4s'\xff\xd7\xe0\xd9\xf1 \x19\xe9\xd4\x96\xffEo"
DEBUG:sending payload
DEBUG:final payload: b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673837820","cid":"bf26...m2gu"}'
DEBUG:payload encrypted=b'0000669900000000000600000010000000ad3031323334353637383961629288b08e551d27e58cb9aa06e8e8cbbc46bc3fc704e4117b54bc7b08b19a8a36084ab23522d3620990ce48adec6f947bf20ce7a8851355cf563e3c3a153899dccc58bc904fc98c7f9fdac0039e79243b9579a311749a4b603b18fc952587eff85e0053308c463df77625867a1a981d79d8aa78b73b336a91d8c31e6a7db3eea7c7f3fdfd351212429aad0f96021d75e733c81aab126c706dcb2cf0893d013a810e00009966'
DEBUG:Network connection error in _send_receive() - retry 2/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 817, in _recv_all
    newdata = self.socket.recv(length)
ConnectionResetError: [Errno 54] Connection reset by peer
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000007000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac837b34e26167b3bca22b792ebbac87eb500009966'
DEBUG:received data=b'000066990000000010c200000004000000503164373630636165376430646fb0763b4e220c2b77f9ade30d4430123df1dbb4d82164afe78879be89bc9c97509a42046f182cdee27125282b5307a9ef0e0ba9375889471bf63f0caf39238a7eb72aa500009966'
DEBUG:decrypted session key negotiation step 2 payload=b'\x00\x00\x00\x009acb3fc9301e8fad&r\x0eK\x81\x90\xc8\xad\x88\xf3\x18\x8bzy\xe8\xc2\x13c\xd2]Q\x91\xe9\xe67\xe3\n\x1c\x02\xbf\xe6\xf0'
DEBUG:payload type = <class 'bytes'> len = 52
DEBUG:session key negotiation step 2 failed HMAC check! wanted=b'26720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c02bfe6f0' but got=b'3866616426720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c'
DEBUG:session local nonce: b'0123456789abcdef' remote nonce: b'\x00\x00\x00\x009acb3fc9301e'
DEBUG:sending payload quick
DEBUG:final payload: b"4!\xa05X\x17\xfd\x03'6\xcb<u\xe8(\x9d \x8f\xbd\x9ecvsU\xa3r\xcb\x95f.\xf9\xf5"
DEBUG:payload encrypted=b'00006699000000000008000000050000003c30313233343536373839616217a9da3dddc0136a8e4819b729550733bcddd7220de7ecf6299d7b0dad6c1f8fc2de08aacf78efc019b52fd366324cd200009966'
DEBUG:Session nonce XOR'd: b'0123\rTUU\x0b_\x02[PTT\x03'
DEBUG:Session key negotiate success! session key: b'_\x81D\x08z\x17:\x1cO\xc0\xcc\x81n Ut'
DEBUG:sending payload
DEBUG:final payload: b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673837820","cid":"bf26...m2gu"}'
DEBUG:payload encrypted=b'0000669900000000000900000010000000ad3031323334353637383961624ceb8c498f6fa2e401e603723984c8a98d5511bc7b6299fdbe8c463d1f7a5c0e4d58bb19a745f9d92359f221051473c7546a67be15c188c87c36a0297a34eeced090407bae77c6543cb203e73225071d892d694b928d73342b43e32596c381f9b639896bbf5a404ec13cc61e251439604a78f7f6798e72396dbe06ac946555840167de7760e69cc17114dfc18690bf82301b61ad414503eec78219a164dd5f4c7d00009966'
DEBUG:Network connection error in _send_receive() - retry 3/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 817, in _recv_all
    newdata = self.socket.recv(length)
ConnectionResetError: [Errno 54] Connection reset by peer
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'0000669900000000000a000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac895dc3399deee5df2bb03e554b229bf2500009966'
DEBUG:received data=b'0000669900000000c76b0000000400000050376138353165356539333637fd6dde15cd6e8d3743bb67ec8992a1fdf32446e741e362494c59289972a5630d429f16bb6f3f94351636c4c8898e7aadd0dbb9e08b6755d529be6484ad804c3e8c9b87c200009966'
DEBUG:decrypted session key negotiation step 2 payload=b'\x00\x00\x00\x00cfc7c8ab54e57dc6&r\x0eK\x81\x90\xc8\xad\x88\xf3\x18\x8bzy\xe8\xc2\x13c\xd2]Q\x91\xe9\xe67\xe3\n\x1c\x02\xbf\xe6\xf0'
DEBUG:payload type = <class 'bytes'> len = 52
DEBUG:session key negotiation step 2 failed HMAC check! wanted=b'26720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c02bfe6f0' but got=b'3764633626720e4b8190c8ad88f3188b7a79e8c21363d25d5191e9e637e30a1c'
DEBUG:session local nonce: b'0123456789abcdef' remote nonce: b'\x00\x00\x00\x00cfc7c8ab54e5'
DEBUG:sending payload quick
DEBUG:final payload: b'\xf4\xfcV\x17\xae\xa0\x8fI\xfa\xe4>\xa6\x13\xee\xa4\x8d\xa1q \x8f;\xb5\xcd\xba\x94\x0e8\\\x18\xcf\xd6-'
DEBUG:payload encrypted=b'0000669900000000000b000000050000003c303132333435363738396162d7742c1f2b776120539aec2d4f538b233d234a33552452191ee188c4d38d30572c6eddd2aa2920dcb20ec9e9f04763a200009966'
DEBUG:Session nonce XOR'd: b'0123WSU\x00[\x01\x00\x00VP\x00S'
DEBUG:Session key negotiate success! session key: b'\xcd\\\xec&\xf9[\xbb\x00{\x82\x06\x8e\xea\xf6\xc4\x9b'
DEBUG:sending payload
DEBUG:final payload: b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673837820","cid":"bf26...m2gu"}'
DEBUG:payload encrypted=b'0000669900000000000c00000010000000ad303132333435363738396162947244ade53c7d82f533cb664a7732a46b3e9ba5eb000c9a86a295d077386ac0df3250698862993c20e8f0f2e40d386ef66ea345012731d9c4cf3f0c5fc4ddff903d0bf42ec7d2b7d4c8fd73bdb76629bd4410cb3655611cfbc9cdc591674c3fb6aa7e2a29ce7564a22ae405e05a2ebbf6003a84e00672f42c7dbc816009b1d5ad3e95de8e63c094261a2198beaed6f091f9a8f455c39f103808b542566db8816400009966'
DEBUG:_recv_all(): no data? b''
DEBUG:_recv_all(): no data? b''
DEBUG:Error decoding received data - read retry 0/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 823, in _recv_all
    raise DecodeError('No data received - connection closed?')
tinytuya.core.DecodeError: No data received - connection closed?
DEBUG:_recv_all(): no data? b''
DEBUG:_recv_all(): no data? b''
DEBUG:Error decoding received data - read retry 1/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 823, in _recv_all
    raise DecodeError('No data received - connection closed?')
tinytuya.core.DecodeError: No data received - connection closed?
DEBUG:_recv_all(): no data? b''
DEBUG:_recv_all(): no data? b''
DEBUG:Error decoding received data - read retry 2/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 823, in _recv_all
    raise DecodeError('No data received - connection closed?')
tinytuya.core.DecodeError: No data received - connection closed?
DEBUG:_recv_all(): no data? b''
DEBUG:_recv_all(): no data? b''
DEBUG:Error decoding received data - read retry 3/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 823, in _recv_all
    raise DecodeError('No data received - connection closed?')
tinytuya.core.DecodeError: No data received - connection closed?
DEBUG:_recv_all(): no data? b''
DEBUG:_recv_all(): no data? b''
DEBUG:Error decoding received data - read retry 4/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 823, in _recv_all
    raise DecodeError('No data received - connection closed?')
tinytuya.core.DecodeError: No data received - connection closed?
DEBUG:_recv_all(): no data? b''
DEBUG:_recv_all(): no data? b''
DEBUG:Error decoding received data - read retry 5/5
Traceback (most recent call last):
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 952, in _send_receive
    rmsg = self._receive()
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 842, in _receive
    data = self._recv_all( min_len )
  File "/Users/romica.iarca/tinytuya-v35/tinytuya/core.py", line 823, in _recv_all
    raise DecodeError('No data received - connection closed?')
tinytuya.core.DecodeError: No data received - connection closed?
DEBUG:ERROR Unexpected Payload from Device - 904 - payload: null
DEBUG:status() received data={'Error': 'Unexpected Payload from Device', 'Err': '904', 'Payload': None}
{'Error': 'Unexpected Payload from Device', 'Err': '904', 'Payload': None}

These outputs are right after I've restarted the gateway (unplug from socket for 1 minute and plugged in back.

An other think that I've noticed now, is that on router UI I have the information that the device type is: udhcp 1.13.4 In Tuya app I can see the following version information:

Main Module: V1.2.33 ZigBee Module: V1.0.9

uzlonewolf commented 1 year ago

Ok, I think I finally got it fixed. Please git checkout https://github.com/uzlonewolf/tinytuya/tree/pv35 then git pull and try it again. You may need to reboot the gateway since they tend to get mad on too many connection/handshake failures.

romicaiarca commented 1 year ago

Ok, I think I finally got it fixed. Please git checkout https://github.com/uzlonewolf/tinytuya/tree/pv35 then git pull and try it again. You may need to reboot the gateway since they tend to get mad on too many connection/handshake failures.

Ok, I have good news :). Not sure if somewhere is mentioned what cid is, reported to the TuyaAPI response attributes, but it looks like the corresponded of cid from is node_idfrom TuyaAPI response. After your update in the repo, and my change on ... I have the following output:

❯ python -m sensor_local
DEBUG:TinyTuya [1.10.0]

GW IP found: 192.168.100.107 - getting status
DEBUG:status() entry (dev_type is v3.5)
DEBUG:building command 10 payload=b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673840508","cid":"a4c...6c4"}'
DEBUG:sending payload quick
DEBUG:final payload: b'0123456789abcdef'
DEBUG:payload encrypted=b'00006699000000000001000000030000002c30313233343536373839616213b9483bb1e2d85e9147b3e93fd94ac8b7f44f3b3599e5a01532962557888a8600009966'
DEBUG:received data=b'000066990000000014440000000400000050616666323131316434653633eee030974c2544061bcc848852ee20e0ec20c5741b0d454e3217f5b8cf495f27fa45337605728cec3ee9d092a2227bd9ceb5a84de837f4c7bfa32e726ce1be69e89bcae800009966'
DEBUG:decrypted session key negotiation step 2 payload=b'86019f2b3208b816&r\x0eK\x81\x90\xc8\xad\x88\xf3\x18\x8bzy\xe8\xc2\x13c\xd2]Q\x91\xe9\xe67\xe3\n\x1c\x02\xbf\xe6\xf0'
DEBUG:payload type = <class 'bytes'> len = 48
DEBUG:session local nonce: b'0123456789abcdef' remote nonce: b'86019f2b3208b816'
DEBUG:sending payload quick
DEBUG:final payload: b'&\x07Cp\x0f\xe6t*z\x9d\x87\xb8"v[Y\xbf\xf8\xbd\xd9\x13\x9f>\xb5\xe6LQN\x0e\xf3\xeb\x9b'
DEBUG:payload encrypted=b'00006699000000000002000000050000003c303132333435363738396162058f39788a319a43d3e355337ecb74f723aad7657d0ea1166ca3e1d6c5b10de14e04e37ce04343769d6a21d7e0ad24a800009966'
DEBUG:Session nonce XOR'd: b'\x08\x07\x02\x02\rS\x04U\x0b\x0bQZ\x01\\TP'
DEBUG:Session IV: b'0123456789ab'
DEBUG:Session key negotiate success! session key: b'+\x8fx\n\x88\x84\xea<\xa2u\x83\xd1]\xe1{\xfe'
DEBUG:sending payload
DEBUG:final payload: b'{"gwId":"bf58...zgjr","devId":"bf26...m2gu","uid":"bf26...m2gu","t":"1673840508","cid":"a4c...6c4"}'
DEBUG:payload encrypted=b'0000669900000000000300000010000000a730313233343536373839616239bae2e530fc5fd6a939905be28c1a0c9134aa5d5e837662f454af5556eeeeee202734045f533e229b6feb8a604b3bfeeb9cd918b6affbe51b73c973a060976c8f68a2cb322b863e72a1894a1b41f6726dc8ff64e3d085cc397e557c9933bf7238a94cd7225f4a641bfe6131d5616bc78bcd95fc8428f8accb2a4141b41f7594e3f9f218a54992e24fdeceb7597270d1ce5e0715a216e2e500224b00009966'
DEBUG:received data=b'00006699000000001445000000100000005830336338396361326263653010e963bbb45a1ef82a1d7f7e09e6e5c842323ffc5cd56520d26eb754a32c5eb4eee0e4c38362df9b0ccc24c1182f72e72d8911c25dc00aca7432ad4970c7a2971c9ca4229114cf11835cd0c200009966'
DEBUG:received message=TuyaMessage(seqno=5189, cmd=16, retcode=0, payload=b'{"dps":{"1":177,"2":833,"4":1},"cid":"a4c...6c4"}', crc=b'p\xc7\xa2\x97\x1c\x9c\xa4"\x91\x14\xcf\x11\x83\\\xd0\xc2', crc_good=True, prefix=26265, iv=b'03c89ca2bce0')
DEBUG:raw unpacked message = TuyaMessage(seqno=5189, cmd=16, retcode=0, payload=b'{"dps":{"1":177,"2":833,"4":1},"cid":"a4c...6c4"}', crc=b'p\xc7\xa2\x97\x1c\x9c\xa4"\x91\x14\xcf\x11\x83\\\xd0\xc2', crc_good=True, prefix=26265, iv=b'03c89ca2bce0')
DEBUG:decode payload=b'{"dps":{"1":177,"2":833,"4":1},"cid":"a4c...6c4"}'
DEBUG:decoded results='{"dps":{"1":177,"2":833,"4":1},"cid":"a4c...6c4"}'
DEBUG:status() received data={'dps': {'1': 177, '2': 833, '4': 1}, 'cid': 'a4c...6c4', 'device': OutletDevice( 'bf26...m2gu', address=None, local_key='', dev_type='v3.5', connection_timeout=5, version=3.5, persist=False, cid='a4c...6c4', parent='bf58...zgjr', children={} )}
{'dps': {'1': 177, '2': 833, '4': 1}, 'cid': 'a4c...6c4', 'device': OutletDevice( 'bf26...m2gu', address=None, local_key='', dev_type='v3.5', connection_timeout=5, version=3.5, persist=False, cid='a4c...6c4', parent='bf58...zgjr', children={} )}

So it looks like you made it! Thank you for your hard work! Congrats!

Update: It looks like scanning for devices still returning only one device (only the gateway). Not sure if this is how it should work ;)

Update 2:

running this code:

def getStatusDeviceViaGateway():
    # tinytuya.set_debug(True, False)
    # configure the parent device
    gw = tinytuya.Device(devices["gateway"]["ZIGBEE_GATEWAY_ID"], address=devices["gateway"]["ZIGBEE_GATEWAY_IP_ADDRESS"], local_key=devices["gateway"]["ZIGBEE_GATEWAY_LOCAL_KEY"], persist=True, version=TUYA_API["VERSION"])

    print('GW IP found:', gw.address, '- getting status')

    for deviceKey, deviceDetails in devices["devices"].items():
        # configure one or more children.  Every dev_id must be unique!
        device = tinytuya.OutletDevice(deviceDetails["id"], cid=deviceDetails["node_id"], parent=gw)
        # print(deviceKey, " status: ", device.status())

    print(" > Begin Monitor Loop <")
    pingtime = time.time() + 9

    while(True):
        if( pingtime <= time.time() ):
            payload = gw.generate_payload(tinytuya.HEART_BEAT)
            gw.send(payload)
            pingtime = time.time() + 9

        # receive from the gateway object to get updates for all sub-devices
        print('recv:')
        data = gw.receive()
        print( data )

I have the following output:

recv:
{'protocol': 4, 't': 1673842240, 'data': {'dps': {'2': 631}, 'cid': 'a4c...d89'}, 'dps': {'2': 631}}

So it looks like that the protocol that we receive from the gateway/sensors is 4, not 3.5. I don't think that this have any big importance, but I just wanted to mention this. :)

jasonacox commented 1 year ago

Thanks for helping test @romicaiarca . The changes from @uzlonewolf have been merged into the main repo, closing out the remaining protocol v3.5 issues.

A huge thanks to @uzlonewolf ! 🙇 You are truly brilliant!

Please report any other issues. I'll continue my standard battery of test (even though I don't have a 3.5 device) and if all goes well, will be releasing tinytuya v1.10.0 to PyPI later tonight.

jasonacox commented 1 year ago

New release published https://pypi.org/project/tinytuya/1.10.0/

pip install --upgrade tinytuya