900 Invalid JSON from specific device

[3735943886]

I recently purchased and installed a Tuya window opener Zigbee version. It automatically announces its status approximately every 2 hours. However, the logs display errors consistently.

Observations: The device status is announced at regular intervals, but the JSON response is malformed. The Device.set_value() function works without any issues.

2024-05-08 02:00:05,417 - ERROR - 다용도실게이트웨이 {'Error': 'Invalid JSON Response from Device', 'Err': '900', 'Payload': '"102":"open","103":false,"104":85,"105":"none","106":false,"107":"none","109":"left","110":true,"112":"outside","113":28,"114":"pending"},"cid":"385cfbfffe8cfa67"}'}
2024-05-08 04:00:05,420 - ERROR - 다용도실게이트웨이 {'Error': 'Invalid JSON Response from Device', 'Err': '900', 'Payload': '"102":"open","103":false,"104":85,"105":"none","106":false,"107":"none","109":"left","110":true,"112":"outside","113":28,"114":"pending"},"cid":"385cfbfffe8cfa67"}'}
2024-05-08 06:00:05,415 - ERROR - 다용도실게이트웨이 {'Error': 'Invalid JSON Response from Device', 'Err': '900', 'Payload': '"102":"open","103":false,"104":85,"105":"none","106":false,"107":"none","109":"left","110":true,"112":"outside","113":28,"114":"pending"},"cid":"385cfbfffe8cfa67"}'}
2024-05-08 08:00:05,419 - ERROR - 다용도실게이트웨이 {'Error': 'Invalid JSON Response from Device', 'Err': '900', 'Payload': '"102":"open","103":false,"104":85,"105":"none","106":false,"107":"none","109":"left","110":true,"112":"outside","113":28,"114":"pending"},"cid":"385cfbfffe8cfa67"}'}
2024-05-08 10:00:05,423 - ERROR - 다용도실게이트웨이 {'Error': 'Invalid JSON Response from Device', 'Err': '900', 'Payload': '"102":"open","103":false,"104":85,"105":"none","106":false,"107":"none","109":"left","110":true,"112":"outside","113":28,"114":"pending"},"cid":"385cfbfffe8cfa67"}'}

Expected Behavior: The device should provide a well-formed JSON response without errors.

Actual Behavior: The device consistently returns an invalid JSON response with error code 900. The payload appears to be missing an opening brace '{', resulting in the error.

When I try to get status by Device.status(nowait=True), it responds the same as above with an error 900.

All other Tuya devices work flawlessly.

Any idea? Thank you in advance.

[jasonacox]

It seems like there is some buggy firmware here.

Suggestion:

Modify a montior script to turn Debug mode on tinytuya.set_debug(True) to see exactly what you are getting from the device: https://github.com/jasonacox/tinytuya/blob/master/examples/zigbee_gateway.py

[3735943886]

Thank you for prompt comment. I've tested with set_debug(True) and I found out something strange log.

DEBUG:received data=b'00006699000000007a4a00000040000000daa5000b8c481c687e54faf7caca4f6140f3c86c3d17d2a0dd425c5e9433d1331d75c75ed13c9977bf5c51729eafe526451c50bbfbcb6c051eb968c42389ee6a00871292855d378b9f4c97fa2e271aaadbd0268a40b63febd18012cc5da4b57df41d5d0245e4c2845ce864e9f249813e8a94a0b702c0855d4db03945378fd74dcfeadd663ca057d9f1a0c181d6e80df984f32eeb70e37bfe6962bb69d293a9a094626d247dcf63ecfcf07c86e820e28f17057dc9b26f11e947863c5b4f59f17002c9255ab4b29b3e24fb4acd514ae9851bc57c0c5a3014d068b73c00009966'
DEBUG:received message=TuyaMessage(seqno=31306, cmd=64, retcode=0, payload=b'{"reqType":"subdev_online_stat_report","data":{"online":["a4c13822bf8f2c50","a4c138ebec13914c","385cfbfffe8cfa67","a4c138c29d105fa3","a4c13854874f777d","a4c138a78f151123"],"offline":[]}}', crc=b'\xcdQJ\xe9\x85\x1b\xc5|\x0cZ0\x14\xd0h\xb7<', crc_good=True, prefix=26265, iv=b'\xa5\x00\x0b\x8cH\x1ch~T\xfa\xf7\xca')
DEBUG:raw unpacked message = TuyaMessage(seqno=31306, cmd=64, retcode=0, payload=b'{"reqType":"subdev_online_stat_report","data":{"online":["a4c13822bf8f2c50","a4c138ebec13914c","385cfbfffe8cfa67","a4c138c29d105fa3","a4c13854874f777d","a4c138a78f151123"],"offline":[]}}', crc=b'\xcdQJ\xe9\x85\x1b\xc5|\x0cZ0\x14\xd0h\xb7<', crc_good=True, prefix=26265, iv=b'\xa5\x00\x0b\x8cH\x1ch~T\xfa\xf7\xca')
DEBUG:decode payload=b'{"reqType":"subdev_online_stat_report","data":{"online":["a4c13822bf8f2c50","a4c138ebec13914c","385cfbfffe8cfa67","a4c138c29d105fa3","a4c13854874f777d","a4c138a78f151123"],"offline":[]}}'
DEBUG:removing device22 3.x header=b'dev_online_stat_report","data":{"online":["a4c13822bf8f2c50","a4c138ebec13914c","385cfbfffe8cfa67","a4c138c29d105fa3","a4c13854874f777d","a4c138a78f151123"],"offline":[]}}'
DEBUG:decoded results='dev_online_stat_report","data":{"online":["a4c13822bf8f2c50","a4c138ebec13914c","385cfbfffe8cfa67","a4c138c29d105fa3","a4c13854874f777d","a4c138a78f151123"],"offline":[]}}'
DEBUG:ERROR Invalid JSON Response from Device - 900 - payload: "dev_online_stat_report\",\"data\":{\"online\":[\"a4c13822bf8f2c50\",\"a4c138ebec13914c\",\"385cfbfffe8cfa67\",\"a4c138c29d105fa3\",\"a4c13854874f777d\",\"a4c138a78f151123\"],\"offline\":[]}}"
ERROR:다용도실게이트웨이 {'Error': 'Invalid JSON Response from Device', 'Err': '900', 'Payload': 'dev_online_stat_report","data":{"online":["a4c13822bf8f2c50","a4c138ebec13914c","385cfbfffe8cfa67","a4c138c29d105fa3","a4c13854874f777d","a4c138a78f151123"],"offline":[]}}'}

Formerly, I guessed the payload's missing opening brace '{' but it was not. Instead, first 15 bytes of payload were deleted with debug message removing device22 3.x What's wrong? and How can I fix it?

Thank you.

[3735943886]

After further investigation, I've observed that my Zemismart M1 gateway transforms into a device22 type under certain conditions. However, I haven't pinpointed the exact condition triggering this transformation. It seems to occur when I create the sixth tinytuya.Device object for sub-devices.

In the core.py file, around line 1401, there's a conditional statement:

if not self.disabledetect and "data unvalid" in payload:
    self.dev_type = "device22"
    # set at least one DPS
    self.dps_to_request = {"1": None}
    log.debug(
        "'data unvalid' error detected: switching to dev_type %r",
        self.dev_type,
    )

It looks like that if the payload contains the phrase data unvalid, the gateway's device type is set to device22. However, it should not occur since Zemismart M1 gateway works fine without changing into device22.

Currently, I've resolved the issue by adding Device.disabledetect = True immediately after creating the gateway device. However, I think this solution is temporary. It seems like there might be an error in the device22 detection logic.

[jasonacox]

Unfortunately data unvalid is the only way to detect device22 conditions. There seem to be a lot of those devices so keeping that logic in there probably helps the majority of users. We know there are likely exceptions (I believe another one was reported as an issue) hence the disabledetect was added.

I do think your discovery is great. I wonder why your device is randomly sending back data unvalid. Are you able to determine what conditions causes that?

TODO: As a workaround, we should document your finding and the Device.disabledetect = True patch as a troubleshooting step for this class of device (window opener / Zigbee).

[3735943886]

Thank you for your supports and assistance in resolving the detection issue with Device22. After investigation, it appears that the problem did not originate from the window opener device itself, as initially suspected. My analysis suggests that certain combinations of subdevices and Zigbee hubs may trigger this detection problem. While I cannot find out the exact conditions causing this behavior, it may require more trial and error. Now, I would mark this issue as complete in case that Device.disabledetect = True is safe and proper way to prevent the device from being automatically detected, though it is not explicitly documented.

[jasonacox]

Thanks @3735943886