ZHA delays with Tuya mmWave sensors

[a-d-r-i-a-n-d]

The problem

Since I've upgraded to 2024.1.0 my ZigBee devices (lights, switches, fans etc) are having a few seconds/minutes delay in responding to commands. I have tried to restore to previous versions but the issue still persist, no matter the version I am restoring to. I have also tried restarting in safe mode, but it didn't helped. Any help is much appreciated!

What version of Home Assistant Core has the issue?

core-2024.1.0

What was the last working version of Home Assistant Core?

core-2023.12.3

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ZHA

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha/

Diagnostics information

config_entry-zha-1122b1901b65684e9e9a05c89bd2a54d.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.components.zha.core.cluster_handlers
Source: components/zha/core/cluster_handlers/__init__.py:537
Integration: Zigbee Home Automation (documentation, issues)
First occurred: 23:36:04 (7 occurrences)
Last logged: 23:40:47

[0xEA69:1:0x0702]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>')]
[0xEA69:1:0x0b04]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>')]
[0x85AF:1:0x0702]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>')]
[0x85AF:1:0x0006]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>')]
[0x85AF:1:0x0b04]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>'), DeliveryError('Failed to deliver message: <EmberStatus.DELIVERY_FAILED: 102>')]

Additional information

System Information

version	core-2024.1.0
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.11.6
os_name	Linux
os_version	6.1.63-haos
arch	x86_64
timezone	Europe/London
config_dir	/config

Home Assistant Community Store

GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 4955 Installed Version | 1.33.0 Stage | running Available Repositories | 1371 Downloaded Repositories | 7 HACS Data | ok

Home Assistant Cloud

logged_in | true -- | -- subscription_expiration | 22 January 2024 at 00:00 relayer_connected | true relayer_region | eu-central-1 remote_enabled | true remote_connected | true alexa_enabled | true google_enabled | false remote_server | eu-central-1-9.ui.nabu.casa certificate_status | ready instance_id | 8bf80b83acdc431081381385ed2692e2 can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok

Home Assistant Supervisor

host_os | Home Assistant OS 11.2 -- | -- update_channel | stable supervisor_version | supervisor-2023.12.1 agent_version | 1.6.0 docker_version | 24.0.7 disk_total | 58.0 GB disk_used | 12.2 GB healthy | true supported | true board | generic-x86-64 supervisor_api | ok version_api | ok installed_addons | EMQX (0.3.0), Samba share (12.2.0), Studio Code Server (5.14.2), openWakeWord (1.8.2), Advanced SSH & Web Terminal (17.0.1)

Dashboards

dashboards | 2 -- | -- resources | 1 views | 12 mode | storage

MercedesME 2020

websocket_connection_state | connected -- | -- api_endpoint_reachable | ok

Recorder

oldest_recorder_run | 21 December 2023 at 17:55 -- | -- current_recorder_run | 4 January 2024 at 23:22 estimated_db_size | 308.77 MiB database_engine | sqlite database_version | 3.41.2

[ferdyvi]

Hi all, if it can be helpful to code developers, I've fixed my situation. I've increased the default buffering time in xbee router firmware for incoming/outcoming messages from/to spleeping and devices to 20 sec. I didn't test with smaller values, but after several hours no messages to end devices were lost. It could seems a router issue, but in december I didn't have such problems. (are there any change about buffer/timing/timeout/latency between current version and december one?)

[tabrisnet]

fwiw, I find myself in a similar situation. high channel util%, but it's a TubesZB EFR32 POE [Ethernet] coordinator. The nearest WiFi AP is ~12 feet away. There are also two Olimex-ESP32-POE units [~6 feet away]. One running ESPresense, the other ESPHome as a BT proxy. Both connected via Ethernet. It's been problematic on and off for a few months [occasional crashes requiring a reboot of the coordinator and a reload of ZHA], but 2024.1.6 has made my zigbee devices unusable.

One of the devices having trouble with 2024.1 has a clear air path [maybe I'm in the way], right under the WiFI AP. Other devices potentially requiring routers also have the same issue. Under the January release, it either does not respond to on/off commands or takes over a minute.

I have not tried changing the channel, and it is the default 15.

[frank777777]

Anyone already tried 2024.1.6 ? -- i can do it tomorrow morning .

[tabrisnet]

That is the first 2024.1 I tested. And yes, the problem presented itself, much moreso than 2023.12

[netsoft-ruidias]

@frank777777

Anyone already tried 2024.1.6 ? -- i can do it tomorrow morning .

yes, as this version had changes in some Zigbee related packages, I tried it before but nothing changed, all the problems still remain.

Only downgrading to 2023.12 solved the issue.

[wouterjdb]

I can confirm I also tried 2024.1.6 and I've had endless zha crashes with that version and earlier versions in 2024. I went back to 2023.12.1 and zha is fully stable again (didn't try later version than 2023.12.1 now - I recall stability issues for me started just before Christmas last year).

[puddly]

@wouterjdb Can you upload a debug log from 2023.12.x after letting ZHA run for half an hour?

[wouterjdb]

Maybe worth mentioning: I'm running a Conbee 2. Currently the stick is directly in a USB3 port - but I tried all the mentioned solutions with changing channels, using extension cords and moving the coordinator around. That has no influence on the crashes: downgrading has. home-assistant_zha_2024-02-04T18-25-39.706Z.log.txt

[frank777777]

Hi, I can confirm after update 2024.1.6 problem return

[dmulcahey]

If you are on a SI radio (Skyconnect, husbzb-1, sonoff E, etc) today's release has some improvements that should help stability. Give it a try when it is released.

[tjerkw]

I can confirm having a more stable connection on yellow. Still slowness sometimes but overall it’s workable.

-- Tjerk Wolterink

On Wed, Feb 7, 2024 at 14:13 David F. Mulcahey @.***> wrote:

If you are on a SI radio (Skyconnect, husbzb-1, sonoff E, etc) today's release has some improvements that should help stability. Give it a try when it is released.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/107200#issuecomment-1932024701, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGH5MIET4Z2G64WCW3JYMLYSN4XFAVCNFSM6AAAAABBNVOAPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZSGAZDINZQGE . You are receiving this because you were mentioned.Message ID: @.***>

[puddly]

The root cause of this issue is unfortunately Tuya devices. Some types of Tuya devices (especially plugs/outlets with energy monitoring and mmWave sensors) repeatedly send a lot of traffic on the network and ask that the coordinator reply to every single packet with an acknowledgement.

The reason you are seeing this now and not in a previous Core release is that we fixed a bug in ZHA that preventing it from properly replying to these acknowledgements. This is the correct behavior for 99% of devices. Unfortunately, this fix exposed the fundamental problem with these Tuya devices: they continuously send a lot of data, nonstop, and ask for an ACK every single time.

Normally, devices send a report every couple of minutes. Sometimes a few times a minute if they're being actively used. These Tuya devices are sending data often times multiple times per second. If you have 15 of these devices on your network, it will be continuously flooded with data.

---

To anybody affected by this issue

Can you please post the ZHA diagnostics information for your Tuya devices? I want to keep track of which specific ones are causing these problems and purchase them to see if I can test a fix myself.

You can find this diagnostics data here:

Alternatively, if you have a lot of devices or don't know which Tuya devices are causing problems, you can instead enable ZHA debug logging:

And then reload the integration:

After 60 seconds, click the "Disable debug logging" button and your browser will download a log file. Please upload that log file instead.

If possible, could you also post a link to where you bought your devices so that I can look into purchasing a few to test out fixes?

I appreciate the help everyone has provided with tracking down this issue and we can hopefully have a fix released very soon!

[marc4s]

https://de.aliexpress.com/item/1005003446740203.html?spm=a2g0o.order_list.order_list_main.133.e54f5c5f0NVMNg&gatewayAdapt=glo2deu

https://www.amazon.de/Steckdose-ZigBee-Stecker-Metering-Sprachsteuerung/dp/B0CR38TWWH/ref=sr_1_2?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=FR5MB13ZTHTF&keywords=Tuya+Smart+Zigbee+3%2C0+Power&qid=1707330164&sprefix=tuya+smart+zigbee+3+0+power+%2Caps%2C774&sr=8-2

[puddly]

@marc4s It's important to include the diagnostics information. Tuya devices often have different randomly-generated "manufacturer" names and behave differently even if they look the same.

[frank777777]

@puddly we all appreciate YOUR help bro. Thanks .

Now stupid question .. as I have different Tuya (purchased via AliExpress ) … do you need the debug log on the update version right ?? As now I’m running 2023.12.2 .. is useless right ?

[marc4s]

sorry, I thought the log was already here, but maybe it was in another bugreport regarding this.

https://we.tl/t-8E2ShG7HsJ

[Tronnic]

Hello @puddly

Interesting find, you recommended to me in #109839 to restart (unplug and plug) all my tuya devices, this has definitely helped! 1 minute of Debug-logging ended up with a 1186KB file instead of 60-100MB. So the crazy sending of the devices did stop.

I have created a bunch of logs after the re-plugging. I hope they help. My logs before the replugging are in #109839 Post

Home Assistant Core 2024.2.0b11 Supervisor 2024.01.1 Operating System 11.5 Frontend 20240205.0

1 Minute Debug-Logging debuglog-home-assistant_zha_2024-02-07T19-25-17.239Z.log

HomeAssistant Log home-assistant_2024-02-07T19-29-29.860Z.log

Tuya Plug v2 - Aliexpress zha-tuyaplugv2-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_w0qqde0g TS011F-0c9074a7d85d8bdce019f828521072ed.json

Tuya Plug v2 zha-tuyaplugv2-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_w0qqde0g TS011F-bfcb4ad69f8f51db82dc70b00305b5f3.json

Tuya Plug v2 zha-tuyaplugv2-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_w0qqde0g TS011F-c149586c2e9ac7551ebeebf93556977b.json

Tuya Temp & Humidity Sensor Aliexpress zha-tempsensor-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_bguser20 TS0201-67af00943c50fc7fc83ea538b0436f70.json

Tuya Door Contact Aliexpress zha-doorcontact-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_yxqnffam TS0203-47a29ee27ed3f4efd7b4126547d08239.json

[pgroene]

Interesting, I have the same kind of plugs. Is there something that can be done to ignore the messages from these plugs?

[holgrrr]

Hi, I have a couple of mains powered Tuya devices,

Overview: what it is, the NWK (should be in the debug log I guess) and where I ordered it.

Tuya/Aubess Zigbee Plug 0x5569, 0x69de, 0x6456, 0x5e28 https://de.aliexpress.com/item/1005006025696749.html https://de.aliexpress.com/item/1005004910203202.html -> Note: 0x69de, 0x6456 sometimes have connectivity issues due to being far off

Tuya Zigbee Plug 0xe47d https://de.aliexpress.com/item/1005005762698053.html https://de.aliexpress.com/item/1005005734493134.html (got some more waiting to be deployed)

IR Zigbee Remote control x4609 https://de.aliexpress.com/item/1005003878194474.html

Tuya RGB LED GU10 Zigbee - listing eWeLink as manufacturer though Nwk: 0x5eb3 https://de.aliexpress.com/item/1005005729836971.html -> active

Tuya mmwave 0x7d29, 0x487d https://de.aliexpress.com/item/1005006010216719.html -> disabled since weeks, see older comments

Please let me know if you need further details on additional tuya devices, I have a wild bunch of others, but these are not mains powered.

Also, here's the log home-assistant_zha_2024-02-07T20-23-25.048Z.zip

[netsoft-ruidias]

@dmulcahey Updating to version 2014.2.0 (today's release) didn't solve anything, the problem continues to appear (I'm reverting to v2023.12.3 now)

@puddly Did you mean a report for each of the devices? I have many. I'm send the debug report instead. home-assistant_zha_2024-02-07T21-28-31.540Z.log

Please tell me if you need anything else from me, thank you.

[dmulcahey]

@dmulcahey Updating to version 2014.2.0 (today's release) didn't solve anything, the problem continues to appear (I'm reverting to v2023.12.3 now)

@puddly Did you mean a report for each of the devices? I have many. I'm send the debug report instead. home-assistant_zha_2024-02-07T21-28-31.540Z.log

Please tell me if you need anything else from me, thank you.

If you have any of the Tuya devices mentioned above there is no relief in this release. Sorry for any confusion.

[a-d-r-i-a-n-d]

I bravely ventured into the upgrade today, hoping to banish the pesky problem once and for all. Alas, the upgrade was more of a sidegrade, and the problem remained as stubborn as a mule. I decided to test the waters with my bathroom light, flicking it on and off from the dashboard like a disco DJ, but the light was having none of it.

Here's the kicker: I sauntered off for a bath, and about 15 minutes in, my bathroom turned into a surprise rave! The light started flashing on and off like it was party time. My first thought was, "Ah, my wife's discovered her inner prankster!" But no, it was just my earlier commands finally deciding to show up to the party.

Versions up to 2023.12.3 have been as steady as a rock for years. I'm no developer, but I'd wager my last cookie that there's something fishy going on with the messaging queue. The logs are as clean as a whistle, but the commands are taking their sweet time, strolling in minutes later like they've been out for a leisurely walk.

@puddly, unless you've got some secret superpower to see invisible delays and queues, I doubt there's much to glean from the current logs. It's like real life, when the queue at the coffee shop is out the door, it usually means there aren't enough baristas. So, the million-dollar questions are: Did we let go of any baristas since 2023.12.*? Or did we introduce something that's causing a traffic jam in the queues (like debounce isn't working)?

[netsoft-ruidias]

Hey @puddly

I have compiled a list of all the ZigBee devices I have on my network. I took the opportunity to remove some that I was no longer using, and I also removed three of the mmw sensors to see if the network improved (it didn't).

Here is the list: devices.csv I hope it helps you find the culprit

[puddly]

unless you've got some secret superpower to see invisible delays and queues, I doubt there's much to glean from the current logs.

I summarized the root cause of the delays above. I'm trying to compile a list of Tuya devices that are flooding peoples' networks like this and am trying to figure out if there is a generic way to narrow their deluge of packets. The reason for the logs (including ZHA startup) is twofold: I need to see what devices are doing this, and I need to see how they are spamming your network to see if my proposed fix will be enough.

[a-d-r-i-a-n-d]

Thank you, @puddly. Your proposed solution seems promising. I'm hopeful it will work, fingers crossed!

Below is a list of my top 15 devices that generate the most noise, out of a total of 95. This data was collected from a 10-minute debug session. Clicking on the ID will redirect you to the corresponding item on AliExpress.

ID	Message Count	Model1	Model2	Friendly Name	Quirk	Details
0x07A8	7429	TS0601	_TZE204_sxm7l9xa	Bathroom Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant3	5.8G Zigbee Ceiling
0xFC6A	7182	TS0601	_TZE204_qasjif9e	Master Bedroom Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant2	5.8G Zigbee Ceiling
0x1EDC	7174	TS0601	_TZE204_qasjif9e	Hall Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant2	5.8G Zigbee Ceiling
0xDA19	7072	TS0601	_TZE204_qasjif9e	Landing Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant2	5.8G Zigbee Ceiling
0x5289	6985	TS0601	_TZE204_sxm7l9xa	Ensuite Bathroom Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant3	5.8G Zigbee Ceiling
0x61BF	6894	TS0601	_TZE204_qasjif9e	Guest Bedroom Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant2	5.8G Zigbee Ceiling
0x9BBB	6814	TS0601	_TZE204_qasjif9e	David’s Bedroom Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant2	5.8G Zigbee Ceiling
0x75A5	6771	TS0601	_TZE204_qasjif9e	Adrian’s Bedroom Presence	ts0601_motion.TuyaMmwRadarOccupancyVariant2	5.8G Zigbee Ceiling
0x47AA	6133	TS0601	_TZE200_aoclfnxz	Hall Thermostat	zhaquirks.tuya.ts0601_electric_heating.MoesBHT	gas boiler
0x7DF5	5844	TS0601	_TZE200_aoclfnxz	Master Bedroom Thermostat	zhaquirks.tuya.ts0601_electric_heating.MoesBHT	gas boiler
0x4DC9	4786	TS0601	_TZE200_aoclfnxz	Playroom Underfloor Heating	zhaquirks.tuya.ts0601_electric_heating.MoesBHT	16A for underfloor heating
0x5FDE	1589	TS0001	_TZ3000_kqvb5akv	Downstairs Boiler Valve	zhaquirks.tuya.ts000x.Switch_1G_Metering	zigbee with power monitoring
0xE160	1534	TS0001	_TZ3000_kqvb5akv	Kitchen TRV	zhaquirks.tuya.ts000x.Switch_1G_Metering	zigbee with power monitoring
0x35F7	1201	TS011F	_TZ3000_okaz9tjs	Christmas Tree	zhaquirks.tuya.ts011f_plug.Plug_v2	zigbee with power monitoring
0x25A7	951	TS011F	_TZ3000_okaz9tjs	Garage Light	zhaquirks.tuya.ts011f_plug.Plug_v2	zigbee with power monitoring

[puddly]

I'm hopeful it will work, fingers crossed!

Unfortunately, it will not. The Tuya mmWave sensors spam the network at 2 requests per second each and explicitly ask the coordinator to reply to every packet with an ACK. You're using a custom quirk (ts0601_motion) because the device is not supported by ZHA yet. I looked at the traffic from your debug log and unfortunately the device sends unique data in every packet, so there's no way to filter duplicates.

The WIP quirk has to be updated to disable sending a default response: https://github.com/zigpy/zha-device-handlers/pull/2525.

---

Note that network issues caused by custom or WIP quirks are not something that ZHA has control over! You're running untested code to support a device that isn't properly supported yet.

[a-d-r-i-a-n-d]

Thanks for explication, I now understand my situation. As I'm not a developer, I'm unsure whether disabling the default response requires a simple or complex update. If it's a straightforward change, could you please modify the code? This would allow us to use it until the comprehensive quirk is fully developed.

ts0601_motion.py

```python """BlitzWolf IS-3/Tuya motion rechargeable occupancy sensor.""" import math from typing import Dict, Optional, Tuple, Union from zigpy.profiles import zgp, zha from zigpy.quirks import CustomDevice import zigpy.types as t from zigpy.zcl import foundation from zigpy.zcl.clusters.general import ( AnalogInput, AnalogOutput, Basic, GreenPowerProxy, Groups, Identify, Ota, Scenes, Time, ) from zigpy.zcl.clusters.measurement import ( IlluminanceMeasurement, OccupancySensing, RelativeHumidity, TemperatureMeasurement, ) from zigpy.zcl.clusters.security import IasZone from zhaquirks import Bus, LocalDataCluster, MotionOnEvent from zhaquirks.const import ( DEVICE_TYPE, ENDPOINTS, INPUT_CLUSTERS, MODELS_INFO, MOTION_EVENT, OUTPUT_CLUSTERS, PROFILE_ID, ) from zhaquirks.tuya import ( NoManufacturerCluster, TuyaLocalCluster, TuyaManufCluster, TuyaNewManufCluster, ) from zhaquirks.tuya.mcu import ( DPToAttributeMapping, TuyaAttributesCluster, TuyaMCUCluster, ) ZONE_TYPE = 0x0001 class TuyaMmwRadarSelfTest(t.enum8): """Mmw radar self test values.""" TESTING = 0 TEST_SUCCESS = 1 TEST_FAILURE = 2 OTHER = 3 COMM_FAULT = 4 RADAR_FAULT = 5 class TuyaOccupancySensing(OccupancySensing, TuyaLocalCluster): """Tuya local OccupancySensing cluster.""" class TuyaAnalogInput(AnalogInput, TuyaLocalCluster): """Tuya local AnalogInput cluster.""" class TuyaIlluminanceMeasurement(IlluminanceMeasurement, TuyaLocalCluster): """Tuya local IlluminanceMeasurement cluster.""" class TuyaTemperatureMeasurement(TemperatureMeasurement, TuyaLocalCluster): """Tuya local TemperatureMeasurement cluster.""" class TuyaRelativeHumidity(RelativeHumidity, TuyaLocalCluster): """Tuya local RelativeHumidity cluster.""" class TuyaMmwRadarSensitivity(TuyaAttributesCluster, AnalogOutput): """AnalogOutput cluster for sensitivity.""" _CONSTANT_ATTRIBUTES = { AnalogOutput.AttributeDefs.description.id: "sensitivity", AnalogOutput.AttributeDefs.min_present_value.id: 1, AnalogOutput.AttributeDefs.max_present_value.id: 9, AnalogOutput.AttributeDefs.resolution.id: 1, } class TuyaMmwRadarMinRange(TuyaAttributesCluster, AnalogOutput): """AnalogOutput cluster for min range.""" _CONSTANT_ATTRIBUTES = { AnalogOutput.AttributeDefs.description.id: "min_range", AnalogOutput.AttributeDefs.min_present_value.id: 0, AnalogOutput.AttributeDefs.max_present_value.id: 950, AnalogOutput.AttributeDefs.resolution.id: 10, AnalogOutput.AttributeDefs.engineering_units.id: 118, # 31: meters } class TuyaMmwRadarMaxRange(TuyaAttributesCluster, AnalogOutput): """AnalogOutput cluster for max range.""" _CONSTANT_ATTRIBUTES = { AnalogOutput.AttributeDefs.description.id: "max_range", AnalogOutput.AttributeDefs.min_present_value.id: 10, AnalogOutput.AttributeDefs.max_present_value.id: 950, AnalogOutput.AttributeDefs.resolution.id: 10, AnalogOutput.AttributeDefs.engineering_units.id: 118, # 31: meters } class TuyaMmwRadarDetectionDelay(TuyaAttributesCluster, AnalogOutput): """AnalogOutput cluster for detection delay.""" _CONSTANT_ATTRIBUTES = { AnalogOutput.AttributeDefs.description.id: "detection_delay", AnalogOutput.AttributeDefs.min_present_value.id: 000, AnalogOutput.AttributeDefs.max_present_value.id: 20000, AnalogOutput.AttributeDefs.resolution.id: 100, AnalogOutput.AttributeDefs.engineering_units.id: 159, # 73: seconds } class TuyaMmwRadarFadingTime(TuyaAttributesCluster, AnalogOutput): """AnalogOutput cluster for fading time.""" _CONSTANT_ATTRIBUTES = { AnalogOutput.AttributeDefs.description.id: "fading_time", AnalogOutput.AttributeDefs.min_present_value.id: 2000, AnalogOutput.AttributeDefs.max_present_value.id: 200000, AnalogOutput.AttributeDefs.resolution.id: 1000, AnalogOutput.AttributeDefs.engineering_units.id: 159, # 73: seconds } class TuyaMmwRadarTargetDistance(TuyaAttributesCluster, AnalogInput): """AnalogInput cluster for target distance.""" _CONSTANT_ATTRIBUTES = { AnalogOutput.AttributeDefs.description.id: "target_distance", AnalogOutput.AttributeDefs.engineering_units.id: 118, # 31: meters } class NeoBatteryLevel(t.enum8): """NEO battery level enum.""" BATTERY_FULL = 0x00 BATTERY_HIGH = 0x01 BATTERY_MEDIUM = 0x02 BATTERY_LOW = 0x03 USB_POWER = 0x04 class NeoMotionManufCluster(TuyaNewManufCluster): """Neo manufacturer cluster.""" attributes = TuyaNewManufCluster.attributes.copy() attributes.update( { 0xEF0D: ("dp_113", t.enum8, True), # ramdom attribute ID } ) dp_to_attribute: Dict[int, DPToAttributeMapping] = { 101: DPToAttributeMapping( TuyaOccupancySensing.ep_attribute, "occupancy", ), 104: DPToAttributeMapping( TuyaTemperatureMeasurement.ep_attribute, "measured_value", lambda x: x * 10, ), 105: DPToAttributeMapping( TuyaRelativeHumidity.ep_attribute, "measured_value", lambda x: x * 100, ), 113: DPToAttributeMapping( TuyaNewManufCluster.ep_attribute, "dp_113", ), } data_point_handlers = { 101: "_dp_2_attr_update", 104: "_dp_2_attr_update", 105: "_dp_2_attr_update", 113: "_dp_2_attr_update", } class TuyaMmwRadarClusterBase(NoManufacturerCluster, TuyaMCUCluster): """Mmw radar cluster, base class.""" attributes = TuyaMCUCluster.attributes.copy() attributes.update( { # ramdom attribute IDs 0xEF01: ("occupancy", t.uint32_t, True), 0xEF02: ("sensitivity", t.uint32_t, True), 0xEF03: ("min_range", t.uint32_t, True), 0xEF04: ("max_range", t.uint32_t, True), 0xEF06: ("self_test", TuyaMmwRadarSelfTest, True), 0xEF09: ("target_distance", t.uint32_t, True), 0xEF65: ("detection_delay", t.uint32_t, True), 0xEF66: ("fading_time", t.uint32_t, True), 0xEF67: ("cli", t.CharacterString, True), 0xEF68: ("illuminance", t.uint32_t, True), } ) class TuyaMmwRadarClusterV1(TuyaMmwRadarClusterBase): """Mmw radar cluster, variant 1.""" dp_to_attribute: Dict[int, DPToAttributeMapping] = { 1: DPToAttributeMapping( TuyaOccupancySensing.ep_attribute, "occupancy", ), 2: DPToAttributeMapping( TuyaMmwRadarSensitivity.ep_attribute, "present_value", ), 3: DPToAttributeMapping( TuyaMmwRadarMinRange.ep_attribute, "present_value", endpoint_id=2, ), 4: DPToAttributeMapping( TuyaMmwRadarMaxRange.ep_attribute, "present_value", endpoint_id=3, ), 6: DPToAttributeMapping( TuyaMCUCluster.ep_attribute, "self_test", ), 9: DPToAttributeMapping( TuyaMmwRadarTargetDistance.ep_attribute, "present_value", lambda x: x / 100, ), 101: DPToAttributeMapping( TuyaMmwRadarDetectionDelay.ep_attribute, "present_value", converter=lambda x: x * 100, dp_converter=lambda x: x // 100, endpoint_id=4, ), 102: DPToAttributeMapping( TuyaMmwRadarFadingTime.ep_attribute, "present_value", converter=lambda x: x * 100, dp_converter=lambda x: x // 100, endpoint_id=5, ), 103: DPToAttributeMapping( TuyaMCUCluster.ep_attribute, "cli", ), 104: DPToAttributeMapping( TuyaIlluminanceMeasurement.ep_attribute, "measured_value", converter=lambda x: int(math.log10(x) * 10000 + 1) if x > 0 else int(0), ), } data_point_handlers = { 1: "_dp_2_attr_update", 2: "_dp_2_attr_update", 3: "_dp_2_attr_update", 4: "_dp_2_attr_update", 6: "_dp_2_attr_update", 9: "_dp_2_attr_update", 101: "_dp_2_attr_update", 102: "_dp_2_attr_update", 103: "_dp_2_attr_update", 104: "_dp_2_attr_update", } class TuyaMmwRadarClusterV2(TuyaMmwRadarClusterBase): """Mmw radar cluster, variant 2.""" dp_to_attribute: Dict[int, DPToAttributeMapping] = { 1: DPToAttributeMapping( TuyaOccupancySensing.ep_attribute, "occupancy", ), 2: DPToAttributeMapping( TuyaMmwRadarSensitivity.ep_attribute, "present_value", ), 3: DPToAttributeMapping( TuyaMmwRadarMinRange.ep_attribute, "present_value", endpoint_id=2, ), 4: DPToAttributeMapping( TuyaMmwRadarMaxRange.ep_attribute, "present_value", endpoint_id=3, ), 9: DPToAttributeMapping( TuyaMmwRadarTargetDistance.ep_attribute, "present_value", ), 101: DPToAttributeMapping( TuyaMmwRadarDetectionDelay.ep_attribute, "present_value", converter=lambda x: x * 100, dp_converter=lambda x: x // 100, endpoint_id=4, ), 102: DPToAttributeMapping( TuyaMmwRadarFadingTime.ep_attribute, "present_value", converter=lambda x: x * 100, dp_converter=lambda x: x // 100, endpoint_id=5, ), 104: DPToAttributeMapping( TuyaIlluminanceMeasurement.ep_attribute, "measured_value", converter=lambda x: int(math.log10(x) * 10000 + 1) if x > 0 else int(0), ), } data_point_handlers = { 1: "_dp_2_attr_update", 2: "_dp_2_attr_update", 3: "_dp_2_attr_update", 4: "_dp_2_attr_update", 9: "_dp_2_attr_update", 101: "_dp_2_attr_update", 102: "_dp_2_attr_update", 104: "_dp_2_attr_update", } class TuyaMmwRadarClusterV3(TuyaMmwRadarClusterBase): """Tuya MMW radar cluster, variant 3.""" dp_to_attribute: Dict[int, DPToAttributeMapping] = { 103: DPToAttributeMapping( TuyaMCUCluster.ep_attribute, "cli", ), 104: DPToAttributeMapping( TuyaIlluminanceMeasurement.ep_attribute, "measured_value", converter=lambda x: int(math.log10(x) * 10000 + 1) if x > 0 else int(0), ), 105: DPToAttributeMapping( TuyaOccupancySensing.ep_attribute, "occupancy", ), 106: DPToAttributeMapping( TuyaMmwRadarSensitivity.ep_attribute, "present_value", ), 107: DPToAttributeMapping( TuyaMmwRadarMaxRange.ep_attribute, "present_value", endpoint_id=3, ), 108: DPToAttributeMapping( TuyaMmwRadarMinRange.ep_attribute, "present_value", endpoint_id=2, ), 109: DPToAttributeMapping( TuyaMmwRadarTargetDistance.ep_attribute, "present_value", ), 110: DPToAttributeMapping( TuyaMmwRadarFadingTime.ep_attribute, "present_value", converter=lambda x: x * 100, dp_converter=lambda x: x // 100, endpoint_id=5, ), 111: DPToAttributeMapping( TuyaMmwRadarDetectionDelay.ep_attribute, "present_value", converter=lambda x: x * 100, dp_converter=lambda x: x // 100, endpoint_id=4, ), } data_point_handlers = { 103: "_dp_2_attr_update", 104: "_dp_2_attr_update", 105: "_dp_2_attr_update", 106: "_dp_2_attr_update", 107: "_dp_2_attr_update", 108: "_dp_2_attr_update", 109: "_dp_2_attr_update", 110: "_dp_2_attr_update", 111: "_dp_2_attr_update", } class MotionCluster(LocalDataCluster, MotionOnEvent): """Tuya Motion Sensor.""" _CONSTANT_ATTRIBUTES = {ZONE_TYPE: IasZone.ZoneType.Motion_Sensor} reset_s = 15 class TuyaManufacturerClusterMotion(TuyaManufCluster): """Manufacturer Specific Cluster of the Motion device.""" def handle_cluster_request( self, hdr: foundation.ZCLHeader, args: Tuple[TuyaManufCluster.Command], *, dst_addressing: Optional[ Union[t.Addressing.Group, t.Addressing.IEEE, t.Addressing.NWK] ] = None, ) -> None: """Handle cluster request.""" tuya_cmd = args[0] self.debug("handle_cluster_request--> hdr: %s, args: %s", hdr, args) if hdr.command_id == 0x0001 and tuya_cmd.command_id == 1027: self.endpoint.device.motion_bus.listener_event(MOTION_EVENT) class TuyaMotion(CustomDevice): """BW-IS3 occupancy sensor.""" def __init__(self, *args, **kwargs): """Init device.""" self.motion_bus = Bus() super().__init__(*args, **kwargs) signature = { # endpoint=1 profile=260 device_type=0 device_version=0 input_clusters=[0, 3] # output_clusters=[3, 25]> MODELS_INFO: [("_TYST11_i5j6ifxj", "5j6ifxj"), ("_TYST11_7hfcudw5", "hfcudw5")], ENDPOINTS: { 1: { PROFILE_ID: zha.PROFILE_ID, DEVICE_TYPE: zha.DeviceType.ON_OFF_SWITCH, INPUT_CLUSTERS: [Basic.cluster_id, Identify.cluster_id], OUTPUT_CLUSTERS: [Identify.cluster_id, Ota.cluster_id], } }, } replacement = { ENDPOINTS: { 1: { PROFILE_ID: zha.PROFILE_ID, DEVICE_TYPE: zha.DeviceType.OCCUPANCY_SENSOR, INPUT_CLUSTERS: [ Basic.cluster_id, Identify.cluster_id, MotionCluster, TuyaManufacturerClusterMotion, ], OUTPUT_CLUSTERS: [Identify.cluster_id, Ota.cluster_id], } } } class NeoMotion(CustomDevice): """NAS-PD07 occupancy sensor.""" signature = { # endpoint=1 profile=260 device_type=81 device_version=0 input_clusters=[0, 4, 5, 61184] # output_clusters=[10, 25]> MODELS_INFO: [ ("_TZE200_7hfcudw5", "TS0601"), ("_TZE200_ppuj1vem", "TS0601"), ], ENDPOINTS: { 1: { PROFILE_ID: zha.PROFILE_ID, DEVICE_TYPE: zha.DeviceType.SMART_PLUG, INPUT_CLUSTERS: [ Basic.cluster_id, Groups.cluster_id, Scenes.cluster_id, NeoMotionManufCluster.cluster_id, ], OUTPUT_CLUSTERS: [Time.cluster_id, Ota.cluster_id], } }, } replacement = { ENDPOINTS: { 1: { PROFILE_ID: zha.PROFILE_ID, DEVICE_TYPE: zha.DeviceType.OCCUPANCY_SENSOR, INPUT_CLUSTERS: [ Basic.cluster_id, Groups.cluster_id, Scenes.cluster_id, NeoMotionManufCluster, TuyaOccupancySensing, TuyaTemperatureMeasurement, TuyaRelativeHumidity, ], OUTPUT_CLUSTERS: [Time.cluster_id, Ota.cluster_id], } } } class TuyaMmwRadarOccupancyVariant1GPP(CustomDevice): """Millimeter wave occupancy sensor.""" signature = { # endpoint=1, profile=260, device_type=81, device_version=1, # input_clusters=[4, 5, 61184, 0], output_clusters=[25, 10]) MODELS_INFO: [ ("_TZE200_ar0slwnd", "TS0601"), ("_TZE200_sfiy5tfs", "TS0601"), ("_TZE200_mrf6vtua", "TS0601"), ], ENDPOINTS: { 1: { PROFILE_ID: zha.PROFILE_ID, DEVICE_TYPE: zha.DeviceType.SMART_PLUG, INPUT_CLUSTERS: [ Basic.cluster_id, Groups.cluster_id, Scenes.cluster_id, TuyaNewManufCluster.cluster_id, ], OUTPUT_CLUSTERS: [Time.cluster_id, Ota.cluster_id], }, 242: { #

[marc4s]

will there be ota firmware updates with 2024.2 for those tuya devices possible? maybe those behaviour will be fixed with them? could not find much information yet for those tuya firmware updates but due to this thread it could be https://github.com/Koenkk/zigbee2mqtt/issues/10487 https://forum.phoscon.de/t/new-tuya-smart-plug-nas-wr01b-or-tz3000-w0qqde0g/819/104

[puddly]

Tuya firmware updates are dangerous and won't be rolled out in this release because Tuya devices reuse firmware identifiers and will ask for the wrong firmware image. You can brick your device by uploading "compatible" firmware.

[ckbisek]

Tuya firmware updates are dangerous and won't be rolled out in this release because Tuya devices reuse firmware identifiers and will ask for the wrong firmware image. You can brick your device by uploading "compatible" firmware.

Is the issue isolated to Tuya devices in general or specific to the mmWave devices?

[pgroene]

I'm hopeful it will work, fingers crossed!

Unfortunately, it will not. The Tuya mmWave sensors spam the network at 2 requests per second each and explicitly ask the coordinator to reply to every packet with an ACK. You're using a custom quirk (ts0601_motion) because the device is not supported by ZHA yet. I looked at the traffic from your debug log and unfortunately the device sends unique data in every packet, so there's no way to filter duplicates.

The WIP quirk has to be updated to disable sending a default response: zigpy/zha-device-handlers#2525.

Note that network issues caused by custom or WIP quirks are not something that ZHA has control over! You're running untested code to support a device that isn't properly supported yet.

What would be the place to update the quirk? I can do the update with the right directions and test if these are the only devices that are misbehaving in my network.

[a-d-r-i-a-n-d]

@pgroene, your assistance would be greatly appreciated. @puddly, it appears that my Tuya thermostats (_TZE200_aoclfnxz) are flooding the network as much as the mmwave sensors. Since these are supported by ZHA, should I open a separate issue to address this? It also seems like a broader issue with Tuya devices. Would it be more efficient to apply a universal fix to all Tuya devices instead of addressing them individually?

[marc4s]

Tuya firmware updates are dangerous and won't be rolled out in this release because Tuya devices reuse firmware identifiers and will ask for the wrong firmware image. You can brick your device by uploading "compatible" firmware.

Is the issue isolated to Tuya devices in general or specific to the mmWave devices?

it is NOT specific to mmWave devices!

[marc4s]

Tuya firmware updates are dangerous and won't be rolled out in this release because Tuya devices reuse firmware identifiers and will ask for the wrong firmware image. You can brick your device by uploading "compatible" firmware.

OK thanks. does this also apply if i use a tuya bridge and tuya app?

[sankethub]

it is NOT specific to mmWave devices!

Agreed. The following wall switches (manufacturer: Makegood technologies) are also flooding the network causing a lot of delayed commands: _TZE204_gbagoilo _TZE204_nh9m9emk _TZE204_go3tvswy

They all use the _ts0601switch.py custom quirk.

Unfortunately, I am unable to provide logs as I have reverted back to v11.3 for now.

[thegreatdane6]

The root cause of this issue is unfortunately Tuya devices. Some types of Tuya devices (especially plugs/outlets with energy monitoring and mmWave sensors) repeatedly send a lot of traffic on the network and ask that the coordinator reply to every single packet with an acknowledgement.

Well that is very unfortunate since almost all my devices are tuya devices :(

The reason you are seeing this now and not in a previous Core release is that we fixed a bug in ZHA that preventing it from properly replying to these acknowledgements. This is the correct behavior for 99% of devices. Unfortunately, this fix exposed the fundamental problem with these Tuya devices: they continuously send a lot of data, nonstop, and ask for an ACK every single time.

Normally, devices send a report every couple of minutes. Sometimes a few times a minute if they're being actively used. These Tuya devices are sending data often times multiple times per second. If you have 15 of these devices on your network, it will be continuously flooded with data.

Thank you for the insights :)

After 60 seconds, click the "Disable debug logging" button and your browser will download a log file. Please upload that log file instead.

home-assistant_zha_2024-02-14T10-42-14.719Z.log config_entry-zha-1a6692054dbaf124eb2e8ed0bea429ed.json

If possible, could you also post a link to where you bought your devices so that I can look into purchasing a few to test out fixes?

I am not using the mm wave sensor though

https://de.aliexpress.com/item/1005005875872668.html https://de.aliexpress.com/item/1005005941325665.html https://de.aliexpress.com/item/1005005601289812.html https://de.aliexpress.com/item/1005004796727335.html https://de.aliexpress.com/item/1005005686454600.html https://de.aliexpress.com/item/1005004347962095.html https://de.aliexpress.com/item/1005006248206930.html (6n1) https://de.aliexpress.com/item/1005003956219784.html https://de.aliexpress.com/item/1005005197476235.html https://de.aliexpress.com/item/1005004861758253.html https://de.aliexpress.com/item/1005004838591092.html https://de.aliexpress.com/item/1005004838554210.html https://de.aliexpress.com/item/1005005854203557.html https://de.aliexpress.com/item/1005004946095782.html

I appreciate the help everyone has provided with tracking down this issue and we can hopefully have a fix released very soon!

Appreciate your help within this issue!!

[pgroene]

Here are my logs for the first few minutes after startup:

home-assistant_zha_2024-02-14T21-02-17.804Z.log

[frank777777]

Still no solution

[1NFR4R3D]

So I decided to take apart my mmWave sensor because I want to see how hard it can be to fix this problem in firmware. Here are PCB images -

The mmWave module itself seems to be connected via 3 pins, R, T, and O. Pin R on the mmWave goes to Pin 17 (PA9) of the MCU, and Pin T goes to Pin 18 (PA10). Pin O on the module doesn't seem to be connected anywhere.

The controller seems to be a GigaDevice GD32E230F8P6. It has what seems to be a programming port by it. The programming port has Pin 1 going to SWCLK, Pin 2 to SWDIO, Pin 3 to GND, and Pin 4 to MCU VDD (3.3V).

The next stage would be to see if I can dump the code from the MCU, but I'll need to buy a SWD programmer. I'll update here as I go on, but I cannot make any promises! I'm not a seasoned hardware hacker (yet)!

[disruptivepatternmaterial]

The next stage would be to see if I can dump the code from the MCU, but I'll need to buy a SWD programmer. I'll update here as I go on, but I cannot make any promises! I'm not a seasoned hardware hacker (yet)!

So you can remove the CPU and such from that and replace it with ESP32, it is not the easiest thing in the world, but not that hard. The problem is that they are not WiFI only, blakadder has lots of stuff on this.

I'd gladly replace all my tuya mmWave sensors (I have them in every room!) with something else, but there are not that many choices without making your own...

[frank777777]

Sostitute with aqara P1?

[tabrisnet]

Still no solution

Am I to take it then that 2024.2 does not resolve the issue?

[1NFR4R3D]

@tabrisnet The issue (see here) is the Tuya devices being naughty. So while the wonderful folk working on ZHA are trying to do something about it, I believe that it's not going to be a straightforward fix, and will need a bit of testing and tweaking.

[1NFR4R3D]

@disruptivepatternmaterial I did stumble upon blackadder, but I like the idea of my IoT hardware being on a physically separate device (at least until I move out of student accommodations, and am able to deploy my own network with a separate WiFi network for IoT devices). So... My adventures into hardware hacking shall continue!

[tabrisnet]

@tabrisnet The issue (see here) is the Tuya devices being naughty. So while the wonderful folk working on ZHA are trying to do something about it, I believe that it's not going to be a straightforward fix, and will need a bit of testing and tweaking.

I understood from reading this bug that it was various Tuya devices' issue. But this did used to work. and although I've had "move to Z2M" on my list for at least 6 months, I'm not looking forward to re-pairing everything to a new coordinator.

[pgroene]

Does Z2M does not have the same issue if they ack all the request from these devices? Let me know if it works, because I like to be able to update my home assistant to a newer version. In total I have about 30 for these tuya devices

[tabrisnet]

Does Z2M does not have the same issue if they ack all the request from these devices? Let me know if it works, because I like to be able to update my home assistant to a newer version. In total I have about 30 for these tuya devices

a) I do not know b) my plan was to also change my coordinator type to CC2652p7

[netsoft-ruidias]

Status update, for all those who are interested in this subject

1. I understand that this is not a BUG nor is there a simple FIX and therefore there will be no solution soon;
2. I understand that the ZHA team does not feel committed to finding a solution, as we are using devices that are not officially supported (that's why we are using custom zha-quirks);
3. I understand that the justification for this BUG lies in a hardware issue that floods the network with unnecessary status updates, and to which it always requires a response;
4. I also understand that Tuya's business is selling hardware devices and therefore its interest in firmware updates is none;

That being said, at least for me, it is not feasible to be forever stuck with version 2023.12.x... So I solved my problem as follows:

1. I had a spare SkyConnect at home that I used to do some tests and experiments;
2. I had a spare RaspberryPI that I used to do some experiments and lately it was serving as a game console;
3. So I bought a memory card, installed the RaspberryPI OS, Docker and zigbee2mqtt on it.
4. And I started pairing the MMW devices that were flooding the network;
5. I only moved my 12 Human Presence Sensors, everything else I kept in the ZHA;
6. With MQTT already properly setup in Home Assistant, the integration was stress-free;
7. I had to manually update some dashboards and automations, but it was the least of the problems;
8. I deleted the ZHA devices and boom...
9. My network is stable and I was able to upgrade to the latest version.

[dmulcahey]

Status update, for all those who are interested in this subject

1. I understand that this is not a BUG nor is there a simple FIX and therefore there will be no solution soon;
2. I understand that the ZHA team does not feel committed to finding a solution, as we are using devices that are not officially supported (that's why we are using custom zha-quirks);
3. I understand that the justification for this BUG lies in a hardware issue that floods the network with unnecessary status updates, and to which it always requires a response;
4. I also understand that Tuya's business is selling hardware devices and therefore its interest in firmware updates is none;

That being said, at least for me, it is not feasible to be forever stuck with version 2023.12.x... So I solved my problem as follows:

1. I had a spare SkyConnect at home that I used to do some tests and experiments;
2. I had a spare RaspberryPI that I used to do some experiments and lately it was serving as a game console;
3. So I bought a memory card, installed the RaspberryPI OS, Docker and zigbee2mqtt on it.
4. And I started pairing the MMW devices that were flooding the network;
5. I only moved my 12 Human Presence Sensors, everything else I kept in the ZHA;
6. With MQTT already properly setup in Home Assistant, the integration was stress-free;
7. I had to manually update some dashboards and automations, but it was the least of the problems;
8. I deleted the ZHA devices and boom...
9. My network is stable and I was able to upgrade to the latest version.

Item 2 is incorrect. We are still trying to see what can be done to solve this.

[frank777777]

@dmulcahey sir , I’m testing Z2M + ZHA - in same one .. works the Z2M.. but can’t see the entity

I’ve just set a Z2M network . In the raspberry where I also have a ZHA . (2 dongles of course)

Works but

I’ve successful paired a Tuya device , I can see on Z2M that works .. but once I look on entities list .. I can find it .. what can be ??

I of course paired only in one network .. anyway I can see only stuff from ZHA…