1.29.0 causing lots of issues - Devices just dropping off - devices unresponsive in HA. Its a mess atm

[cloudbr34k84]

log.txt

What happened?

9 devices just dropped off the network. Nothing has changed in my setup except the new upgrade of z2m.

What did you expect to happen?

No response

How to reproduce it (minimal and precise)

No response

Zigbee2MQTT version

1.29.0 commit: unknown

Adapter firmware version

20220219

Adapter

Sonoff Coordinator P

Debug log

2023-01-06 11:13:12Failed to configure 'Upstairs Lounge Quad Board', attempt 1 (Error: Bind 0x5c0272fffe03cc99/1 genOnOff from '0x00124b00256c503a/1' failed (SRSP - ZDO - bindReq after 6000ms) at Timeout._onTimeout (/app/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35) at listOnTimeout (node:internal/timers:559:17) at processTimers (node:internal/timers:502:7))
2023-01-06 11:13:42Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
2023-01-06 11:14:00Publish 'set' 'state' to 'Master Bedroom Right Bedside Lamp' failed: 'Error: Command 0x00124b0023a54359/1 genOnOff.on({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
2023-01-06 11:14:06Publish 'set' 'state' to 'Gin & Tea Lounge Pendant Light Switch' failed: 'Error: Command 0x50325ffffea6af07/1 genOnOff.on({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
2023-01-06 11:14:18Publish 'set' 'state' to 'Master Bedroom Right Bedside Lamp' failed: 'Error: Command 0x00124b0023a54359/1 genOnOff.off({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
2023-01-06 11:14:24Publish 'set' 'state' to 'Gin & Tea Lounge Pendant Light Switch' failed: 'Error: Command 0x50325ffffea6af07/1 genOnOff.on({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
2023-01-06 11:14:36Publish 'get' 'state' to 'Master Bedroom Right Bedside Lamp' failed: 'Error: Read 0x00124b0023a54359/1 genOnOff(["onOff"], {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
2023-01-06 11:14:42Publish 'set' 'state' to 'Gin & Tea Lounge Pendant Light Switch' failed: 'Error: Command 0x50325ffffea6af07/1 genOnOff.on({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'

[cloudbr34k84]

these is what my devices are showing

Error 2023-01-06 12:58:26Failed to read state of 'Void Lounge Quad Board' after reconnect (Read 0x70b3d52b60016c72/1 genOnOff(["onOff"], {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms))
Warning 2023-01-06 12:58:32Failed to ping 'Fireplace Plug' (attempt 1/1, Read 0x086bd7fffe5aee40/1 genBasic(["zclVersion"], {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":true,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms))
Info 2023-01-06 12:58:34MQTT publish: topic 'zigbee2mqtt/bridge/response/device/configure', payload '{"data":{"id":"Bedroom 3 Climate Sensor"},"error":"Device 'Bedroom 3 Climate Sensor' cannot be configured","status":"error","transaction":"b9goe-1"}'
Error 2023-01-06 12:58:44Publish 'set' 'state' to 'Linen Closet Light Switch' failed: 'Error: Command 0xb4e3f9fffe09067e/1 genOnOff.on({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
Info 2023-01-06 12:58:44MQTT publish: topic 'zigbee2mqtt/bridge/response/device/configure', payload '{"data":{"id":"Bedroom 3 Climate Sensor"},"error":"Device 'Bedroom 3 Climate Sensor' cannot be configured","status":"error","transaction":"b9goe-2"}'
Error 2023-01-06 12:58:50Publish 'set' 'state' to 'Office Light Switch' failed: 'Error: Command 0xcc86ecfffea0bd57/1 genOnOff.off({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms)'
Warning 2023-01-06 12:58:56Failed to ping 'Back Patio Light Switch' (attempt 1/1, Read 0x0c4314fffe74b312/1 genBasic(["zclVersion"], {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":true,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (SRSP - AF - dataRequest after 6000ms))

[cloudbr34k84]

What is this

Info 2023-01-06 13:02:00Zigbee: allowing new devices to join.
Error 2023-01-06 13:02:06Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SRSP - ZDO - mgmtPermitJoinReq after 6000ms'
Info 2023-01-06 13:02:06MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SRSP - ZDO - mgmtPermitJoinReq after 6000ms","status":"error","transaction":"b9goe-5"}'

[cloudbr34k84]

i know something is definatly wrong when my climate sensors have not been seen as they are usually reporting every second

[Koenkk]

Does going back to 1.28.4 fix the issues?

[cloudbr34k84]

No it doesn't, as I couldn't even get the Web UI to load. I had to upgrade back to the latest version. Even right now, devices are not responding. No idea 🤔

[Koenkk]

This seems very similar to https://github.com/Koenkk/zigbee2mqtt/issues/15856, I'll continue investigating there.

[cloudbr34k84]

I have heaps of lights On, and they are not showing up. When I press permit join it doesn't do anything? No timer, just a notification.

I don't know what's going on lol

[cloudbr34k84]

@Koenkk has there been an discoveries. Im now unable to use Z2M, as it causes my HA to restart, slow down, pages become unresponsive. As soon as i remove the stick, everything smooths out and I can use HA again, just not Z2m..

[Koenkk]

@cloudbr34k84 we are getting closer, I will update in https://github.com/Koenkk/zigbee2mqtt/issues/15856

[cloudbr34k84]

@Koenkk that's great but i think i stuffed by coordinator up by hitting the reset button instead if Boot button (Sonoff P)

debug 10-01-2023 07:08:45: Loaded state from file /config/zigbee2mqtt/state.json
info  10-01-2023 07:08:45: Logging to console and directory: '/config/zigbee2mqtt/log/2023-01-10.07-08-45' filename: log.txt
debug 10-01-2023 07:08:45: Removing old log directory '/config/zigbee2mqtt/log/2023-01-09.19-56-42'
info  10-01-2023 07:08:45: Starting Zigbee2MQTT version 1.28.4 (commit #unknown)
info  10-01-2023 07:08:45: Starting zigbee-herdsman (0.14.76)
debug 10-01-2023 07:08:45: Using zigbee-herdsman with settings: '{"adapter":{"concurrent":null,"delay":null,"disableLED":false},"backupPath":"/config/zigbee2mqtt/coordinator_backup.json","databaseBackupPath":"/config/zigbee2mqtt/database.db.backup","databasePath":"/config/zigbee2mqtt/database.db","network":{"channelList":[25],"extendedPanID":[221,221,221,221,221,221,221,221],"networkKey":"HIDDEN","panID":6754},"serialPort":{"path":"/dev/ttyUSB0"}}'
error 10-01-2023 07:09:05: Error while starting zigbee-herdsman
error 10-01-2023 07:09:05: Failed to start zigbee
error 10-01-2023 07:09:05: Check https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html for possible solutions
error 10-01-2023 07:09:06: Exiting...
error 10-01-2023 07:09:06: Error: Failed to connect to the adapter (Error: SRSP - SYS - ping after 6000ms)
    at ZStackAdapter.start (/app/node_modules/zigbee-herdsman/src/adapter/z-stack/adapter/zStackAdapter.ts:103:27)
    at Controller.start (/app/node_modules/zigbee-herdsman/src/controller/controller.ts:132:29)
    at Zigbee.start (/app/lib/zigbee.ts:58:27)
    at Controller.start (/app/lib/controller.ts:101:27)
    at start (/app/index.js:107:5)

[Koenkk]

Error: Failed to connect to the adapter (Error: SRSP - SYS - ping after 6000ms) means z2m cannot connect to the coordinator. This cannot be fixed from z2m side, check: https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html#error-srsp-sys-ping-after-6000ms

[cloudbr34k84]

Yeah I looked there doesnt help much tbh

I think I'm.stuffed, and need a new coordinator

[etlweather]

This information may be relevant or it could be disrelated, just providing it in case it may help.

I originally subscribed to follow this issue because I started having something similar - I didn't see any errors in logs but suddently, Z2M would stop showing me any new output in the logs (and no messages were going to MQTT) - I have an air quality monitoring device that sends like 3 messages every 2-3 seconds so it's pretty obvious. This would occur sometimes after 24 hours, sometimes after a few hours. There was no precise recurrence.

I also had one Aeotec button that would drop off the network and come back every once in a while with something about the interview being incomplete.

At the time, I was using ZBDongle-E. I thought my problems were due to ZBDongle-E - after all it's marked as not fully supported.

I replaced it with a ZBDongle-P and it has been running for 2 days without hicup. There are a few minor difference - I have most of the devices joinned to this new network but a few of my Aeotec buttons aren't currently there. I doubt that these buttons were causing the hang up.

The ZBDongle-P is running firmware 20210708.

I am running the Docker image 1.29.0 on a i386 system (Ubuntu) under k3s (kubernetes).

I also set up a paralel system for testing in a different location with a ZBDongle-P, running Z2M directly on the host (Ubuntu on a Raspeberry PI 4). This is running the latest firmware 20220219 and Z2M 1.29.1 commit 7d67ffc. It has only been about 24 hours but I didn't have an issue with it.

[RubenKelevra]

@cloudbr34k84 did 1.29.2 fix your issue? :)

[cloudbr34k84]

@cloudbr34k84 did 1.29.2 fix your issue? :)

Hard to say, I decided to start again with HA lol

[pavkamlc]

I've the same problem on [1.29.2] commit: [bb3e8f6d] with TuYa WHD02 on Sonoff EZSP v8