Zigbee2mqtt devices becomes unavailable in HA in version 1.27

[redpixy]

What happened?

Zigbee2mqtt devices becomes unavailable in HA after short period (around 15 - 30 mins). zigbee2mqtt/bridge/state show offline.

Restarting the Zigbee2mqtt temporary restores the function but it becomes unavailable shortly after. HA Controls to switches also experience long delays before action is registered.

I have downgrade back to version 1.26. The issues doesn't seem to occur anymore.

What did you expect to happen?

Zigbee2mqtt device remains available.

How to reproduce it (minimal and precise)

Upgrade to version 1.27 from 1.26.

Zigbee2MQTT version

1.27

Adapter firmware version

20220219

Adapter

Sonoff Itead 3.0

Debug log

No response

[Schubser]

I got the same Problem after upgrading from 1.26 to 1.27. Problem is gone after going back to 1.26.

[jb123567]

Same problem here, can any advise how to downgrade??

[Timminater]

2 different systems, 2 different coordinators. Both getting devices offline after several hours. Both systems where fine on 1.26, problems started with 1.27

[redpixy]

Same problem here, can any advise how to downgrade??

if you are using docker, grab the previous image by adding :1.26

[jb123567]

Same problem here, can any advise how to downgrade??

if you are using docker, grab the previous image by adding :1.26

Running as an Add-On inside Home Assistant OS.

Looking further at this, Zigbee2MQTT is still functioning correctly, clearly the connection to HA that is failing.

[Koenkk]

Please provide the debug log of this.

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable debug logging.

[Grandma-Betty]

Same issue here:

[21:21:59] INFO: Handing over control to Zigbee2MQTT Core ...
> zigbee2mqtt@1.27.0 start
> node index.js
Zigbee2MQTT:info  2022-08-03 21:22:06: Logging to console and directory: '/config/zigbee2mqtt/log/2022-08-03.21-22-06' filename: log.txt
Zigbee2MQTT:info  2022-08-03 21:22:06: Starting Zigbee2MQTT version 1.27.0 (commit #unknown)
Zigbee2MQTT:info  2022-08-03 21:22:06: Starting zigbee-herdsman (0.14.46)
Zigbee2MQTT:error 2022-08-03 21:22:07: Error while starting zigbee-herdsman
Zigbee2MQTT:error 2022-08-03 21:22:07: Failed to start zigbee
Zigbee2MQTT:error 2022-08-03 21:22:07: Check https://www.zigbee2mqtt.io/guide/installation/20_zigbee2mqtt-fails-to-start.html for possible solutions
Zigbee2MQTT:error 2022-08-03 21:22:07: Exiting...
Zigbee2MQTT:error 2022-08-03 21:22:07: Error: Error while opening serialport 'Error: Error: No such file or directory, cannot open /dev/ttyUSB0'
    at SerialPort.<anonymous> (/app/node_modules/zigbee-herdsman/src/adapter/z-stack/znp/znp.ts:146:28)
    at SerialPort._error (/app/node_modules/@serialport/stream/lib/index.js:198:14)
    at /app/node_modules/@serialport/stream/lib/index.js:242:12

I also have the "Bad gateway" thing going on as described here: https://github.com/zigbee2mqtt/hassio-zigbee2mqtt/issues/366

EDIT: Holy crap, the Zigbee USB Gateway was somehow disconnected from the HAOS VM in my case. Shut down the VM and passing through the Zigbee USB Gateway to the VM again solved the issue in my case.

[b-reich]

Mhmm I have the same problem. Providing logs tomorrow.

The devices gets unavailable for me when turning them on. (Bulbs) Even the MQTT shows they are on.

[Minoux78]

Hello, Same issue for me, all devices become unavalaible under ha even if they are functional under zigbee2mqtt

[b-reich]

Nut sure if this a HomeAssistant Problem

[redpixy]

Not sure if its HA or Z2M but I did notice, If you restart HA while on Z2M 1.27, all the Z2M devices would be shown as unavailable. When execute zigbee2mqtt/bridge/state in MQTT it will show up as offline despite Z2M is running. Having said that, I had tried rolling HA version between 2022.7.7 to 2022.7.4 all encountered the same issue when running with Z2M 1.27.

The other issue is the long delays before an action is registered. example turning On a light or smartplugs.

[Minoux78]

There was no issue with 1.26, it's clearly a Z2M 1.27 problem

[Koenkk]

@Grandma-Betty this is clearly an issue on your side. The adapter port is incorrect: Zigbee2MQTT:error 2022-08-03 21:22:07: Error: Error while opening serialport 'Error: Error: No such file or directory, cannot open /dev/ttyUSB0'

@ all others, please provide the debug log, without it I cannot do anything.

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable debug logging.

[b-reich]

@Koenkk here is my log file. log.txt

One lamp is working. 2 lamps in group doenst work.

[Timminater]

Already had 3 home-assistant installations with this problem. But after a complete reboot and linking the dongle to the VM again the problem has not returned. If the problem returns I will try to grab a debug log @Koenkk

[jb123567]

I host reboot seemed to have sorted my issue

[Koenkk]

@b-reich

• which lamps aren't working?
• does it keep working when disabling the availability feature?

[b-reich]

@Koenkk "Nachttisch Lampen" wont work as a group (turning them off works, on wont work) "Nachttisch Links" and "Nachttisch Links" can be switched individually without problems

Disabling availability doesnt fix it.

[JunghaJungha]

@Koenkk

I also loose connection between HA and Zigbee2MQTT every day. So I turned on the debug and I believe around 7:03 this morning it happened again. Here is my log log - geen ww.txt

[Timminater]

Happening to me right now, slowly a lot of devices are losing connection.

changed log settings to debug and this is the log file: z2m_log.txt

[Koenkk]

@Timminater in you case it seems that there is some issue on the coordinator side. What coordinator/firmware are you using?

warn  2022-08-05 07:44:18: Failed to ping 'Portable_Airco' (attempt 2/2, Read 0xa4c1386cc4469f1b/1 genBasic(["zclVersion"], {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (Data request failed with error: 'MAC no resources' (26)))

@Koendejongh in your case it appears that the MQTT broker is down

error 2022-08-05 07:17:43: Not connected to MQTT server!

@b-reich so if I understand correctly, the bulbs Nachttisch Links and Nachttisch Links are in group called Nachttisch Lampen. When turning the group OFF the bulbs respond, when turning ON nothing happens. If yes, can you try to repair those bulbs and re-add them to the group?

[Minoux78]

@ Koenkk Hello, I loose again connection between HA and Zigbee2MQTT this night (it happens every day). This happens since 1.27 update. It seems to be around 04:18 If I restart Zigbee2MQTT, devices become available again. Here is the log file: log.txt

And screenshot from mosquitto broker log :

Thanks in advance for your help.

[navycrow]

Same here 1.27.0-1 release doesn't work at all. It seems there is an issue with mqtt server.

[rotilho]

Similar issue. After upgrade to 1.27 zigbee2mqtt stopped posting events to mqtt after several hours running.

I couldn't find anything in the logs, I'll keep an eye and share it as soon as it happen again.

[Timminater]

@Timminater in you case it seems that there is some issue on the coordinator side. What coordinator/firmware are you using?

@Koenkk I'm using a cc2538+cc2592 with firmware version 20201010

System worked flawless for months, problems started with update 1.27. Changing my own server to a sonoff zb3 dongle next week to see if that solves anything.

A reboot of z2m won't solve issues for me, a complete reset of the dongle (or remove/insert) is needed to solve the connection

[Koenkk]

@Minoux78 thanks for the logging, it seems that z2m indeed disconnects from the broker. I've added some additional logging in the dev branch. Can you update to the latest edge addon (https://www.zigbee2mqtt.io/advanced/more/switch-to-dev-branch.html) and provide me the herdsman debug logging when z2m disconnects from the broker?

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

[Minoux78]

@Koenkk thanks for your feedback. I installed the latest edge addon at around 21:50 (french time). Devices disconnect at around 21:58 Here the debug logging : log.txt

[fairwood136]

@Koenkk I was having similar issues. When I recompiled, I had ignored the Nodejs version mismatch error. Once I updated my version from v7.x to v16.6, my issues went away. Is there a validation check that the correct version of Node is being run?

[Koenkk]

@Minoux78 I'm missing the logging, not that this is only logged to STDOUT and not to the log files. You need to execute the docker logs command to get the logs, see https://www.zigbee2mqtt.io/guide/usage/debug.html#home-assistant-os-supervised-addon

@fairwood136 without logs I cannot determine what the root cause is.

[Minoux78]

@koenkk I'm directly under os ha. I tried to to grab all the logs via SSH & terminal add on by executing "docker ps" to determine the CONTAINER_ID first but I have the following message "Bash: docker: command not found"... Not sure it's the good process.

[Koenkk]

@Minoux78 I think you need to follow https://developers.home-assistant.io/docs/operating-system/debugging/ first

[jeroen85]

Possibly the same issue happens at my system (Synology NAS with Zigbee2MQTT, Mqtt and HA dockers).

It started last night after mqtt-docker was updated (via watchtower). Entities became unavailable in HA. A Zigbee2MQTT restart was required to fix the problem.

Can easily reproduce the problem by a mqtt restart. Will try to get debug logs tomorrow when I am behind a laptop :-)

[jeroen85]

Ok, spent a bit more time on this.

When I restart my MQTT docker I get the following debug logs: logs.txt

All MQTT messages continue to be delivered to the MQTT broker (checked with MQTT explorer). Also visible in HA (MQTT info of entities). But the entities stay unavailable in HA.

It appears that the the bridge status goes to offline when losing MQTT connection. After the connection is restored, is stays offline.

                {
                  "payload": "{\"state\":\"offline\"}",
                  "qos": 0,
                  "retain": 0,
                  "time": "2022-08-11T06:22:08.203418+00:00",
                  "topic": "zigbee2mqtt/bridge/state"
                },
                {
                  "payload": "{\"state\":\"offline\"}",
                  "qos": 0,
                  "retain": 1,
                  "time": "2022-08-11T06:22:17.788701+00:00",
                  "topic": "zigbee2mqtt/bridge/state"
                }

When I manually (via MQTT explorer) push a {"state":"online"} to zigbee2mqtt/bridge/state all entities become available again.

[Gurgeh88]

Same problem here. Everything seems fine in Zigbee2MQTT but in Mosquitto broker, everything is reported as unavailable so remains unavailable in HA. EDIT: I think this may be more of an MQTT broker issue than a Zigbee2MQTT issue. Other things I'm using that don't rely on Zigbee2MQTT but do run through my MQTT broker are also unavailable. Have tried restarting HA and host system to no effect.

[Koenkk]

@jjansen85 pushed a possible fix, can you check if it works? Z2M should now publish online 30 seconds after HA comes online.

Changes will be available in the dev branch in a few hours from now. (https://www.zigbee2mqtt.io/advanced/more/switch-to-dev-branch.html)

[jeroen85]

@Koenkk This works for my described issue. When restarting MQTT, the online status goed offline. After about 30 seconds, it turns back to online.

[Koenkk]

@jjansen85 great, I found a better fix. Can you check if it works? Should immediately go to online now instead of after 30 seconds.

Changes will be available in the dev branch in 1 hour from now. (https://www.zigbee2mqtt.io/advanced/more/switch-to-dev-branch.html)

[jeroen85]

@Koenkk Perfect! Works much faster.

[b-reich]

@Koenkk after I did nothing except update HA from time to time. Now everything works again for me.

[navycrow]

Do you plan to fix the stable branch ?

[Koenkk]

@navycrow yes, the next release is on 1 September

@redpixy can you confirm the issue has been fixed now? Then this can be closed.

[dml33]

I have the same issue. My sonoff ZBMINI and SNZB-02 devices have become Unavailable in HA.

Zigbee2MQTT as addon with version 1.27.0-1 Home Assistant 2022.8.4 Supervisor 2022.08.3 Operating System 8.4 Frontend 20220802.0

In case this issue is fixed, could it be pushed to the stable branch before 1st of September? 15 days waiting for the fix to be propagated while the Zigbee devices are useless seems quite a lot.

Thanks!

[Grandma-Betty]

15 days waiting for the fix to be propagated while the Zigbee devices are useless seems quite a lot.

Indeed.

[jeroen85]

As a temp. 'solution', you can create a template sensor for the bridge status in combination with an automation which check bridge status and performs a restart if required.

Template sensor:

mqtt:
  sensor:
    # Sensor for monitoring the bridge state
    - name: Zigbee2MQTT Bridge state
      unique_id: zigbee2mqtt_bridge_state_sensor
      state_topic: "zigbee2mqtt/bridge/state"
      value_template: "{{ value_json.state }}"
      icon: mdi:router-wireless

Automation

alias: Zigbee2MQTT - Status check + restart
description: ""
trigger:
  - platform: state
    entity_id:
      - sensor.zigbee2mqtt_bridge_state
    from: online
    to: offline
    for:
      hours: 0
      minutes: 0
      seconds: 10
condition: []
action:
  - service: mqtt.publish
    data:
      topic: zigbee2mqtt/bridge/request/restart
  - service: notify.persistent_notification
    data:
      message: Zigbee2MQTT offline, trying restart.
  - delay:
      hours: 0
      minutes: 5
      seconds: 0
      milliseconds: 0
mode: single

[Creat]

Another, much easier, temporary solution is to just roll back to 1.26. If you're using the home assistant plugin, turn off auto update for Z2M (if it's even enabled), go to your backups (Settings -> System -> Backups) and first make a backup of the current state, then restore the last Z2M 1.26 backup. This obviously assumes you haven't made any drastic changes since that backup was taken.

[Koenkk]

@dml33 > I have the same issue. My sonoff ZBMINI and SNZB-02 devices have become Unavailable in HA.

This sound more like range issues or issues with the device itself. If you are affected by this issue all devices will show offline, not a few.

If some more people confirm the dev branches fixes the issue I'm willing to create a hotfix release.

[dml33]

@dml33 > I have the same issue. My sonoff ZBMINI and SNZB-02 devices have become Unavailable in HA.

This sound more like range issues or issues with the device itself. If you are affected by this issue all devices will show offline, not a few.

Sorry. At this moment, I only have those Zigbee devices and both of them do not work. I was specific with the brand and model just in case it helped.

I had Zigbee2MQTT version 1.27.0-1 installed on 2nd of August and since then, I have these issues. Before, they were working properly.

If some more people confirm the dev branches fixes the issue I'm willing to create a hotfix release.

Thanks! 👍

[wixoff]

I'm experiencing this problem on 1.27.0 with a pair of AduroSmart 81813 bulbs - they appear to go offline after some fairly short period of time, but can still be operated through the Z2M front end (which causes them to come back online).

I'm testing latest-dev right now; it's my evening now so we will see how it goes tomorrow.

[wixoff]

Ah, it didn't work. Same symptoms while running 1.27.0-dev commit 040d2a1.

I need to increase my logging (I'm currently on "warn" and I don't see any messages) and I will check in with results later.

[rotilho]

@wixoff that sounds an unrelated issue. This thread is about the whole Zigbee network being unavailable to home assistant not a specific device.