Devices unavailable and unable to re-pair after upgrade

[gaggio]

Bug Report

What happened

After upgrade to 1.14.1, some devices have started to appear and disappear. The main issue was that an OSRAM Smart Plug turned unavailable, and many devices were routed through it. Upgraded to edge, some other devices got back up, with their messages routed through other devices (bulbs). However, many still unavalable. Then updated to 1.15.0, got even worse. Most of the devices are unavailable. I'm not able to pair again the OSRAM Smart Plug which is quite an important piece in my network. Tried to remove it, force remove it. I can't seem to re-pair it even very close to the coordinator.

What did you expect to happen

1. first, I did not expect an update to screw things up. I have 25 devices and none were recently added, so it should still be manageable by the CC2531 coordinator (as it was before upgrade)
2. I expect to be able to remove and re-pair my devices to fix the network

How to reproduce it (minimal and precise)

I'm not sure there's a way to reproduce it.

Debug Info

Zigbee2MQTT version: 1.15.0 Adapter hardware: CC2531 Adapter firmware version: 20190608

[Koenkk]

Can you try switching to the source routing firmware and see if things improve? https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_Home_1.2/bin/source_routing

[skandragon]

I am using the firmware Coordinator firmware version: '{"meta":{"maintrel":3,"majorrel":2,"minorrel":6,"product":0,"revision":20190619,"transportrev":2},"type":"zStack12"}' and also cannot re-pair some devices.

I have been able to re-pair:

• 7 Sengled bulbs of various types (out of about 15 that I had working)
• 1 Sylvania smart-plug (out of 4)
• 2 Bosch motion sensors (out of 4)
• 0 SmartThings plugs (out of 2)
• 0 magnetic switches (out of 8, various types)
• 2 SmartThings water leak sensors (out of 3) etc.

Even with debugging on, I see nothing in the logs showing there is even an attempt made. I do have joining hard-coded on from the config file.

I'm using the Docker image: Starting Zigbee2MQTT version 1.15.0 (commit #1bccc5d)

[Davincible]

Same thing happened to me. My whole setup went cold turkey after the update. The update reset the config to default, and had to put in my creds + network key. But my devices are still unavailable

[VanQz]

Im having a similar issue. Unable to pair more devices. Logs does not say if device is being interviewed as well. After reflashing cc2531 with source routing version im able to add 1 more device total 6 now and unable to add more devices. Any solutions?

[ThePirat91]

Same problem here, no more pairing mode on 1.15.0.

Oct 03 13:22:32 homeassistant npm[25054]: Zigbee2MQTT:info  2020-10-03 13:22:32: Zigbee: allowing new devices to join.
Oct 03 13:22:32 homeassistant npm[25054]: Zigbee2MQTT:error 2020-10-03 13:22:32: Request 'zigbee2mqtt/bridge/request/permit_join' failed with error: 'SREQ '--> AF - dataRequestExt - {"dstaddrmode":2,"dstaddr":"0x000000000000fffd","destendpoint":242,"dstpanid":0,"srcendpoint":242,"clusterid":33,"transid":2,"options":0,"radius":30,"len":6,"data":{"type":"Buffer","data":[25,3,2,11,254,0]}}' failed with status '(0x10: MEM_ERROR)' (expected '(0x00: SUCCESS)')'
Oct 03 13:22:32 homeassistant npm[25054]: Zigbee2MQTT:info  2020-10-03 13:22:32: MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{},"error":"SREQ '--> AF - dataRequestExt - {\"dstaddrmode\":2,\"dstaddr\":\"0x000000000000fffd\",\"destendpoint\":242,\"dstpanid\":0,\"srcendpoint\":242,\"clusterid\":33,\"transid\":2,\"options\":0,\"radius\":30,\"len\":6,\"data\":{\"type\":\"Buffer\",\"data\":[25,3,2,11,254,0]}}' failed with status '(0x10: MEM_ERROR)' (expected '(0x00: SUCCESS)')","status":"error"}'

[vlfesko]

Similar issue for me, with Xiaomi Smart Power Plug ZNCZ02LM, they just stopped to work and did not re-pair, nothing in logs, no attempt to re-pair in logs.

Then, firmware upgrade of CC2531 on this URL https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_Home_1.2/bin/source_routing solved the problem, two sockets in a row re-paired instantly.

Additionally, try to move your CC2531 stick away from Pi (use USB extension cable), some interference may be the case of issues as well.

[VanQz]

had to spam reflash of coordinator and spam pairing then it works. quite a hassle

[skandragon]

@vlfesko I am using that firmware already...

[Koenkk]

@skandragon try completely powering off the devices which you did not pair yet, those could still form a second zigbee network and repair.

@Davincible what is your setup? (hassio/bare metal/docker)?

[skandragon]

@Koenkk All the routers are turned off other than the one that has joined. The network didn't actually change, but for some reason after upgrading to 1.15.0 all the devices were shown in Home Assistant as gone, and only re-appeared if they did join. Some did on their own, but most did not.

[ThePirat91]

Similar issue for me, with Xiaomi Smart Power Plug ZNCZ02LM, they just stopped to work and did not re-pair, nothing in logs, no attempt to re-pair in logs.

Then, firmware upgrade of CC2531 on this URL https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_Home_1.2/bin/source_routing solved the problem, two sockets in a row re-paired instantly.

Additionally, try to move your CC2531 stick away from Pi (use USB extension cable), some interference may be the case of issues as well.

Yea, that's OK, but I am on 15-device network with no router, only a cc2531 coordinator.... So no source-routing fw for me....

[Davincible]

@Koenkk Just Hassio on a pi

[skandragon]

Also, while I did remove all the routers but the one that is connected and the associated device works, the other devices I am attempting to join are not showing up as joined. They continue to show an unjoined warning.

So far I'm up to 8 different device types across 4 different manufacturers, where some do and some do not pair. Some older ones will not re-join even when reset, and some new devices will not, but some older and some newer will. It seems to be random.

[skandragon]

I stopped the docker container, removed all the devices from the config, deleted the state and the database, changed the network key and network PAN id by one, and restarted the container. The devices which did not properly join started joining again. I've gone through all the devices I can reach easily and reset them, and all joined.

It is possible I had a rogue router (that I don't know where it actually is) somewhere, but I don't think things should have gotten into this state to start with.

[ThePirat91]

Similar issue for me, with Xiaomi Smart Power Plug ZNCZ02LM, they just stopped to work and did not re-pair, nothing in logs, no attempt to re-pair in logs. Then, firmware upgrade of CC2531 on this URL https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_Home_1.2/bin/source_routing solved the problem, two sockets in a row re-paired instantly. Additionally, try to move your CC2531 stick away from Pi (use USB extension cable), some interference may be the case of issues as well.

Yea, that's OK, but I am on 15-device network with no router, only a cc2531 coordinator.... So no source-routing fw for me....

Confirmed. With the "source-routing 20200619" fw version, my CC2531 started to permit joying again. I am still wondering about the possible causes..... Thanks guys

[Davincible]

@Koenkk any updates on this? My system is still offline

[Koenkk]

@Davincible I'm not sure what is the cause, but I have seen a few more reports for hassio users about this. Not sure what caused this yet and doesn't seem to repeat. In your case I would therefore recommend to repair the devices.

[Davincible]

@Koenkk I tried to repair an ikea bulb, which failed for some reason. Resulting log:

Zigbee2MQTT:warn  2020-10-19 00:27:23: Device '0x14b457fffe7efe8d' left the network
Zigbee2MQTT:info  2020-10-19 00:27:23: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"ieee_address":"0x14b457fffe7efe8d"},"type":"device_leave"}'
Zigbee2MQTT:info  2020-10-19 00:27:23: MQTT publish: topic 'zigbee2mqtt/bridge/devices', payload '[{"definition":null,"endpoints":{"1":{"bindings":[],"clusters":{"input":[],"output":[]}},"11":{"bindings":[],"clusters":{"input":["ssIasAce"],"output":["ssIasZone","ssIasWd"]}},"110":{"bindings":[],"clusters":{"input":[],"output":[]}},"12":{"bindings":[],"clusters":{"input":[],"output":[]}},"13":{"bindings":[],"clusters":{"input":["genOta"],"output":[]}},"2":{"bindings":[],"clusters":{"input":[],"output":[]}},"242":{"bindings":[],"clusters":{"input":[],"output":[]}},"3":{"bindings":[],"clusters":{"input":[],"output":[]}},"4":{"bindings":[],"clusters":{"input":[],"output":[]}},"47":{"bindings":[],"clusters":{"input":[],"output":[]}},"5":{"bindings":[],"clusters":{"input":[],"output":[]}},"6":{"bindings":[],"clusters":{"input":[],"output":[]}},"8":{"bindings":[],"clusters":{"input":[],"output":[]}}},"friendly_name":"Coordinator","ieee_address":"0x00124b00193e541e","interview_completed":true,"interviewing":false,"network_address":0,"supported":false,"type":"Coordinator"}]'
Zigbee2MQTT:info  2020-10-19 00:27:23: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"left_network","meta":{"friendly_name":"0x14b457fffe7efe8d"},"type":"device_removed"}'
Zigbee2MQTT:info  2020-10-19 00:28:38: Device '0x14b457fffe7efe8d' joined
Zigbee2MQTT:info  2020-10-19 00:28:38: Starting interview of '0x14b457fffe7efe8d'
Zigbee2MQTT:info  2020-10-19 00:28:38: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0x14b457fffe7efe8d","ieee_address":"0x14b457fffe7efe8d"},"type":"device_joined"}'
Zigbee2MQTT:info  2020-10-19 00:28:38: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0x14b457fffe7efe8d","ieee_address":"0x14b457fffe7efe8d","status":"started"},"type":"device_interview"}'
Zigbee2MQTT:info  2020-10-19 00:28:38: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":{"friendly_name":"0x14b457fffe7efe8d"},"type":"device_connected"}'
Zigbee2MQTT:info  2020-10-19 00:28:38: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"interview_started","meta":{"friendly_name":"0x14b457fffe7efe8d"},"type":"pairing"}'
Zigbee2MQTT:error 2020-10-19 00:30:56: Failed to interview '0x14b457fffe7efe8d', device has not successfully been paired
Zigbee2MQTT:info  2020-10-19 00:30:56: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0x14b457fffe7efe8d","ieee_address":"0x14b457fffe7efe8d","status":"failed"},"type":"device_interview"}'
Zigbee2MQTT:info  2020-10-19 00:30:56: MQTT publish: topic 'zigbee2mqtt/bridge/devices', payload '[{"definition":null,"endpoints":{"1":{"bindings":[],"clusters":{"input":[],"output":[]}},"11":{"bindings":[],"clusters":{"input":["ssIasAce"],"output":["ssIasZone","ssIasWd"]}},"110":{"bindings":[],"clusters":{"input":[],"output":[]}},"12":{"bindings":[],"clusters":{"input":[],"output":[]}},"13":{"bindings":[],"clusters":{"input":["genOta"],"output":[]}},"2":{"bindings":[],"clusters":{"input":[],"output":[]}},"242":{"bindings":[],"clusters":{"input":[],"output":[]}},"3":{"bindings":[],"clusters":{"input":[],"output":[]}},"4":{"bindings":[],"clusters":{"input":[],"output":[]}},"47":{"bindings":[],"clusters":{"input":[],"output":[]}},"5":{"bindings":[],"clusters":{"input":[],"output":[]}},"6":{"bindings":[],"clusters":{"input":[],"output":[]}},"8":{"bindings":[],"clusters":{"input":[],"output":[]}}},"friendly_name":"Coordinator","ieee_address":"0x00124b00193e541e","interview_completed":true,"interviewing":false,"network_address":0,"supported":false,"type":"Coordinator"},{"definition":null,"endpoints":{},"friendly_name":"0x14b457fffe7efe8d","ieee_address":"0x14b457fffe7efe8d","interview_completed":false,"interviewing":false,"network_address":27303,"supported":false}]'
Zigbee2MQTT:info  2020-10-19 00:30:56: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"interview_failed","meta":{"friendly_name":"0x14b457fffe7efe8d"},"type":"pairing"}'

Old config before it reset itself:

{
  "data_path": "/share/zigbee2mqtt",
  "devices": "devices.yaml",
  "groups": "groups.yaml",
  "homeassistant": true,
  "permit_join": true,
  "mqtt": {
    "base_topic": "zigbee2mqtt",
    "server": "mqtt://core-mosquitto",
    "user": "...",
    "password": "..."
  },
  "serial": {
    "port": "/dev/ttyACM0",
    "disable_led": true
  },
  "advanced": {
    "pan_id": 6754,
    "channel": 11,
    "network_key": [
      114,
    ],
    "availability_blacklist": []
  },
  "ban": [],
  "whitelist": [],
  "queue": {},
  "socat": {
    "enabled": false,
    "master": "pty,raw,echo=0,link=/dev/ttyZ2M,mode=777",
    "slave": "tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5",
    "restartdelay": 1.0,
    "initialdelay": 1.0,
    "options": "-d -d",
    "log": false
  }
}

New config:

data_path: /share/zigbee2mqtt
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: true
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://core-mosquitto'
  user: ...
  password: ...
serial:
  port: /dev/ttyACM0
  disable_led: true
advanced:
  log_level: info
  pan_id: 6754
  channel: 11
  network_key:
    - 114
  availability_blocklist: []
  availability_passlist: []
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental:
  new_api: true
socat:
  enabled: false
  master: 'pty,raw,echo=0,link=/dev/ttyZ2M,mode=777'
  slave: 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5'
  restartdelay: 1
  initialdelay: 1
  options: '-d -d'
  log: false

[Koenkk]

Make sure your Zigbee network is fully offline first before repairing, https://www.zigbee2mqtt.io/information/FAQ.html#why-does-my-device-not-or-fail-to-pair

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days