All devices are lost after Z2M update

[paavokurkela]

What happened?

After add-on update on Home Assistant, all the devices are lost from Z2M and Homa Assistant (no devices listed under MQTT instance)

Restoring a backup restores devices on Home Assistant but not on Z2M. Same issue happened to me also with earlier update, unfortunately I don't have any details on that. Log is literally empty.

What did you expect to happen?

No response

How to reproduce it (minimal and precise)

Update Z2M

Zigbee2MQTT version

1.38.0-1

Adapter firmware version

20230507

Adapter

Slaesh

Setup

Add-on under HA on Intel Nuc

Debug log

No response

[kligithu]

The same here

What did you expect to happen? No response

How to reproduce it (minimal and precise) Update Z2M

Zigbee2MQTT version 1.38.0-1 to 1.39.0-1

Adapter firmware version core v2.3.6 : Zigbee 20240315

Adapter SMLIGHT SLZB-06

Setup Add-on under HA on Synology VMM

Debug log No response

[raffoul]

Same. Rollback done.

[Webmonsteril]

Have same issue rolled back also

[Najihel]

Hello,

Pretty same issue here, devices are not missing but all entities are duplicated. The original one is broken and Z2M create duplicate suffixed by _2

[jmcollin78]

Same here ! Full rollback running. What is strange is that the duplication of entities occurs after a full restart of HA and not just after the update to 1.39.0.

This is a severe issue and 1.39.0 deployment should be aborted if you want my opinion.

Edit: the full restor have restored the entities.

[Najihel]

Same here ! Full rollback running. What is strange is that the duplication of entities occurs after a full restart of HA and not just after the update to 1.39.0.

This is a severe issue and 1.39.0 deployment should be aborted if you want my opinion.

Edit: the full restor have restored the entities.

I confirm the trigger is the restart of HA on my side too.

[gijsh070]

Same issue here, double sensors and entities. They even remain double after restoring.....

also all devices are offline. Can only control when directly go to zigbeetomqtt

[countrr]

Same here, had to roll back to 1.38.0. Will not update again until this is solved.

[Zver5]

Same here, had to roll back to 1.38.0. Will not update again until this is solved.

[raphmur]

Had same issue updating today to 1.39.0. Rolling back to 1.38.0 What kind of log would you need? How to support? I am running HAOS on RPi 4B

[kefrens]

I have same issue; How do you rollback zigbee2mqtt?

[raphmur]

@kefrens in home assistant settings> system> backups Then select the backup that was normally done at z2m update and backup. Then reboot and you should be good

[Vidjer]

Hello have same issue. rollback 1.38 ok

[paavokurkela]

In my case rollback to 1.38 did not restore devices to Z2M.

[raphmur]

@paavokurkela , did you restart HA?

[jmcollin78]

In my case rollback to 1.38 did not restore devices to Z2M.

I have to do a full restore and not only a rollback on Z2M 1.38

[gijsh070]

In my case rollback to 1.38 did not restore devices to Z2M.

I have to do a full restore and not only a rollback on Z2M 1.38

Same here, full restore needed

[paavokurkela]

@raphmur Yes I did. I also tried full restore from backup. That restored entities on HA but Z2M devices did not restore.

[Koenkk]

Could you:

• Provide your z2m configuration.yaml and if applicable, screenshot of the Z2M addon config page
• The debug log when starting z2m 1.39.0
• The version of HA that you are using

See this on how to enable debug logging.

[paavokurkela]

@Koenkk thank you for looking on this.

As I restored my system I currently have 1.38 installed. I was able to reproduce the problem by pairing two Ikea lights and upgrade from 1.38 to 1.39.

Please note that this very same problem existed already earlier. I believe it was from 1.37 to 1.38 and it was first time this problem happened to me.

Please find infos requested below:

Couple devices paired to 1.38

Devices gone 1.39

HA version:

configuration.yaml (with debugs enabled)

data_path: /homeassistant/zigbee2mqtt/
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: "-d -d"
  log: true
mqtt:
  server: mqtt://core-mosquitto:1883
  user: ****
  password: ****
  client_id: Z2M
serial:
  port: /dev/ttyUSB0
zigbee_herdsman_debug: false
homeassistant: true
advanced:
  channel: 14
  log_level: debug

Add-on config screen (credentials hidden)

log (1.38 with two Ikea lights paired)

[12:40:58] INFO: Preparing to start...
[12:40:58] INFO: Socat not enabled
[12:40:59] INFO: Starting Zigbee2MQTT...
[2024-07-07 12:41:02] info:     z2m: Logging to console, file (filename: log.log)
[2024-07-07 12:41:02] info:     z2m: Starting Zigbee2MQTT version 1.38.0 (commit #unknown)
[2024-07-07 12:41:02] info:     z2m: Starting zigbee-herdsman (0.49.2)
[2024-07-07 12:41:02] info:     zh:zstack:znp: Opening SerialPort with {"path":"/dev/ttyUSB0","baudRate":115200,"rtscts":false,"autoOpen":false}
[2024-07-07 12:41:02] info:     zh:zstack:znp: Serialport opened
[2024-07-07 12:41:02] info:     zh:zstack:znp: Writing CC2530/CC2531 skip bootloader payload
[2024-07-07 12:41:03] info:     zh:zstack:znp: Skip bootloader for CC2652/CC1352
[2024-07-07 12:41:27] info:     zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-07 12:41:27] info:     z2m: zigbee-herdsman started (reset)
[2024-07-07 12:41:27] info:     z2m: Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20230507,"transportrev":2},"type":"zStack3x0"}'
[2024-07-07 12:41:27] info:     z2m: Currently 0 devices are joined:
[2024-07-07 12:41:27] info:     z2m: Zigbee: disabling joining new devices.
[2024-07-07 12:41:27] info:     z2m:mqtt: Connecting to MQTT server at mqtt://core-mosquitto:1883
[2024-07-07 12:41:27] info:     z2m:mqtt: Connected to MQTT server
[2024-07-07 12:41:27] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 12:41:28] info:     z2m: Started frontend on port 8099
[2024-07-07 12:41:28] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 12:41:28] info:     z2m: Zigbee2MQTT started!
[2024-07-07 12:41:33] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/connection_state/config', payload '{"device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.38.0"},"device_class":"connectivity","entity_category":"diagnostic","name":"Connection state","object_id":"zigbee2mqtt_bridge_connection_state","origin":{"name":"Zigbee2MQTT","sw":"1.38.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"offline","payload_on":"online","state_topic":"zigbee2mqtt/bridge/state","unique_id":"bridge_0x00124b00259bb8b4_connection_state_zigbee2mqtt","value_template":"{{ value_json.state }}"}'

log (startup during upgrade process to 1.39 and all devices lost)

[13:05:07] INFO: Preparing to start...
[13:05:08] INFO: Socat not enabled
[13:05:09] INFO: Starting Zigbee2MQTT...
Starting Zigbee2MQTT without watchdog.
[2024-07-07 13:05:11] info:     z2m: Logging to console, file (filename: log.log)
[2024-07-07 13:05:11] info:     z2m: Starting Zigbee2MQTT version 1.39.0 (commit #unknown)
[2024-07-07 13:05:11] info:     z2m: Starting zigbee-herdsman (0.50.1)
[2024-07-07 13:05:12] info:     zh:zstack:znp: Opening SerialPort with {"path":"/dev/ttyUSB0","baudRate":115200,"rtscts":false,"autoOpen":false}
[2024-07-07 13:05:12] info:     zh:zstack:znp: Serialport opened
[2024-07-07 13:05:12] info:     zh:zstack:znp: Writing CC2530/CC2531 skip bootloader payload
[2024-07-07 13:05:13] info:     zh:zstack:znp: Skip bootloader for CC2652/CC1352
[2024-07-07 13:05:37] info:     zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-07 13:05:37] info:     z2m: zigbee-herdsman started (reset)
[2024-07-07 13:05:37] info:     z2m: Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20230507,"transportrev":2},"type":"zStack3x0"}'
[2024-07-07 13:05:37] info:     z2m: Currently 0 devices are joined:
[2024-07-07 13:05:37] info:     z2m: Zigbee: disabling joining new devices.
[2024-07-07 13:05:37] info:     z2m: Connecting to MQTT server at mqtt://core-mosquitto:1883
[2024-07-07 13:05:37] info:     z2m: Connected to MQTT server
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 13:05:37] info:     z2m: Started frontend on port 8099
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 13:05:37] info:     z2m: Zigbee2MQTT started!
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x90395efffee6d2b7/update_available/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x287681fffe9cb162/update_available/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/effect/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/effect/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/color_options/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/linkquality/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/update_state/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/color_options/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/linkquality/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/update_state/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x90395efffee6d2b7/light/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x287681fffe9cb162/light/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x90395efffee6d2b7/update/config', payload 'null'
[2024-07-07 13:05:37] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x287681fffe9cb162/update/config', payload 'null'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/connection_state/config', payload '{"device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"connectivity","entity_category":"diagnostic","name":"Connection state","object_id":"zigbee2mqtt_bridge_connection_state","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"offline","payload_on":"online","state_topic":"zigbee2mqtt/bridge/state","unique_id":"bridge_0x00124b00259bb8b4_connection_state_zigbee2mqtt","value_template":"{{ value_json.state }}"}'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/restart_required/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"problem","enabled_by_default":false,"entity_category":"diagnostic","name":"Restart required","object_id":"zigbee2mqtt_bridge_restart_required","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_off":false,"payload_on":true,"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_restart_required_zigbee2mqtt","value_template":"{{ value_json.restart_required }}"}'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/button/1221051039810110150109113116116_0x00124b00259bb8b4/restart/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","command_topic":"zigbee2mqtt/bridge/request/restart","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"restart","name":"Restart","object_id":"zigbee2mqtt_bridge_restart","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_press":"","unique_id":"bridge_0x00124b00259bb8b4_restart_zigbee2mqtt"}'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/1221051039810110150109113116116_0x00124b00259bb8b4/log_level/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","command_template":"{\"options\": {\"advanced\": {\"log_level\": \"{{ value }}\" } } }","command_topic":"zigbee2mqtt/bridge/request/options","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"entity_category":"config","name":"Log level","object_id":"zigbee2mqtt_bridge_log_level","options":["error","warning","info","debug"],"origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_log_level_zigbee2mqtt","value_template":"{{ value_json.log_level | lower }}"}'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/version/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"entity_category":"diagnostic","icon":"mdi:zigbee","name":"Version","object_id":"zigbee2mqtt_bridge_version","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_version_zigbee2mqtt","value_template":"{{ value_json.version }}"}'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/coordinator_version/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"enabled_by_default":false,"entity_category":"diagnostic","icon":"mdi:chip","name":"Coordinator version","object_id":"zigbee2mqtt_bridge_coordinator_version","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_coordinator_version_zigbee2mqtt","value_template":"{{ value_json.coordinator.meta.revision }}"}'
[2024-07-07 13:05:42] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/network_map/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"enabled_by_default":false,"entity_category":"diagnostic","json_attributes_template":"{{ value_json.data.value | tojson }}","json_attributes_topic":"zigbee2mqtt/bridge/response/networkmap","name":"Network map","object_id":"zigbee2mqtt_bridge_network_map","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/response/networkmap","unique_id":"bridge_0x00124b00259bb8b4_network_map_zigbee2mqtt","value_template":"{{ now().strftime('%Y-%m-%d %H:%M:%S') }}"}'
[2024-07-07 13:05:43] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/permit_join_timeout/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"duration","entity_category":"diagnostic","name":"Permit join timeout","object_id":"zigbee2mqtt_bridge_permit_join_timeout","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_permit_join_timeout_zigbee2mqtt","unit_of_measurement":"s","value_template":"{{ iif(value_json.permit_join_timeout is defined, value_json.permit_join_timeout, None) }}"}'
[2024-07-07 13:05:43] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/switch/1221051039810110150109113116116_0x00124b00259bb8b4/permit_join/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","command_topic":"zigbee2mqtt/bridge/request/permit_join","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"icon":"mdi:human-greeting-proximity","name":"Permit join","object_id":"zigbee2mqtt_bridge_permit_join","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"false","payload_on":"true","state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_permit_join_zigbee2mqtt","value_template":"{{ value_json.permit_join | lower }}"}'

[dcyou]

hi guys I had same issue 😞 too but I manage myself by taking time to remove the old entities (without _2) then renaming all the entities by removing the "_2" (+/- 10 devices with 4 entity each) 💪 But I struggle with how to remove the zigbee2mqtt_bridge entities, if someone how it is possible? 🆘

[jmcollin78]

Could you:

• Provide your z2m configuration.yaml and if applicable, screenshot of the Z2M addon config page
• The debug log when starting z2m 1.39.0
• The version of HA that you are using

See this on how to enable debug logging.

Hello @Koenkk ,

Is there any chance to have a fix in the 1.39.1 release (https://github.com/Koenkk/zigbee2mqtt/pull/23221) ? It is not listed in the release note.

[Koenkk]

@paavokurkela in the 1.38.0 log I see [2024-07-07 12:41:27] info: z2m: Currently 0 devices are joined:. Could you provide the debug log of:

• Start z2m 1.38.0
• Pair 2 devices
• Restart z2m 1.38.0
• Upgrade to z2m 1.39.0

See this on how to enable debug logging.

[paavokurkela]

@Koenkk thank you for your request.

While gathering logs you requested I found out that devices are gone every time I just simply restart Z2M, no even update needed. Please see log I was able to catch for you below:

Something I want to point out is that while I tried to store log.txt under /homeassistant/zigbee2mqtt/log/%TIMESTAMP% nothing was stored there. Also as log below says: [2024-07-10 08:41:39] info: zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json' the file exists but it's timestamp is for Jan 14th which I believe is date I have installed the system at first place.

Hope this leads us to right direction.

[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,2,0,18]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [12,0,0,0] - 91
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":12}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SREQ: --> SYS - nvRead - {"sysid":1,"itemid":7,"subid":2,"offset":0,"len":12}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,8,33,51,1,7,0,2,0,0,0,12,18]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,14,97,51,0,12,0,0,0,0,46,65,18,57,193,108,55,244,122]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,14,97,51,0,12,0,0,0,0,46,65,18,57,193,108,55,244,122]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 14 - 3 - 1 - 51 - [0,12,0,0,0,0,46,65,18,57,193,108,55,244] - 122
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvRead - {"status":0,"len":12,"value":{"type":"Buffer","data":[0,0,0,0,46,65,18,57,193,108,55,244]}}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SREQ: --> SYS - nvLength - {"sysid":1,"itemid":7,"subid":3}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,3,0,19]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [12,0,0,0] - 91
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":12}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SREQ: --> SYS - nvRead - {"sysid":1,"itemid":7,"subid":3,"offset":0,"len":12}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,8,33,51,1,7,0,3,0,0,0,12,19]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,14,97,51,0,12,0,0,0,0,248,166,248,169,17,156,175,132,249]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,14,97,51,0,12,0,0,0,0,248,166,248,169,17,156,175,132,249]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 14 - 3 - 1 - 51 - [0,12,0,0,0,0,248,166,248,169,17,156,175,132] - 249
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvRead - {"status":0,"len":12,"value":{"type":"Buffer","data":[0,0,0,0,248,166,248,169,17,156,175,132]}}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SREQ: --> SYS - nvLength - {"sysid":1,"itemid":7,"subid":4}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,4,0,20]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [12,0,0,0] - 91
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":12}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SREQ: --> SYS - nvRead - {"sysid":1,"itemid":7,"subid":4,"offset":0,"len":12}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,8,33,51,1,7,0,4,0,0,0,12,20]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,14,97,51,0,12,158,144,20,0,255,255,255,255,255,255,255,255,74]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,14,97,51,0,12,158,144,20,0,255,255,255,255,255,255,255,255,74]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 14 - 3 - 1 - 51 - [0,12,158,144,20,0,255,255,255,255,255,255,255,255] - 74
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvRead - {"status":0,"len":12,"value":{"type":"Buffer","data":[158,144,20,0,255,255,255,255,255,255,255,255]}}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SREQ: --> SYS - nvLength - {"sysid":1,"itemid":7,"subid":5}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,5,0,21]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: <-- [254,4,97,50,0,0,0,0,87]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext [254,4,97,50,0,0,0,0,87]
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [0,0,0,0] - 87
[2024-07-10 08:41:39] debug:    zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":0}
[2024-07-10 08:41:39] debug:    zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug:    zh:zstack:backup: fetched adapter network security material table (capacity=5, used=5)
[2024-07-10 08:41:39] debug:    zh:zstack:backup: Following devices with link key are missing from new backup but present in old backup and database, adding them back: 
[2024-07-10 08:41:39] info:     zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-10 08:41:39] info:     zh:zstack:znp: closing
[2024-07-10 08:41:39] info:     zh:zstack:znp: Port closed
[2024-07-10 08:41:39] info:     z2m: Stopped zigbee-herdsman
[2024-07-10 08:41:39] info:     z2m: Stopped Zigbee2MQTT
[08:41:40] INFO: Preparing to start...
[08:41:40] INFO: Socat not enabled
[08:41:41] INFO: Starting Zigbee2MQTT...
[2024-07-10 08:41:43] info:     z2m: Logging to console, file (filename: log.log)
[2024-07-10 08:41:43] info:     z2m: Starting Zigbee2MQTT version 1.38.0 (commit #unknown)
[2024-07-10 08:41:43] info:     z2m: Starting zigbee-herdsman (0.49.2)
[2024-07-10 08:41:44] info:     zh:zstack:znp: Opening SerialPort with {"path":"/dev/ttyUSB0","baudRate":115200,"rtscts":false,"autoOpen":false}
[2024-07-10 08:41:44] info:     zh:zstack:znp: Serialport opened
[2024-07-10 08:41:44] info:     zh:zstack:znp: Writing CC2530/CC2531 skip bootloader payload
[2024-07-10 08:41:45] info:     zh:zstack:znp: Skip bootloader for CC2652/CC1352
[2024-07-10 08:42:09] info:     zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-10 08:42:09] info:     z2m: zigbee-herdsman started (reset)
[2024-07-10 08:42:09] info:     z2m: Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20230507,"transportrev":2},"type":"zStack3x0"}'
[2024-07-10 08:42:09] info:     z2m: Currently 0 devices are joined:
[2024-07-10 08:42:09] info:     z2m: Zigbee: disabling joining new devices.
[2024-07-10 08:42:09] info:     z2m:mqtt: Connecting to MQTT server at mqtt://core-mosquitto:1883
[2024-07-10 08:42:09] info:     z2m:mqtt: Connected to MQTT server
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-10 08:42:09] info:     z2m: Started frontend on port 8099
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-10 08:42:09] info:     z2m: Zigbee2MQTT started!
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x90395efffee6d2b7/update_available/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x287681fffe9cb162/update_available/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/effect/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/effect/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/color_options/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/linkquality/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/update_state/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/color_options/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/linkquality/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/update_state/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x90395efffee6d2b7/light/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x287681fffe9cb162/light/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x90395efffee6d2b7/update/config', payload 'null'
[2024-07-10 08:42:09] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x287681fffe9cb162/update/config', payload 'null'
[2024-07-10 08:42:14] info:     z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/connection_state/config', payload '{"device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.38.0"},"device_class":"connectivity","entity_category":"diagnostic","name":"Connection state","object_id":"zigbee2mqtt_bridge_connection_state","origin":{"name":"Zigbee2MQTT","sw":"1.38.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"offline","payload_on":"online","state_topic":"zigbee2mqtt/bridge/state","unique_id":"bridge_0x00124b00259bb8b4_connection_state_zigbee2mqtt","value_template":"{{ value_json.state }}"}'
[2024-07-10 08:43:02] info:     z2m: Successfully changed options
[2024-07-10 08:43:02] info:     z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/response/options', payload '{"data":{"restart_required":false},"status":"ok","transaction":"fsygo-1"}'

[lecourtb]

I noticed that in Z2M MQTT parameters, the base topic which was "zigbee2mqtt" before upgrade was renamed to "zigbee2mqttHA" after upgrade.

I reproduced two times the problem (thanks to ha backups) and give up try to upgrade !

Later, I also tried a third upgrade positioning base_topic: zigbee2mqtt in configuration.yaml but without success.

Related to #23294

Hope it will help

[Koenkk]

@lecourtb probably you have set base_topic: zigbee2mqttHA in the HA addon configuration page, this should be set correctly (zigbee2mqtt in your case)

[Aseg]

I have the same issue, I cant update to 1.39 without break my devices and automations because devices duplicate with suffix '_2'

[lecourtb]

That was effectively the case (base_topic: zigbee2mqttHA set in the HA addon configuration page) I will try to upgrade Z2M as soon as possible after the change.

[dystechnic]

Same issue here. Restoring a backup didn't help. I ended up deleting and renaming devices. I'm afraid to reboot HA...

[dermotos]

Has anyone with this issue upgraded to 1.39.1 yet? If so did this issue persist? I don't see anything in the release notes that suggests this issue has been fixed yet.

[thomasknuchel]

Tested and issue is still persisting

[Wendigogo]

You should take a look at #23294 . I'm pretty sure the given solution will work here.

Enjoy!

[Najihel]

You should take a look at #23294 . I'm pretty sure the given solution will work here.

Enjoy!

Yes it is ! Thanks

[raffoul]

Rename base topic to zigbee2mqtt works for me. Thank you.

[paavokurkela]

I got my system back to working state.

I deleted z2m completely as well all files and folders. I reinstalled z2m (1.39.1) and used my old configuration. Z2M did not create any folders or files, not even zigbee2mqtt which I created then manually. On my old config I had/homeassistant/zigbee2mqtt which I changed to /config/zigbee2mqtt which seemed to make a trick and Z2M begun to write files. This is of course very odd as /config is simply a symlink to /homeassistant folder.

This or that my system works again.

[jmcollin78]

Renaming base_topic to zigbee2mqtt and installing yhe 1.19.1 resolves the issue for me.

[Koenkk]

Removing the base_topic from the Z2M HA addon should fix this indeed.

[kefrens]

For me doesn't work as well. In the addon config I have: base_topic: core-mosquitto

In the config of Zigbee2MQTT I have as well: base_topic: core-mosquitto

But it still creates all entities with a '_2'

[kefrens]

For me doesn't work as well. In the addon config I have: base_topic: core-mosquitto

In the config of Zigbee2MQTT I have as well: base_topic: core-mosquitto

But it still creates all entities with a '_2'

Responding to myself.

1. I had to delete in the the addon configuration.yaml the base topic.
2. make sure base topic in the addon and on the zigbee2mqtt website is the same
3. reboot the system (stop/start wasn't enough)

[gijsh070]

Thanks a lot @kefrens

This solved my issue finally. They need to match the addon base_topic and mqtt site.

[gijsh070]

Anyone noticing that triggers somehow are named differently after the latest update? I get a lot of uknown triggers suddenly after updating.