Migrating working deconz installation with Conbee II to a new PC

airbone42 commented 3 years ago

Describe the question or issue you are having

In the beginning I had set up my smart home using an old Windows laptop. Up to now already the Zigbee network using deconz and a Conbee II stick has 65 nodes. Therefore, my plan is to move this to a little bit more professional setup and wanted to migrate now deconz into a docker container on a Raspberry Pi.

Unfortunately, I'm unable to move that whole network, without reinitialising all 65 nodes to that new system. I found several short tutorials mentioning to copy the zll.db or using the backup/restore option in Phoscon, or keeping the same Conbee stick, but that didn't really work. As it seems this issue is very much deconz-related and not specific to the docker image marthoc/docker-deconz I'm writing here.

By doing an export and import using the Phoscon web interface it was quite easy to get a read-access to the Zigbee network. Seconds later I could already see all devices in Phoscon as well as the deconz GUI, even when using a new Conbee II stick. Also the states of the nodes have been updated (and communication is indicating by blinking dots in the deconz GUI). But it is not possible to actively change the state of a node from the Phoscon interface, e.g. lights do not turn on/off and the old installation running in parallel still has the full control over the network. I have already activated debug-info, but there's no specific message when I use the Phoscon interface of the new installation to trigger a node state.

The whole behavior doesn't change even when I copy over the zll.db or shutdown the old deconz-installation and even when I switch to the old Conbee II stick (as described here https://flemmingss.com/how-to-backup-and-restore-deconz-from-a-defective-installation/).

Any hint how to get this migration done would be appreciated. Ideally by running both deconz installations in parallel with full control (so I could in the future even integrate some kind of fallback). But for now I would already be happy if I can shut off the old Windows machine and use the network as-is on the new hardware (doesn't matter which Conbee II stick). Thanks in advance!

Screenshots

n/a

Environment

Old Windows installation - Conbee II: 2.07.01 / 8.12.2020 26680700

New Raspberry Pi (Raspbian) installation - Conbee II: 2.10.04 / 14.3.2021 26580700

deCONZ Logs

15:23:12:787 HTTP Server listen on address 0.0.0.0, port: 8080, root: /usr/share/deCONZ/webapp/ 15:23:12:816 CTRL. 3.27.215:23:12:911 dev /dev/ttyAMA0 15:23:12:911 ZCLDB init file /root/.local/share/dresden-elektronik/deCONZ/zcldb.txt 15:23:13:068 parent process /bin/sh 15:23:13:069 gw run mode: docker 15:23:13:069 GW sd-card image version file does not exist: /root/.local/share/dresden-elektronik/deCONZ/gw-version 15:23:13:069 sd-card cid: 275048534431364760ffffffff013b00 15:23:13:071 DB sqlite version 3.27.2 15:23:13:072 DB PRAGMA page_count: 59 15:23:13:072 DB PRAGMA page_size: 4096 15:23:13:073 DB PRAGMA freelist_count: 3 15:23:13:073 DB file size 241664 bytes, free pages 3 15:23:13:073 DB PRAGMA user_version: 7 15:23:13:073 DB cleanup 15:23:13:074 DB create temporary views 13:23:13:244 started websocket server at port 8443 13:23:13:247 [INFO] - Found file containing button maps. Parsing data... 13:23:13:250 [WARNING] - Button map 'sunricherCCTMap' in JSON file has no assigned ModelIDs. Skip loading button map... 13:23:13:258 [INFO] - Button maps loaded. 13:23:13:260 dlg action: Read binding table 13:23:13:260 found node plugin: libde_rest_plugin.so - REST API Plugin 13:23:13:263 found node plugin: libde_signal_plugin.so - Signal Monitor Plugin 13:23:13:277 found node plugin: libstd_otau_plugin.so - STD OTAU Plugin 13:23:13:490 LightNode 1: Configuration tool 1 added 13:23:13:506 SensorNode 7 set node 0x000d6f000f83f89e 13:23:13:508 SensorNode 6 set node 0x000d6f000f83f89e 13:23:13:508 Init Poll Control for 0x000d6f000f83f89e 13:23:13:517 LightNode 2: Bad added 13:23:13:520 LightNode 3: Flur 1. OG added 13:23:13:523 LightNode 5: Büro Panel 1 added 13:23:13:525 LightNode 6: Büro Panel 2 added 13:23:13:528 LightNode 7: Büro Panel 3 added 13:23:13:530 SensorNode 12 set node 0x14b457fffe2ec925 13:23:13:531 SensorNode 13 set node 0x14b457fffe2ec925 13:23:13:536 LightNode 8: Treppe 1. OG/DG added 13:23:13:538 LightNode 9: Treppe EG/1. OG added 13:23:13:541 LightNode 10: Treppe Keller/EG added 13:23:13:543 LightNode 11: Keller added 13:23:13:545 SensorNode 20 set node 0x000d6f000fbfa739 13:23:13:545 Init Poll Control for 0x000d6f000fbfa739 13:23:13:546 SensorNode 21 set node 0x000d6f000fbfa739 13:23:13:552 SensorNode 23 set node 0x000d6f000fbf7838 13:23:13:553 SensorNode 22 set node 0x000d6f000fbf7838 13:23:13:553 Init Poll Control for 0x000d6f000fbf7838 13:23:13:559 SensorNode 25 set node 0x000d6f000f842487 13:23:13:560 SensorNode 24 set node 0x000d6f000f842487 13:23:13:560 Init Poll Control for 0x000d6f000f842487 13:23:13:566 SensorNode 27 set node 0x000d6f000fbf969f 13:23:13:567 SensorNode 26 set node 0x000d6f000fbf969f 13:23:13:567 Init Poll Control for 0x000d6f000fbf969f 13:23:13:573 SensorNode 38 set node 0x00158d00047b6d9b 13:23:13:574 SensorNode 39 set node 0x00158d00047b6d9b 13:23:13:579 LightNode 13: Küche added 13:23:13:580 SensorNode 48 set node 0x00158d00047b98f5 13:23:13:582 SensorNode 49 set node 0x00158d00047b98f5 13:23:13:585 SensorNode 50 set node 0x00158d00047b9a85 13:23:13:587 SensorNode 51 set node 0x00158d00047b9a85 13:23:13:592 LightNode 15: Schrank rechts added 13:23:13:595 LightNode 16: Schrank links added 13:23:13:598 LightNode 17: Extended color light 17 added 13:23:13:601 LightNode 18: Extended color light 18 added 13:23:13:603 LightNode 19: Extended color light 19 added 13:23:13:607 LightNode 20: Ava added 13:23:13:609 SensorNode 43 set node 0x00158d00047ba538 13:23:13:610 SensorNode 46 set node 0x00158d00047ba538 13:23:13:614 SensorNode 47 set node 0x00158d00045cf2c4 13:23:13:617 SensorNode 53 set node 0x001788010801bae0 13:23:13:618 SensorNode 54 set node 0x001788010808716f 13:23:13:623 LightNode 21: Gästeklo großes Licht added 13:23:13:626 LightNode 22: Esszimmer added 13:23:13:629 LightNode 23: Flur EG added 13:23:13:631 LightNode 4: Dimmable light 4 added 13:23:13:633 SensorNode 31 set node 0x00158d00052233e7 13:23:13:636 SensorNode 32 set node 0x00158d0005422ef0 13:23:13:639 SensorNode 33 set node 0x00158d000522509f 13:23:13:642 SensorNode 34 set node 0x00158d00052250a1 QObject::connect: Cannot queue arguments of type 'QTextBlock' (Make sure 'QTextBlock' is registered using qRegisterMetaType().) QObject::connect: Cannot queue arguments of type 'QTextCursor' (Make sure 'QTextCursor' is registered using qRegisterMetaType().) 13:23:13:645 SensorNode 40 set node 0x00158d0005225be8 13:23:13:649 SensorNode 41 set node 0x00158d0005225a54 13:23:13:652 SensorNode 55 set node 0x00158d00052296b4 13:23:13:655 SensorNode 56 set node 0x00158d0005422655 13:23:13:658 SensorNode 60 set node 0x00158d0005447ba1 13:23:13:659 SensorNode 59 set node 0x00158d0005447ba1 13:23:13:660 SensorNode 58 set node 0x00158d0005447ba1 13:23:13:666 LightNode 12: Steckdose Keller added 13:23:13:668 LightNode 14: Steckdose 1. OG added 13:23:13:670 LightNode 24: Steckdose Keller Waschraum added 13:23:13:672 LightNode 25: Steckdose EG added 13:23:13:673 SensorNode 61 set node 0x00158d000522b07d 13:23:13:677 SensorNode 62 set node 0x00158d000522b099 13:23:13:680 SensorNode 64 set node 0x00158d000543ac9d 13:23:13:683 SensorNode 57 set node 0x00124b001c4b2978 13:23:13:685 SensorNode 52 set node 0x00124b001c4b2978 13:23:13:689 SensorNode 65 set node 0x00158d0005226eb5 13:23:13:692 SensorNode 67 set node 0x00158d0004667d9e 13:23:13:693 SensorNode 66 set node 0x00158d0004667d9e 13:23:13:697 SensorNode 68 set node 0x00158d00052250af 13:23:13:700 SensorNode 70 set node 0x00124b0022293db7 13:23:13:702 LightNode 26: Dachterrasse added 13:23:13:704 SensorNode 72 set node 0x00124b0022659d5b 13:23:13:706 SensorNode 71 set node 0x00124b0022659d5b 13:23:13:709 SensorNode 73 set node 0x00158d0005225ab8 13:23:13:712 SensorNode 74 set node 0x00158d00052272c1 13:23:13:715 SensorNode 75 set node 0x00158d00049fd020 13:23:13:717 SensorNode 76 set node 0x00158d00049fd020 13:23:13:721 SensorNode 79 set node 0x00158d00057b00c4 13:23:13:724 SensorNode 77 set node 0x00158d00057b014e 13:23:13:735 CTRL [00:0b:57:ff:fe:8c:1d:4b] not viable in zll.db, skip load cache (101) 13:23:13:737 CTRL [00:0d:6f:00:0f:bd:41:1c] not viable in zll.db, skip load cache (101) 13:23:13:742 CTRL [00:15:8d:00:03:2d:50:f4] not viable in zll.db, skip load cache (101) 13:23:13:792 dev /dev/ttyAMA0 13:23:14:887 dev /dev/ttyAMA0 13:23:14:925 COM check bootloader 13:23:14:937 COM detected application 13:23:15:002 Device firmware version 0x26580700 ConBee II 13:23:15:006 unlocked max nodes: 200 13:23:15:078 Device protocol version: 0x010B 13:23:15:223 Current channel 20 13:23:15:235 CTRL got nwk update id 1 13:23:15:239 CTRL ANT_CTRL 0x03 13:23:15:263 Device protocol version: 0x010B 13:23:15:336 Current channel 20 13:23:15:348 CTRL got nwk update id 1 13:23:15:352 CTRL ANT_CTRL 0x03 13:23:18:721 dev /dev/ttyAMA0 13:23:18:722 GW update firmware found: /usr/share/deCONZ/firmware/deCONZ_ConBeeII_0x26660700.bin.GCF 13:23:18:722 GW firmware version: 0x26580700 13:23:18:722 GW firmware version shall be updated to: 0x26660700 13:23:18:854 Announced to internet https://phoscon.de/discover 13:23:18:855 discovery server date: Sun, 11 Apr 2021 13:23:18 GMT 13:23:18:855 local time seems to be ok 13:23:23:843 [INFO] - No button map for: lumi.sensor_motion.aq2, unicast to: 0x0000, endpoint: 0x01, cluster: 0x0406, command: 0x0A, payload: 00001801, zclSeq: 4 13:23:23:844 ZCL attribute report 0x00158D0004667D9E for cluster: 0x0406, ep: 0x01, frame control: 0x18, mfcode: 0x0000 13:23:24:228 Set sensor check interval to 100 milliseconds 13:23:43:249 [INFO] - No button map for: TH01, unicast to: 0x0000, endpoint: 0x01, cluster: 0x0405, command: 0x0A, payload: 0000211B18, zclSeq: 178 13:23:43:250 ZCL attribute report 0x00124B001C4B2978 for cluster: 0x0405, ep: 0x01, frame control: 0x18, mfcode: 0x0000 13:23:58:229 Skip idle timer callback, too early: elapsed 906 msec 13:23:59:595 ZCL attribute report 0x000D6F000F842487 for cluster: 0x0500, ep: 0x01, frame control: 0x08, mfcode: 0x0000 13:23:59:596 Write poll cluster check-in interval for 0x000d6f000f842487 13:23:59:597 write attribute of 0x000D6F000F842487 ep: 0x01 cluster: 0x0020: 0x0000 13:24:02:141 New websocket 192.168.1.40:10361 (state: 3) 13:24:07:191 ZCL attribute report 0x000D6F000F83F89E for cluster: 0x0500, ep: 0x01, frame control: 0x08, mfcode: 0x0000 13:24:07:191 Write poll cluster check-in interval for 0x000d6f000f83f89e 13:24:07:192 write attribute of 0x000D6F000F83F89E ep: 0x01 cluster: 0x0020: 0x0000 13:24:07:246 Websocket disconnected 192.168.1.40:10361, state: 0, close-code: 1001, reason: 13:24:08:646 New websocket 192.168.1.40:10431 (state: 3) 13:24:12:742 Current channel 20 13:24:12:746 CTRL got nwk update id 1 13:24:12:750 Device TTL 3188 s flags: 0x7 13:24:13:887 ZCL attribute report 0x000D6F000FBF969F for cluster: 0x0500, ep: 0x01, frame control: 0x08, mfcode: 0x0000 13:24:13:888 Write poll cluster check-in interval for 0x000d6f000fbf969f 13:24:13:889 write attribute of 0x000D6F000FBF969F ep: 0x01 cluster: 0x0020: 0x0000 13:24:15:939 0x000D6F000F842487 error APSDE-DATA.confirm: 0xA7 on task 13:24:15:971 Multi state present value: 0x0002 (2), lumi.sensor_cube.aqgl01 13:24:15:972 [INFO] - Button 7000 (null) 13:24:15:973 [INFO] - No button map for: lumi.sensor_cube.aqgl01, unicast to: 0x0000, endpoint: 0x02, cluster: MULTISTATE_INPUT (0x0012), command: ATTRIBUTE_REPORT (0x0A), payload: 5500210200, zclSeq: 20 13:24:15:974 ZCL attribute report 0x00158D00057B00C4 for cluster: 0x0012, ep: 0x02, frame control: 0x18, mfcode: 0x0000 13:24:22:886 0x000D6F000F83F89E error APSDE-DATA.confirm: 0xA7 on task 13:24:26:003 ZCL attribute report 0x000D6F000FBFA739 for cluster: 0x0500, ep: 0x01, frame control: 0x08, mfcode: 0x0000 13:24:26:004 Write poll cluster check-in interval for 0x000d6f000fbfa739 13:24:26:004 write attribute of 0x000D6F000FBFA739 ep: 0x01 cluster: 0x0020: 0x0000 13:24:29:932 0x000D6F000FBF969F error APSDE-DATA.confirm: 0xA7 on task 13:24:41:865 0x000D6F000FBFA739 error APSDE-DATA.confirm: 0xA7 on task 13:24:51:306 [INFO] - No button map for: RWL021, unicast to: 0x0000, endpoint: 0x02, cluster: 0x0001, command: 0x0A, payload: 210020C8, zclSeq: 244 13:24:51:307 ZCL attribute report 0x001788010808716F for cluster: 0x0001, ep: 0x02, frame control: 0x08, mfcode: 0x0000 13:24:54:229 sensor 67 (lumi.sensor_motion.aq2): disable presence 13:24:54:705 write attribute of 0x000D6F000F842487 ep: 0x01 cluster: 0x0500: 0x0010 13:24:54:705 ZCL attribute report 0x000D6F000F842487 for cluster: 0x0500, ep: 0x01, frame control: 0x08, mfcode: 0x0000 13:24:55:128 Set sensor check interval to 1000 milliseconds 13:25:10:990 0x000D6F000F842487 error APSDE-DATA.confirm: 0xA7 on task 13:25:12:744 Current channel 20 13:25:12:748 CTRL got nwk update id 1 13:25:12:752 Device TTL 3128 s flags: 0x7 13:25:28:615 [INFO] - No button map for: Motion Sensor-A, unicast to: 0x0000, endpoint: 0x01, cluster: 0x0402, command: 0x0A, payload: 0000299507, zclSeq: 100 13:25:28:615 ZCL attribute report 0x000D6F000F842487 for cluster: 0x0402, ep: 0x01, frame control: 0x08, mfcode: 0x0000 13:25:28:813 dev /dev/ttyAMA0 13:25:28:814 GW firmware version: 0x26580700 13:25:28:814 GW firmware version shall be updated to: 0x26660700 13:25:31:304 Bind response success for 0x7cb03eaa0a08ff6d ep: 0x03 cluster: 0x0006 13:25:31:449 ZCL configure reporting rsp seq: 173 0x7CB03EAA0A08FF6D for ep: 0x03 cluster: 0x0006 attr: 0x0000 status: 0x00 13:25:32:852 ZCL attribute report 0x7CB03EAA0A08B3B4 for cluster: 0x0006, ep: 0x03, frame control: 0x18, mfcode: 0x0000

Additional context

n/a

Smanar commented 3 years ago

lights do not turn on/off and the old installation running in parallel still has the full control over the network

I think there is something strange here ? Are you using 2 conbee ? A zigbee device can't be on 2 network in same time, if you are using clone, you need to disable one.

airbone42 commented 3 years ago

lights do not turn on/off and the old installation running in parallel still has the full control over the network

I think there is something strange here ? Are you using 2 conbee ? A zigbee device can't be on 2 network in same time, if you are using clone, you need to disable one.

Sorry that I wasn't clear. Yes I'm using two Conbee II sticks on different deconz installations and used the export/import functionality from Phoscon. But also when I switch off the first Conbee II stick, the second still doesn't seem to be the "master" to switch any lights.

Sjonnie2018 commented 3 years ago

What if you use the first stick on the second installation? I am not sure but I "think" the pairing is between the stick and the ZigBee devices. Has little to do with the computer it's running on (that I know for sure because I migrated my stick from an RPi3 to an RPI4 without issues :) ) HTH

Smanar commented 3 years ago

I don't understand what you have done, can you explain using conbee 1/2 and computer 1/2 Remember the network setting is stored on the gateway, and the devices config on the host.

Sjonnie2018 commented 3 years ago

Not sure I am the person to answer your question but as I understand OP has two sticks and two computers. He has, as far as I can tell, migrated the database from computer one with stick one to computer two with stick two.

Something tells me that's never going to work.

If you are referring to my post: I migrated the stick and the database (export and import) to the new computer and re-used the stick.

Smanar commented 3 years ago

If you are referring to my post:

No ^^, that s for @airbone42, because with 2 conbee and 2 computers, there is lot of combinaisons possibles, I think the problem is in his procedure too.

airbone42 commented 3 years ago

If you are referring to my post:

No ^^, that s for @airbone42, because with 2 conbee and 2 computers, there is lot of combinaisons possibles, I think the problem is in his procedure too.

Sorry for the confusion.

Let's give the current setup some names:

The already running Windows-system has a Conbee II stick (named Adam)
The new Rasbian has a new Conbee II stick (named Eva)

But I also tried using Adam in the Rasbian setup and the result was exactly the same, that I could only read the network, but not "write" aka change the state of lights.

Smanar commented 3 years ago

You have made the backup on the windows host with Adam, and the restore on raspbian with Eva ?

And if you use Eva on the windows host ? to check if the problem is coming from the host or the conbee.

But seriously IDK what can happen if you have 2 coordinators with same setting, connected in same time ...

airbone42 commented 3 years ago

You have made the backup on the windows host with Adam, and the restore on raspbian with Eva ?

Yes, exactly.

And if you use Eva on the windows host ? to check if the problem is coming from the host or the conbee.

That I'm actually afraid of that it messes up my whole network.

airbone42 commented 3 years ago

Got it solved now, but only by restarting with a totally clean platform and then using directly Adam on the Rasbian platform. If I use first Eva and switch it later it doesn't work.

Would therefore close this issue.

If anyway someone has experience running two coordinators in a network to have some kind of a fallback, links to more information would be welcome. :)

dresden-elektronik / deconz-rest-plugin