Part 2 (of 2) of handling the latest deconz firmware

chas-iot commented 3 years ago

Original code from @frederic34 in https://github.com/WebThingsIO/serial-prober-node/issues/11#issuecomment-736282755

@dhylands provided the idea to clone the probe in https://github.com/WebThingsIO/serial-prober-node/issues/11#issuecomment-736659917

chas-iot commented 3 years ago

Logs from my testing:


2020-12-05 12:31:53.885 INFO   : Unloading ZigbeeAdapter
2020-12-05 12:31:57.400 INFO   : Loading add-on: zigbee-adapter
2020-12-05 12:31:58.339 INFO   : zigbee-adapter: Opening database: /home/pi/.webthings/config/db.sqlite3
2020-12-05 12:31:58.368 INFO   : zigbee-adapter: Ignoring https://raw.githubusercontent.com/WebThingsIO/gateway-addon-ipc-schema/master/schema.json because it has no messageType
2020-12-05 12:31:59.166 INFO   : zigbee-adapter: Ignoring https://raw.githubusercontent.com/WebThingsIO/gateway-addon-ipc-schema/master/messages/definitions.json because it has no messageType
2020-12-05 12:32:00.487 INFO   : zigbee-adapter: Loading add-on zigbee-adapter from /home/pi/.webthings/addons/zigbee-adapter
2020-12-05 12:32:00.540 INFO   : zigbee-adapter: DEBUG config = 'serialProber'
2020-12-05 12:32:00.541 INFO   : zigbee-adapter: Enabling DEBUG_serialProber
2020-12-05 12:32:00.767 INFO   : zigbee-adapter: SerialProber: /dev/ttyACM0 filter for XBee didn't match.
2020-12-05 12:32:00.768 INFO   : zigbee-adapter: SerialProber: Probing /dev/ttyACM0 at 38400 for deConz
2020-12-05 12:32:00.770 INFO   : zigbee-adapter: SerialProber: Opening /dev/ttyACM0 at 38400 baud
2020-12-05 12:32:00.775 INFO   : zigbee-adapter: SerialProber: Probing /dev/ttyACM0 ...
2020-12-05 12:32:00.777 INFO   : zigbee-adapter: SerialProber: Sent: <Buffer c0 0d 01 00 05 00 ed ff c0>
2020-12-05 12:32:01.277 INFO   : zigbee-adapter: SerialProber: closing /dev/ttyACM0
2020-12-05 12:32:01.279 INFO   : zigbee-adapter: SerialProber: timeout: deConz dongle not detected on /dev/ttyACM0
2020-12-05 12:32:01.287 INFO   : zigbee-adapter: SerialProber: /dev/ttyACM0 filter for cc2531 didn't match.
2020-12-05 12:32:01.288 INFO   : zigbee-adapter: SerialProber: Probing /dev/ttyACM0 at 38400 for deConz
2020-12-05 12:32:01.289 INFO   : zigbee-adapter: SerialProber: Opening /dev/ttyACM0 at 38400 baud
2020-12-05 12:32:01.291 ERROR  : zigbee-adapter: SerialProber: timeout: deConz dongle not detected on /dev/ttyACM0
2020-12-05 12:32:01.292 INFO   : zigbee-adapter: SerialProber: Probing /dev/ttyACM0 ...
2020-12-05 12:32:01.293 INFO   : zigbee-adapter: SerialProber: Sent: <Buffer c0 0d 01 00 09 00 00 00 00 00 e9 ff c0>
2020-12-05 12:32:01.294 INFO   : zigbee-adapter: SerialProber: Rcvd: <Buffer c0 0d 01 00 09 00 00 07 68 26 54 ff c0>
2020-12-05 12:32:01.295 INFO   : zigbee-adapter: SerialProber: Probe successful
2020-12-05 12:32:01.296 INFO   : zigbee-adapter: SerialProber: /dev/ttyAMA0 filter for XBee didn't match.
2020-12-05 12:32:01.297 INFO   : zigbee-adapter: SerialProber: /dev/ttyAMA0 filter for deConz didn't match.
2020-12-05 12:32:01.302 INFO   : zigbee-adapter: SerialProber: /dev/ttyAMA0 filter for cc2531 didn't match.
2020-12-05 12:32:01.303 INFO   : zigbee-adapter: SerialProber: /dev/ttyAMA0 filter for deConz didn't match.
2020-12-05 12:32:01.304 INFO   : zigbee-adapter: SerialProber: /dev/ttyS0 filter for XBee didn't match.
2020-12-05 12:32:01.305 INFO   : zigbee-adapter: SerialProber: /dev/ttyS0 filter for deConz didn't match.
2020-12-05 12:32:01.306 INFO   : zigbee-adapter: SerialProber: /dev/ttyS0 filter for cc2531 didn't match.
2020-12-05 12:32:01.307 INFO   : zigbee-adapter: SerialProber: /dev/ttyS0 filter for deConz didn't match.
2020-12-05 12:32:01.580 INFO   : zigbee-adapter: this.driver = DeconzDriver {}
2020-12-05 12:32:01.582 INFO   : zigbee-adapter: DeconzDriver: Using serial port /dev/ttyACM0
2020-12-05 12:32:01.601 INFO   : zigbee-adapter: DeConz Firmware version: 26680700

chas-iot commented 3 years ago

Part 1 is in https://github.com/WebThingsIO/deconz-api/pull/17

fixes https://github.com/WebThingsIO/serial-prober-node/issues/11

chas-iot commented 3 years ago

@kgiori This is part 2 of the fix that I'd like to help regression test. I have to run, I'll explain later (hopefully today/Saturday)

chas-iot commented 3 years ago

@mrstegeman as I was walking, I had an idea to enhance the serial prober to take an array of probe/response pairs for a single device type, to reduce the duplication/ugliness of this patch.

What do you think of this approach?

mrstegeman commented 3 years ago

Fine by me, as long as the API doesn’t break.

chas-iot commented 3 years ago

Fine by me, as long as the API doesn’t break.

Ah! That's not so easy.

chas-iot commented 3 years ago

@mrstegeman please review and suggest any required changes to these instructions. @kgiori - If you are interested in helping then thank-you 🙂

This is part 2 of 2 changes to support ConBee / RaspBee ZigBee devices with very recent firmware. The purpose of this testing that we are asking you to do is to ensure that the changes do not break anything for with devices without upgraded firmware.

Steps:

Complete part 1 of the testing and leave that change in place
Backup .webthings/addons/zigbee-adapter/index.js - If you are not using a Raspberry Pi, this path may be different
Create a .git directory in .webthings/addons/zigbee-adapter/index.js
Extract https://github.com/chas-iot/zigbee-adapter/blob/firmware-handling/index.js to .webthings/addons/zigbee-adapter
In the gateway UI, go to Settings / Add-ons / ZigBee / Configure
Set debug to serialProber then click Apply - as this setting should already be in place, you may need to add then delete a space at the end before clicking Apply
Wait 30 seconds
Check the Developer logs for startup errors - Please let us know the results.

Possible issues:

The ZigBee adapter may fail to start or hang because the device does not recognise the slightly changed messages
The adapter may try to start two instances of the deconz driver on the same serial port with nasty conflicts
The adapter may try to start two instances of the deconz driver on the same serial port and the 2nd fails as the serial port is already in use - this is OK but may look messy in the logs

Backout

remove any .git directories created above
remove the changed index.js and replace with the version from the backup
remove the debug string from the ZigBee adapter configuration
restore the changed .webthings/addons/zigbee-adapter/node_modules/deconz-api/lib/frame-builder.js from Part 1

mrsteakhouse commented 3 years ago

I applied your test steps and everything works quite well. However the UI seems to not recognize that events were emitted, the attached rules to that events are still triggered and work anyways. When testing this i noticed many error messages stating Invalid message received or Validation error~~, which disappeared once the serialProber debug flag was removed~~.

frederic34 commented 3 years ago

2021-01-26 08:49:00.152 INFO   : zigbee-adapter:          MAC Address: 00212effff05562f
2021-01-26 08:49:00.154 INFO   : zigbee-adapter:      Network PANID16: 499e
2021-01-26 08:49:00.155 INFO   : zigbee-adapter:       Network Addr16: 0000
2021-01-26 08:49:00.156 INFO   : zigbee-adapter:      Network PANID64: 00212effff05562f
2021-01-26 08:49:00.158 INFO   : zigbee-adapter: APS Designated Coordinator: 1
2021-01-26 08:49:00.159 INFO   : zigbee-adapter:        Scan Channels: 02000000
2021-01-26 08:49:00.161 INFO   : zigbee-adapter:          APS PANID64: 0000000000000000
2021-01-26 08:49:00.162 INFO   : zigbee-adapter:  Trust Center Addr64: 00212effff05562f
2021-01-26 08:49:00.163 INFO   : zigbee-adapter:        Security Mode: 3
2021-01-26 08:49:00.165 INFO   : zigbee-adapter:          Network Key: *************************************
2021-01-26 08:49:00.166 INFO   : zigbee-adapter:    Operating Channel: 25
2021-01-26 08:49:00.168 INFO   : zigbee-adapter:     Protocol Version: 267
2021-01-26 08:49:00.170 INFO   : zigbee-adapter:    Network Update ID: 0
2021-01-26 08:49:00.171 INFO   : zigbee-adapter:          Permit Join: 0
2021-01-26 08:49:00.172 INFO   : zigbee-adapter:         WatchDog TTL: 3600
2021-01-26 08:49:00.174 INFO   : zigbee-adapter:              Version: 264a0700
2021-01-26 08:49:00.175 INFO   : zigbee-adapter: EXEC_FUNC adapterInitialized
2021-01-26 08:49:00.177 INFO   : zigbee-adapter: adapterInitialized

I done the changes, test gateway still working (CONBEE II)

chas-iot commented 3 years ago

I done the changes, test gateway still working (CONBEE II)

Did you include part 1? - https://github.com/WebThingsIO/deconz-api/pull/17

frederic34 commented 3 years ago

@chas-iot yes I did

WebThingsIO / zigbee-adapter

Part 2 (of 2) of handling the latest deconz firmware #265