fakuivan / orvibo-gynoid-zigbee-hub-hack

Scripts to replace the vihome software on orvibo/gynoid hubs for a serial gateway to home assistant
1 stars 0 forks source link

Using the hub as ZigBee router #1

Open fakuivan opened 2 months ago

fakuivan commented 2 months ago

So far we've got two options:

  1. Flashing Router firmware to the MCU
  2. Making the controller join an existing network as a router

Option 1 requires a router firmware which so far I wasn't able to find, and flashing could require tapping into the UART comms between the SOC and the MCU, since the program JennicModuleProgrammer doesn't seem to support the baud rate the MCU communicates at.

As for option 2, I was able to use bellows join to make the controller join an existing network, however the device shows up as Device Type: EndDevice, meaning it's not acting as a router. Ideally this could be solved if the controller can be set to join as a router instead of an end device. bellows join seems to be pretty outdated as the functions it calls changed arguments at some point, I had to make some modifications to make it work so that'll be going as a PR at some point.

To be sure I wanted to dump the MCU firmware before doing anything, but that seems to be rather complicated, here's the relevant issue: https://github.com/zigpy/bellows/issues/622.

MattWestb commented 2 months ago

I think the original firmware is using hardware flow control if you looking you have 4 lanes from the WiFi / MCU to the Zigbee chip. If flashing one with software or non its still working but better using hardware then its supported by the hardware. The small mini connector (not populated) naked JTAG can being the new simplistudio or one older pin out for the EM5X chips but you is having all in the silabs docks also its looks like they is on the TP pads plus 3 more so shall being easy finding the SWD pins for dumping and flashing the chip.

By the way Bellows can only joining one network as end device = no router functionality. If you like have one good DYI router take on old IKEA light and use the Zigbee module in it then i have cooking one router firmware for it and its is one modern chip (but out of active support).

MattWestb commented 2 months ago

The 3 extra TP pads look having GND, RTC and CTS that is not on the standard silabs micro SWD contact (or its force bootloader).

fakuivan commented 2 months ago

You're right @MattWestb, the extra tracks are for flow control, and those three pads are GND, TX and RX.

I tried changing the following line from t.EmberNodeType.END_DEVICE to t.EmberNodeType.ROUTER:

https://github.com/zigpy/bellows/blob/b008ccf6d5a0490ac28b113d9d7e64c7a1733040/bellows/cli/network.py#L102

but now the device is stuck at "Starting Interview", previously if the device was set to END_DEVICE, making it leave the network and join again would advance the onboarding process, but in this case I wasn't able to make it go past that.

image

Hedda commented 2 months ago

By the way Bellows can only joining one network as end device = no router functionality.

For reference, MattWestb submitted a feature request to the bellows repositty for such a CLI command function a few years ago:

FYI, zigpy developers have however since made it more i statements that the bellows CLI and the CLI of other radio libraries for zigpy are partially deprecated as stand-alone tools in the way that they no longer maintained on their own because they are no longer meant to be used directly on their own, as instead you are supposed to all radio libraries indirectly via the CLI of zigpy-cli and/or the API of the main zigpy library.

Back-story is that the CLI of some radio libraries like bellows (and zigpy-znp) was developed on their own as a stand-alone libray before the main zigpy library was created and long before the zigpy-cli was created as a unified command line interface for zigpy radios and therefore the CLI of radio libraries like bellows and zigpy-znp may have several additional extra CLI command features that have not yet been implemented into zigpy-cli or the API zigpy library, so there is not full feature-parity, thus I think your best approach would be to submit a new feature request for “command ability to join Zigbee NCP device as Zigbee Router” (or similar) as an open issue for zigpy-znp or continue that discussion under the main zigpy repository.

So the bellows CLI is no longer maintained at this point because it includes a lot of duplicate functionality from zigpy-cli, which uses up-to-date interfaces and works with every radio library for zigpy, (other than packet capture, which can be abstracted into a generic method within zigpy), everything else is radio-agnostic and should be supported within zigpy-cli.

If something is missing in zigpy-cli that is available in bellows then best if you code it and make a pull request adding the functionality you want to zigpy-cli (as that would be much appreciated by everyone).

If you can not code this yourself then I suggest that you post a new feature request for “Zigbee Router mode for Zigbee NCP” (and “Join” command) for zigpy-cli repo:

Btw, I also posted related question about "zigpy router mode" as an open discussion to the zigpy developers a few years ago here:

One possible real-world use-case scenario for "zigpy router mode" having ZHA or zigpy-cli join an existing Zigbee network to be able to capture Zigbee OTA firmware images (by sending OTA image request with the required image_id, hardware_id, etc., which is something that the deCONZ OTA plugin has the capability of doing) after joining commercial Zigbee gateways/bridges/hubs (like the IKEA Trådfri Gateway, Philips Hue Bridge, or the Samsung SmartThings Hub). As bell as joinged other Zigbee networks for troubleshooting and debugging as a developer tool. See example:

Another possible real-world use-case scenario is having two Home Assistant instances with the ZHA in the same home/house but only one of them acting as a Zigbee Gateway and having the optiuon to join the second as a Zigbee Router to the first node instead having to setup a new Zigbee network, (could possible even work in some kind of high-availability solution where both are always active that way and it is possible to fail-over to the other node by restoring ZHA backup). After all, a Zigbee Coordinator is technically more or less a Zigbee Router with the added role of security gatekeeper, holding the keys and allowing other devices to join the network or not.

MattWestb commented 2 months ago

Router is boring and is not doing anything extra but if cooking one Zigbee Controller bridge (ZCB) you can joining it as one router and can controlling all devices in the network with CLI commands (its the same way IKEA TF and Philips HUE is working at the network level) but its not easy doing it for EM8X chips as i was saying.

Hedda commented 2 months ago

Router is boring and is not doing anything extra

Not boring; extra could be a feature for capturing Zigbee OTA images when joining commercial Zigbee gateways/bridges/hubs:

_One possible real-world use-case scenario for "zigpy router mode" having ZHA or zigpy-cli join an existing Zigbee network to be able to capture Zigbee OTA firmware images (by sending OTA image request with the required image_id, hardwareid, etc., which is something that the deCONZ OTA plugin has the capability of doing) after joining commercial Zigbee gateways/bridges/hubs (like the IKEA Trådfri Gateway, Philips Hue Bridge, or the Samsung SmartThings Hub). As bell as joinged other Zigbee networks for troubleshooting and debugging as a developer tool. See example:

Another possible real-world use-case scenario is having two Home Assistant instances with the ZHA in the same home/house but only one of them acting as a Zigbee Gateway and having the optiuon to join the second as a Zigbee Router to the first node instead having to setup a new Zigbee network, (could possible even work in some kind of high-availability solution where both are always active that way and it is possible to fail-over to the other node by restoring ZHA backup). After all, a Zigbee Coordinator is technically more or less a Zigbee Router with the added role of security gatekeeper, holding the keys and allowing other devices to join the network or not.

MattWestb commented 2 months ago

ZCB is better then can doing all things also binding controllers to it and getting all commands the devices is sending also then the coordinator is offline. For sniffing its best wireshark and one router the device is joined to and sniffing then you never knowing if the network is changing the routing and you is loosing the frames you dearly like to have.

Read more of the ZCB in the Zigbee Cluster Liebery and use your fantacy what can being done or not (IKEA and Philips was doing great with them).

Hedda commented 2 months ago

"ZCB"? Do you refer to "Zigbee Control Bridge" from the Zigbee Lighting & Occupancy Device (ZLO) specifications?

https://zigbeealliance.org/wp-content/uploads/2019/11/docs-15-0014-05-0plo-Lighting-OccupancyDevice-Specification-V1.0.pdf

If specifically search for "Silicon Labs" AND "Zigbee" AND "control bridge" I only found reference in UG103.09: ZLL Fundamentals:

https://www.silabs.com/documents/public/user-guides/ug103-09-fundamentals-zll.pdf

Maybe just be but I can not seem to find anything useful when do google searches for "Zigbee Control Bridge" or "ZCB", etc.?

The only real references I found are these other posts mentioning "Zigbee Control Bridge" are these posts in the NXP community:

https://community.nxp.com/t5/Wireless-Connectivity/Using-the-NXP-Zigbee-Control-bridge-in-Router-Mode-and-send/m-p/1573134

https://community.nxp.com/t5/Wireless-Connectivity/Steps-Guidelines-to-add-Light-Endpoints-to-Zigbee-Control-bridge/m-p/1573724#M14217

NXP also provides some documentation and an example application for a ZigBee Control Bridge to facilitate an IP connection:

https://www.nxp.com/webapp/sps/download/license.jsp?colCode=JN-AN-1247&location=null

https://www.nxp.com/webapp/sps/download/license.jsp?colCode=JN-AN-13051&location=null

https://www.nxp.com/docs/en/application-note/JN-AN-1216.zip

https://www.nxp.com/docs/en/application-note/JN-AN-1223.zip

PS: Off-topic but sounds a bit similar to the high-level concept Matter Control bridge (like Silicon Labs "Unify Matter Bridge"), or? Matter is also based on ZCL, and I think there a control bridge is a controller device that allows all lights to be controlled by any Matter controller, or?

https://community.nxp.com/pwmxy87654/attachments/pwmxy87654/imx-processors%40tkb/5759/6/MatterZigbeeBridge-UserGuide-1.0.pdf

https://docs.silabs.com/matter/2.1.0/matter-bridge/readme-user

nRF Connect SDK (Nordic Semiconductor) also mention "Control Bridge" under Switches and Controls device types Matter Data Model and device types but there that "control bridge" is in reference to binding between a switch (or light) and a remote control? So when bound to a lighting device such as a color light, is capable of being used to switch the device on or off, adjust the intensity of the light being emitted and adjust the color of the light being emitted. In addition, a Control Bridge device is capable of being used for setting scenes?

https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/protocols/matter/overview/data_model.html

MattWestb commented 2 months ago

ZLO is integrated in ZCL R8 (was appended to earlier ZCL) and also in Matter 1.2 6.4. Control Bridge.

MattWestb commented 2 months ago

PM Matter cluster is one extended ZCL R8 and have all Zigbee stuff but more and its being the standard in the future (more or less ZCL R9).

Hedda commented 2 months ago

So maybe "Zigbee Control Bridge" is not usable in older Silabs EmberZNet Zigbee versions with old Silicon Labs Gecko SDK? Or?

MattWestb commented 2 months ago

Its in GSDK 3.2 (closed / licence needed) (and many earlier) that was having EM support but i cant compiling it then its needs extra tool change that cost 3800 dollar (Gary have it and the knowledge).

fakuivan commented 2 months ago

I might have been thinking about this "join as router" mode wrong. If the controller joins as a router, does that mean that it'll have to be actively controlled by something like zha or will it work independently?

Hedda commented 2 months ago

If the controller joins as a router, does that mean that it'll have to be actively controlled by something like zha or will it work independently?

I am not sure what you mean? Are you now referring to a standard Zigbee Router device or MattWestb's "Zigbee Control Bridge"?

Zigbee Router device can not work totally independent, it will only act as a Zigbee Router device in the network it is joined to. But a standard Zigbee Router device firmware will always work as a Zigbee Router device in that network after it has been joined to a Zigbee network. Device specific features and functions will depend on the firmware build, but again, a Zigbee Router device will always act as a Zigbee Router device in the network it is joined to (and no other Zigbee networks).

https://community.silabs.com/s/article/what-is-the-difference-between-an-end-device-a-router-and-a-coordinator-do-i

Recommend that you read Zigbee fundamentals and check out some of these example applications:

fakuivan commented 2 months ago

@Hedda here's an example:

I make an EZSP-based controller join a network as a router via bellows or whatever CLI. Will it work as a router without requiring any supporting software commanding it via EZSP?

I'm not asking if a generic ZigBee router can work without a generic ZigBee controller. Another example: I join the controller to a network as a router, disconnect it from my computer, and plug it into a USB charger. Will it still work as a proper router?

Hedda commented 2 months ago

here's an example:

I make an EZSP-based controller join a network as a router via bellows or whatever CLI. Will it work as a router without requiring any supporting software commanding it via EZSP?

I'm not asking if a generic ZigBee router can work without a generic ZigBee controller. Another example: I join the controller to a network as a router, disconnect it from my computer, and plug it into a USB charger. Will it still work as a proper router?

Generally yes, but exactly how it will work depends on the specific firmware that you build. Anyway, as POC that's how these work:

That is even how ITead/Sonoff is marketing their Sonoff Zigbee 3.0 USB Dongle Plus ZBDongle-E and ZBDongle-P adapters:

https://itead.cc/product/zigbee-3-0-usb-dongle/

"The dongle plus has a wide range of supported devices, such as ZBMINI-L, SNZB01, SNZB02, SNZB03, SNZB04, S26R2ZB, S40 LITE ZB. Dongle Plus is pre-flashed with coordinator firmware, you can also flash router firmware to extend the network range."

image

https://sonoff.tech/product/gateway-and-sensors/sonoff-zigbee-3-0-usb-dongle-plus-e/

image

http://sonoff.tech/wp-content/uploads/2022/07/%E8%AF%B4%E6%98%8E%E4%B9%A6-ZBDongle-E-V1.1-20220712.pdf

https://sonoff.tech/product/gateway-and-sensors/sonoff-zigbee-3-0-usb-dongle-plus-p/

https://itead.cc/product/sonoff-zigbee-3-0-usb-dongle-plus/

Hedda commented 2 months ago

Off-topic but FYI, noticed that ITead/Sonoff have a new USB adapter product that basically does that out-of-the-box for $12.90 US:

https://itead.cc/product/sonoff-micro-zigbee-usb-smart-adaptor/

"Besides, ZBMicro can work as a Zigbee router to help transmit the Zigbee signal to ensure a stable connection when multiple Zigbee devices are used for your home. Elevate your network capabilities with Turbo Mode, featuring advanced technology that boosts signal range and strength but also provides a seamless and stable connection for your devices."

https://sonoff.tech/product/diy-smart-switches/zbmicro/

image

fakuivan commented 2 months ago

here's an example: I make an EZSP-based controller join a network as a router via bellows or whatever CLI. Will it work as a router without requiring any supporting software commanding it via EZSP? I'm not asking if a generic ZigBee router can work without a generic ZigBee controller. Another example: I join the controller to a network as a router, disconnect it from my computer, and plug it into a USB charger. Will it still work as a proper router?

Generally yes, but exactly how it will work depends on the specific firmware that you build.

Again, sorry for the confusion, but I mean using the controller firmware, not flashing one that's specifically designed to work as a router. Will the NCP running the controller firmware require some sort of support via EZSP to act as a router after being joined as a router using bellows or other CLI?

Hedda commented 2 months ago

Will the NCP running the controller firmware require some sort of support via EZSP to act as a router after being joined as a router using bellows or other CLI?

It might be possible via EZSP commands but as mentioned in my first post above it is not something that is currently supported by zigpy-cli or zigpy, so suggest you submit a feature request to zigpy-cli or better yet code it and make a pull request to zigpy-cli if you are capable. Again note that it would have to be for zigpy-cli or zigpy today, and not a specific radio library like bellows.

fakuivan commented 2 months ago

I've decided that the most sane option would be to go with commanding the NCP to work as a router, instead of trying to build a firmware specific to this device. Also I wasn't able to get my hands on the required GSDK version so that's that :c.

I've opened this issue to get some help on making the router and end-device "emulation" work https://github.com/zigpy/zigpy/issues/1399

fakuivan commented 1 month ago

Here's what I've come up with to make this work as a standalone router. The program can be compiled against the realtek SDK to run on the SOC.

https://github.com/fakuivan/ezsp-router

There's no ZCL functionality, next it'll be adding that.