search

grobasoz / zigbee-firmware

ZigBee Development Firmware
GNU General Public License v3.0
106 stars 22 forks source link

[REQUEST] Utilize new common "Silicon Labs firmware builder" automated build system from Home Assistant / Nabu Casa developers to create reproducible builds of Zigbee and Thread firmware images #41

Open Hedda opened 1 year ago

Hedda commented 1 year ago

Please consider making use of the "Silicon Labs firmware builder" which is a new open-source firmware build system that Home Assistant founders have now among other things announced as a common script tool set for building firmware images for Silabs based hardware.

https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/#building-firmware-for-silicon-labs-chips

(Note that this is only just one section from https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/ )

In summary, they have open-sourced and released the ""Silicon Labs firmware builder" project which is a collection of automated build scripts to help make it easier to create reproducible firmware builds for Zigbee and Thread based SoCs from Silicon Labs.

Please consider making use of this firmware build system and if possible also contribute any improvements to upstream project:

https://github.com/NabuCasa/silabs-firmware-builder

As I understand it, right now it allows you to build either Zigbee EmberZNet NCP (Zigbee Coordinator on Silicon Labs Network Co-Processor) firmware images, OpenThread RCP (Radio Co-Processor), or RCP Multi-PAN firmware images for OTBR (multi-protocol Zigbee EmberZNet and OpenThead Border Router using Radio Co-Processor) profiled for their own Home Assistant SkyConnect USB stick/dongle and their EFR32MG21-based radio module integrated into the Home Assistant Yellow appliance, however, the idea is that you can build profiles for other Silicon Labs hardware adapters/dongles/sticks/modules as well. It could of course also be extended to not only support different Silicon Labs based hardware adapters/dongles/sticks/modules but also different type of firmware images, (like maybe OpenThead Border Router on NCP).

(Examples of firmware image builds that it can produce can be found here -> https://github.com/NabuCasa/silabs-firmware ).

Note that was just one of their announcements in that blog post; in related news, they also released "SL Web Tools" which enable flashing firmware to Silabs-based adapters directly from a web page as well as a "SiLabs multi-flasher" as a Python script flashing tool:

https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/#silabs-multi-flasher--sl-web-tools

(Again note that this is only just one section from https://www.home-assistant.io/blog/2023/02/08/state-of-matter-and-thread/ )

https://github.com/NabuCasa/universal-silabs-flasher

https://github.com/NabuCasa/sl-web-tools

https://www.youtube.com/watch?v=-88K23e8XYw&t=2s&ab_channel=HomeAssistant

The "SL Web Tools" project is available as open source. It allows users to manage the firmware on Zigbee/Thread sticks using SiLabs chips directly from their browser, with no terminals or compilation necessary. This allows any creator to offer easy firmware updates to their users and make experimental updates available to test bug fixes quickly."

To see it in action, SL Web Tools for flashing firmware has been added to the Home Assistant SkyConnect documentation.

https://skyconnect.home-assistant.io

SL Web Tools is powered by another new open-source Python package “silabs-universal-flasher” → GitHub - NabuCasa/universal-silabs-flasher: Flashes Silicon Labs radios running EmberZNet or CPC multi-pan firmware

For reference; the firmware build system for Silicon Labs was made by Home Assistant developer/engineer and Nabu Casa employee Stefan Agner (a.k.a. agners, who also developed the EFR32MG21-based Home Assistant SkyConnect USB stick/dongle as well as the EFR32MG21-based radio module integrated into the Home Assistant Yellow appliance for Rasberry Pi 4 Compute Module). And the "universal-silabs-flasher" firmware tool was made by puddly who is also one of the lead developers of zigpy and the radio libraries for zigpy that is used in Home Assistant's ZHA integration (among other projects).

PS: They made updates to their documentation as relevant to Silicon Labs based adapters with correct firmware for Zigbee or Thread:

grobasoz commented 1 year ago

Very interesting. I'll be sure to have a look when I get back to the office.

Hedda commented 1 year ago

FYI, ksjh has begun some work on such automated firmware building using Github Actions and started a related discussion on it:

https://github.com/ksjh/silabs-firmware-builder

https://github.com/ksjh/silabs-firmware-builder/discussions/2

Note that most of the automation magic with silabs-firmware-builder looks to happen in Github Actions workflows:

https://github.com/features/actions

https://docs.github.com/en/actions

Hedda commented 1 year ago

FYI, ksjh and darkxst added a few more adapters to their fork (but one big thing still missing is ability to easily set different config):

https://github.com/ksjh/silabs-firmware-builder

Also, tube0013 and giannello have also created their own forks with patches for other hardware models, see:

https://github.com/tube0013/silabs-firmware-builder

and

https://github.com/giannello/silabs-firmware-builder

PS: Hopefully upstream will someday get features to set different Ember configuratiuon parameters per adapter. See this request:

https://github.com/NabuCasa/silabs-firmware-builder/issues/14 (and https://github.com/NabuCasa/silabs-firmware-builder/issues/13)

Hedda commented 1 year ago

@grobasoz perhaps you have some interest and input or feedback on this discussion(?), see -> https://github.com/ksjh/silabs-firmware-builder/issues/17

Please read the whole thread do understand what I mean.

Also check out this repo by darkxst for patches:

https://github.com/darkxst/silabs-builder-remote

Hedda commented 1 year ago

FYI, darkxst fork of Nabu Casa's silabs-firmware-builder has added support for multiple hardware manifests to allow override configs for automatic builds to enable automatic building of firmware images for many different Silabs based EFR32 adapters:

https://github.com/darkxst/silabs-firmware-builder/blob/manifestjs/manifests

As such he is the first to have firmware builds for Silicon Labs EmberZNet 7.3 (though untested since they are automatically built):

https://github.com/darkxst/silabs-firmware-builder/tree/main/firmware_builds

PS: He is also offering experimental builds for the same EmberZNet version and hardware but with different baud rate speeds.

Hedda commented 1 year ago

FYI, darkxst is not only continuing to improve his fork of silabs-firmware-builder and he has now also forked the Silabs Flasher add-on for Home Assistant and is working on making it compatible with many different Silicon Labs based USB adapters, see:

PS: Another tip is to also check out his blog -> https://dialedin.com.au/blog/

Hedda commented 1 year ago

@grobasoz slightly off-topic but thought that you might be interested to know that darkxst have begun working on an experimental "Silicon Labs Multipan Flasher Add-on" as a fork of the official "Silabs Flasher" addon for Home Assistant to let users easily flash the firmware images from Home Assistant's GUI for built with Silicon Labs firmware builder on the other products that the official Home Assistant SkyConnect dongle, e.i. Zigbee Coordinator adapters like Easyiot zigbee 3.0 USB Dongle model ZB-GW04, ITead’s "Sonoff Zigbee 3.0 USB Dongle Plus V2” (model “ZBDongle-E”), and Elelabs ELU013 USB radio stick:

https://github.com/darkxst/multipan_flasher/tree/main

https://github.com/darkxst/multipan_flasher/blob/main/multipan_flasher/README.md

"This is a fork of the official Silabs Flasher HA Addon, but is shipped with my firmware builds. Once configured, It will automatically update dongle firmware on next reboot of HA, or you can manually run it after installing/updating. Currently supports ZBDongle-E and a few others."

Developers and users that have issues with it can create an issue here:

https://github.com/darkxst/silabs-firmware-builder/issues

darkxst is not only continuing to improve his fork of silabs-firmware-builder and he has now also forked the Silabs Flasher add-on for Home Assistant and is working on making it compatible with many different Silicon Labs based USB adapters, see:

PS: Another tip is to also check out his blog -> https://dialedin.com.au/blog/

grobasoz commented 1 year ago

Thanks @Hedda - Will have a look through! Gary.

Hedda commented 2 weeks ago

FYI, many other downstream devs are now collaborating on improving upstream, including added Bluetooth + OpenThread RCP:

PS: Upstream will soon also migrate from Gecko SDK to Simplicity SDK which will bring larger changes to dependecies and builds.

grobasoz commented 2 weeks ago

Thanks @Hedda - Always good to hear from you! At this stage I don't see a need for me to continue with Zigbee Firmware releases, there seems to be plenty of solutions available now.