search

Koenkk / zigbee-herdsman

A Node.js Zigbee library
MIT License
485 stars 301 forks source link

[REQUEST] Support for Silicon Labs EFR32 (Mighty Gecko family) based Zigbee radio adapters using EZSP serial protocols? #168

Closed Hedda closed 4 years ago

Hedda commented 4 years ago

Will you consider adding support for Silicon Labs Zigbee cordinator dongles/sticks via EZSP (EmberZNet Serial Protocol) CLI/API?

It is maybe possible for you to port the EZSP (EmberZNet Serial Protocol) code from the bellows library to zigbee-herdsman (in a similar manner to how zigpy-cc which was ported from zigbee-herdsman)?

I understand that the bellows library itself was based Silabs official EZSP Reference Guides from Silicon Labs, like UG100:

PS: Silicon Labs EFR32 family replaces its old ETR35x however both are Ember NCP and support the same EmberZNet Zigbee stack.

Hedda commented 4 years ago

I understand that an adapter interface code needs to be implemented in zigbee-herdsman for Zigbee2MQTT and ioBroker support?

https://github.com/Koenkk/zigbee-herdsman/blob/master/src/adapter/adapter.ts

https://github.com/Koenkk/zigbee-herdsman/tree/master/src/adapter

Would that be a similar to how deConz (ConBee/RaspBee) adapters and ZiGate adapter has been added to zigbee-herdsman?

https://github.com/Koenkk/zigbee-herdsman/issues/72

https://github.com/Koenkk/zigbee-herdsman/pull/241

@Koenkk if you do not already have a Silicon Labs EFR32MG1 or EFR32MG2 based Zigbee dongle/stick then could I offer to donate hardware to you? Maybe make an earmark PayPal donation of $30 US-dollars to you so that you can purchase on for yourself?

PS: I am only a home automation enthusiast and not affiliated with Ebyte, Silicon Labs, or any resellers of hardware or software. Unfortunately, I do not have the TypeScript/JavaScript coding skills to port or implement EZSP adapter support zigbee-herdsman.

Hedda commented 4 years ago

Silicon Labs also has a few different official EFR32MG2 and EFR32MG1 devkits called "EFR32 Mighty Gecko Wireless Starter Kit".

Perhaps more importantly, buying any "EFR32 Mighty Gecko Wireless Starter Kit" gives the buyer/dweveloer full official access to Silabs Zigbee Stack, SDK, libraries and documentations which should allow you to compile your own firmware for all their chips.

UPDATE! Silabs has now released a new $99 "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) which only contain one mainboard and two low-power radios (Zigbee Green Power compatible):

Silicon Labs SLWRB4180A which is the official EFR32xG21 +20 dBm powerful reference radio board can be bought separately:

https://www.silabs.com/development-tools/wireless/slwrb4180a-efr32xg21-wireless-gecko-radio-board

That same radio board is also included in the more expensive Silicon Labs SLWSTK6006A EFR32xG21 Wireless Gecko Starter Kit:

While the low-power radios are not really usable as Zigbee coordinators (as meant for battery operated endpoint devices), for $99 that kit can be useful to developers as buying such an official kit also gives full access to the Zigbee Stack/SDK and libraries.

Hedda commented 4 years ago

Z-Smart Systems by @cdjackson also has Java code which support EZSP protocol for all Ember dongles at @zsmartsystems

Z Smart System has a Java device driver for Ember based serial dongles (among other dongles):

For reference, Z Smart System also has a sniffer library written in Java which uses same serial interface:

The framework implements a package structure that allows efficient use of re-usable components in a number of different applications.

Package Description
com.zsmartsystems.zigbee The main framework and cluster library implementation
com.zsmartsystems.zigbee.autocode Code generator for the ZigBee cluster library classes
com.zsmartsystems.zigbee.dongle.cc2531 Dongle driver for the Texas Instruments ZNP CC2531
com.zsmartsystems.zigbee.dongle.conbee Dongle driver for the Dresden Electronics Conbee
com.zsmartsystems.zigbee.dongle.ember Dongle driver for the Silabs EZSP Network Co-Processor
com.zsmartsystems.zigbee.dongle.ember.autocode Code generator for the Ember NCP dongle commands
com.zsmartsystems.zigbee.dongle.telegesis Dongle driver for the Telegesis dongle
com.zsmartsystems.zigbee.dongle.telegesis.autocode Code generator for the Telegesis dongle commands
com.zsmartsystems.zigbee.dongle.xbee Dongle driver for the Digi XBee dongle
com.zsmartsystems.zigbee.dongle.xbee.autocode Code generator for the XBee dongle commands
com.zsmartsystems.zigbee.console Console commands for the general framework
com.zsmartsystems.zigbee.console.ember Console commands for the Silabs Ember NCP
com.zsmartsystems.zigbee.console.main Main CLI console application
com.zsmartsystems.zigbee.serial Serial driver implementation
com.zsmartsystems.zigbee.test Overall tests and code coverage

Silicon Labs Ember EM35x / EFR32

The library supports the Silicon Labs EZSP protocol using ASH or SPI protocols over a serial interface. The implementation of the SPI protocol assumes that the SPI provides a TTY-like software interface to the application, or is otherwise abstracted via the ZigBeePort interface.

It is worth noting that EM3588 devices that have an embedded USB core will likely work with any baud rate, where dongles using external USB interface (eg CP2102 used with an EM3581) will likely require a specific baud rate.

Currently there are two main NCP images - one that supports hardware flow control with a baud rate of 115200, and one that supports software flow control with a rate of 57600.

Ember NCP configuration

The library provide a standard set of configuration constants to configure the NCP for use as a coordinator. There are two methods available in the Ember driver to manipulate the configuration maps updateDefaultConfiguration and updateDefaultPolicy. The configuration is sent to the NCP during the initialisation sequence which is performed when calling the ZigBeeNetworkManager.initialize() method, so any changes to configuration must be performed prior to this.

The Ember dongle driver includes a public method getEmberNcp() which returns a EmberNcp class. This class provides high level methods for interacting directly with the NCP.

Ember Reset

By default the library uses the ASH Reset command to reset the NCP when the framework starts. Silabs have stated that this is not reliable due to possible communication problems between the NCP and the host. Where possible the user should implement a hardware reset to reset the NCP. Since this is application specific, the framework provides an interface for the application to implement this reset. The user should implement the EmberNcpResetProvider interface, and set this during NCP initialisation with the ZigBeeDongleEzsp.setEmberNcpResetProvider() method.

Ember MfgLib use

The library provides access to the mfglib functions in the NCP to facilitate device testing. To use this function, create the ZigBeeDongleEzsp and then call the getEmberMfglib(EmberMfglibListener) method to get the EmberMfglib class and also set the callback listener. The EmberMfglib class provides access to the mfglib functions.

Note that this can only be used if the dongle is not configured for use in the network (ie initialize has not been called).

The com.zsmartsystems.zigbee.sniffer project is an example of the use of these features to provide a network sniffer to route frames to Wireshark.

Koenkk commented 4 years ago

Adding a new adapter indeed requires the implementation of the adapter.ts.

Note that adding support for a new adapter is a lot of work, given that we already support 2 powerful adapters (conbee and CC2652R/CC1352P-2 which will also available as USB in the future: https://electrolama.com/projects/zig-a-zig-ah/) I don't see a good time/reward ratio at the moment.

Hedda commented 4 years ago

That is understandable, hopefully someone else might be willing to take on such a project.

My thinking is just that hardware-independence and improved compatibility with multiple Zigbee 3.0 coordinator adapters/dongles from different manufacturers would be beneficial to everyone in the long run.

That is, support for more Zigbee 3.0 coordinator adapters/dongles could democratize to the platform, meaning it makes it more accessible to users around the world, which in turn help drive development by attracting more developers which can contribute to projects like Zigbee2MQTT and ioBroker in the future.

Both Zigbee2MQTT and ioBroker has certainly already achieved a lot in regards to democratizing the entry-point to Zigbee Home Automation with the CC2531 USB-sticks as Zigbee coordinator (which I see Itead now even sells pre-flashed with compatible coordinator firmware at a very low price from China), but improved accessibility in my head also means improved support for enhanced hardware adapters with Zigbee 3.0 capability at lower prices and perhaps more importantly easier to find pre-flashed Zigbee 3.0 adapters with compatible coordinator firmware at low prices all over the world.

IMHO all Zigbee based projects would benefit from lower prices for Zigbee 3.0 coordinator hardware and support for multiple adapters/dongles from different manufacturers, both directly and indirectly, in all open-source projects for home automation.

SillyDay commented 4 years ago

@Hedda Ebyte E180-ZG120A(B) is a Serial port transparent transmission module, running Ebyte private firmware,not EZSP. (However, it can act as a standard ZigBee3.0 router in the Network) For example, I sent AABBCCDDEEFF(not HEX) to the serial port, then I captured Img In other words, the moudule running a Ebyte offical firmware cant't act as a standard coordinator like CC2652R, while it can connect your serial device to ZigBee network easily.(zigbee-herdsman & zigbee-herdsman-converters modification needed)

Hedda commented 4 years ago

@SillyDay Sorry if I was unclear. I suspected that something like would be the case, though I believed that it did not come with any application firmware at all, only a bootloader firmware from the factory.

I am of course suggesting that 'someone' first also need to build/compile a "standard" EmberZNet Zigbee coordinator firmware image for these Silicon Labs EFR32MG12 SoCs using the official Simplicity Studio software from Silabs so that we are able to reflash E180-ZG120B modules and E180-ZG120B-TB boards before they will be able to used with zigbee-herdsman and Zigbee2MQTT and ioBroker as coordinators.

That is, if 'someone' builds/compiles EmberZNet Zigbee coordinator firmware image for these Silicon Labs EFR32MG12 SoCs then those image files can be shared to everyone for flashing your own hardware.

That would be similar concept and principle to how zigbee-herdsman already work with Texas Instruments CC253x & CC26X2R1 SoCs when you flash those with pre-compiled firmware from https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator as the all the CC2530/CC2531 sniffer USB sticks and CC2652R boards can not be used before you flash them either unless you first flash them with a Z-Stack coordinator firmware.

The main issue regarding building a such "standard" Silicon Labs EmberZNet Zigbee coordinator firmware image for an EFR32MG12 SoC is that in order to to get access to the EmberZNet Zigbee stack from the official Simplicity Studio software you first need to buy and register an official development kit for Zigbee from Silicon Labs, to gain license rights to build a firmware with their Zigbee stack so to speak, like example their official devkit called "EFR32 Mighty Gecko Wireless Starter Kit" (article number: SLWSTK6000B) which cost almost $499 US-dollars, though in their defence that kit does contain three (3) complete dev boards/mainboards with six (6) matching EFR32MG12 Zigbee radio modules designed for development.

Or EFR32xG21 Wireless Gecko Starter Kit (SLWSTK6006A) for 3 x EFR32MG21 Series 2 radio boards:

I would have thought that the Ebyte module would most likely only come with standalone bootloader firmware preloaded for boot. Such standalone bootloader firmware usually is just enough to allow you to flash and upgrade the main application firmware via USB (without having to use a JTAG connector). That is, just enough bootloader firmware to allow USB flashing of an application firmware like Silicon Labs EmberZNet Zigbee coordinator firmware image for EFR32MG12 SoCs.

I would think you have to build/compile Zigbee stack (Zigbee EmberZNet) project with coordinator device type config in their SDK ("Simplicity Studio") and then flash application bootloader firmware with Zigbee stack yourself with either "Simplicity Studio" or "Simplicity Commander" or other flashing tools.

You can use Simplicity Studio (Silicon Labs SDK) for non-Zigbee development without buying an official devkit, but I do not think that you can create an EmberZNet Zigbee coordinator firmware image with it without registering an official Zigbee devkit from Silabs as you will not have access to the EmberZNet?

If and when you do have registered an official Zigbee devkit from Silabs so you have access to the EmberZNet Zigbee stack in Simplicity Studio (Silabs SDK) then I believe that these might be a couple of relatively easier to follow how-to guide for building EmberZNet for EFR32 and EM35x firmware, (as I understand they call protocol such as their EmberZNet Zigbee stack for "apps").

I don't own one myself but would imagine that once you bought their official devkit they have made it simple enough start using their latest SDK or other firmware tools (like "Simplicity Studio" and "Simplicity Commander") and it will even detect the hardware once it is plugged in to then start a new project.

Guessing that you first might need to register a serial number together with dev kit article-number to your account for full SDK access, again like for the official "EFR32 Mighty Gecko Wireless Starter Kit" (that kit part number is "SLWSTK6000B") as that official dev board in that dev kit is kind similar to the Ebyte E180-ZG120B-TB eval board as they both use the same type of EFR32MG12 SoC chip in Silicon Labs EFR32 family.

Simplicity Studio

Simplicity Commander

Silicon Labs has its community knowledge-base here:

https://www.silabs.com/community/wireless/zigbee-and-thread/knowledge-base

Silicon Labs also have different user forums here:

Silicon Labs IoT also has a community on Slack:

Silicon Labs IoT also has a community on Gitter:

Hedda commented 4 years ago

Should be noted that Silicon Labs makes it clear that it is OK for multiple developers within a company, organization or team share a single Silabs account for use with their Simplicity Studio SDK.

So if example "team zigbee2mqtt" or "team zigbee-herdsman" (with both Zigbee2MQTT and ioBroker developers) collectively bought an official Silicon Labs Zigbee devkit and register that to a shared account then all developers on that team could, as long as they login with that same account, use the Simplicity Studio SDK with access to EmberZNet Zigbee stack to build Zigbee coordinator firmware images for the EFR32MG12 SoC chip.

Perhaps even Silicon Labs / Silabs would be willing to sponsor Zigbee2MQTT and ioBroker devs with some official EFR32 devkits?

Alternatively, maybe the Zigbee2MQTT and ioBroker community could donate earmarked for a devkit?

I heard that Silicon Labs have donated hardware to Home Assistant developers, so it could also be worth it to try to contact Silicon Labs Salesforce and explain that you would like a logon for an open-source organization/project without owning a dev kit, as an "organization" is a loose term.

After all, projects like Zigbee2MQTT and ioBroker does help Silicon Labs sell chips, and selling chips is the core buisness of Silicon Labs as a company. Guess that the answer to that specific question might differ depending on how Silabs feelings are towards helping open-source projects which are not-for-profit and only help them sell more chips indirectly.

Regardless it would be to their benefit if Zigbee2MQTT and ioBroker via zigbee-herdsman had support for Ember based coordinators as that would both sell more hardware as well as make more developers use/know their software.

Hedda commented 4 years ago

Another tip is this "Zigbee Boot Camp Course" wiki for Silicon Labs @MarkDing has put on GitHub :

He has also put together this PDF with Silicon Labs ZigBee Onboarding Roadmap for beginners:

Hedda commented 4 years ago

@cdjackson from @zsmartsystems also mention two main NCP images in their README.md

Perhaps he would be willing to tip how to build or make EmberZNet Zigbee coordinator firmware for EFR32MG12 SoC chip like the official EFR32 devkit or Ebyte E180-ZG120B / E180-ZG120B-TB?

@cdjackson at least keeps adding new EZSP support from UG100 to com.zsmartsystems.zigbee

cdjackson commented 4 years ago

This statement is probably a little old now - it was made when Silabs provided pre-build firmware images which they no longer do. You now need to compile them for your hardware, so data rates etc are configurable.

It is relatively easy to compile the firmware if you have the SDK etc. I do have a document that I provide to customers but don’t really want to publish it on the net but can answer questions if needed.

Hedda commented 4 years ago

@cdjackson So in your opinion, once someone has registered an official EFR32 devkit to their account for Silicon Labs and installed + login to Simplicity Studio SDK it should be relatively easy for them to build and compile an EmberZNet PRO Zigbee coordinator firmware image file for an EFR32MG12 based module like the E180-ZG120B-TB evaluation board which then could be shared to everyone so that anyone could flash their own Ebyte E180-ZG120B module based hardware with more freely available software? ...again similar to how Koenkk share Z-Stack coordinator firmware images?

cdjackson commented 4 years ago

I’m not familiar with the EByte module so can’t really comment on that. If it is a standard EFR32 under the hood, and doesn’t do anything too non-standard, then yes, it’s reasonably straight forward to create firmware. You will need to know the hardware configuration of the module though (ie what pins of the EFR32 are connected where in the module) - I’m not sure if this information is available?

On 27 Apr 2020, at 10:28, Hedda notifications@github.com wrote:

@cdjackson https://github.com/cdjackson So in your opinion, once someone has registered an official EFR32 devkit to their account for Silicon Labs and installed + login to Simplicity Studio SDK it should be relatively easy for them to build and compile an EmberZNet PRO Zigbee coordinator firmware image file for an EFR32MG12 based module like the E180-ZG120B-TB evaluation board which then could be shared to everyone so that anyone could flash their own Ebyte E180-ZG120B module based hardware with more freely available software? ...again similar to how Koenkk share Z-Stack coordinator firmware images?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Koenkk/zigbee-herdsman/issues/168#issuecomment-619855414, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH6IQ6WBKQ5SR62TMTJYEDROVF5BANCNFSM4MNFW4CQ.

Hedda commented 4 years ago

Ebyte does indeed have public documentation with detailed layouts of the modules pins available.

E180-ZG120B module links:

E180-ZG120B-TB evaluation board links:

These contain a EFR32MG1 SoCs (Series 1) specifically the IC = EFR32MG1B232F256GM48

SillyDay commented 4 years ago

@Hedda I can access the EmberZNet Zigbee stack and try to build a coordinator firmware for EFR32MG1B232F256, but someone have to add support for EmberZNet stack

SillyDay commented 4 years ago

@Hedda Also, programming and read back the firmware is available for E180 series via JLink. I just flashed a customized firmware from EBYTE using my JLink V11.

MPM1107 commented 4 years ago

Well Sonoff made this even more interesting since their new bridge uses the EFR32MG21, so implementing this would add it to the list of supported coordinators as well. @SillyDay could you try and build a firmware file for that chip as well? I don't have one of these Ebyte modules but my Sonoff bridge is about to arrive in the mail, so I could start tinkering with it. I do have some access to prebuilt files myself but they're kinda old. If it's of any importance I think the exact chip they use is the EFR32MG21A020F768IM32-B.

Hedda commented 4 years ago

@SillyDay I understand, thank you! I would great if you or someone could only build Zigbee coordinator firmware images with latest EmberZNet Zigbee stack for the EFR32MG1 Series 1 SoC inside E180-ZG120B and distribute those image files in a GitHub repo or similar location so that everyone could download it without hunting.

Maybe similar to how Koenkk distribute Z-Stack coordinator (and router) firmware but in its own repo?

That would be the first hurdle so that anyone who wants can just buy a cheap E180-ZG120B module or the E180-ZG120B-TB board, download and flash an existing firmware then start developing EZSP interfaces for it (without having to buy the official EFR32 devkit for Zigbee which is relatively expensive).

Is it possible to build separate "debug" and "production" versions of latest EmberZNet firmware?

Anyway, as mention, sharing EmberZNet coordinator firmware could possibly be a great start for latest EmberZNet with Zigbee 3.0 support in more projects than just zigbee-herdsman (Zigbee2MQTT and ioBroker) as other DIY home automation projects could possible also start to develop their own EZSP interface, as long as both inexpensive hardware and compatible firmware is available.

SillyDay commented 4 years ago

@Hedda I have finished building a Customized NCP firmware following AN1010: Building a Customized NCP Application, and posted it on https://github.com/SillyDay/EFR32/blob/master/E180-NCP.bin You can flash this firmware via JLink to E180. In addition, this firmware is untested!

Hedda commented 4 years ago

@MPM1107 FYI, E180-ZG120B is based on EFR32MG12 (EFR32MG1x) making it an EFR32 Series 1.

Sonoff ZBBridge (Sonoff Zigbee Bridge) is based on EFR32MG21 (EFR32MG2x) so it's an EFR32 Series 2.

Confusing naming scheme but as I understand they should still use the same SDK & Zigbee stack, see:

https://www.silabs.com/documents/public/application-notes/an0918.2-efm32-to-efr32xg2x-migration-guide.pdf

EZR32 is also referred to as Series 0

Hedda commented 4 years ago

@SillyDay you should not need to use JTAG hardware for the E180-ZG120B-TB evaluation board, right?

https://github.com/SillyDay/EFR32

EFR32 should come with a bootloader from the factory that should allow software flashing over USB?

The previous generation (Silicon Labs ETRX357) adapters, like example Telegesis ETRX357 USB dongles, have plenty of instructions out there on how to flash firmware over USB with only software.

https://github.com/zigpy/bellows/blob/dev/README.md

https://community.home-assistant.io/t/eu-usb-sticks-for-the-new-zigbee-component/16718/10

Quote bellow originally posted by Frank Aug '17 in linked thread:

I can confirm that the ETRX357USB-LRS+8M can be flashed with an appropriate EZSP firmware without the use of any programming hardware.

So, without any guarantees (you might -but probably won’t- brick your device), here’s how:

Download the firmware. I used some random blob someone put on github that at least seemed to have the right name. What could possibly go wrong, right? :smile: https://github.com/yqyunjie/Zigbee-Project/blob/master/firmware/EmberZNet/EM35x-EZSP/build/em35x-ezsp-images/EM357/em357-ncp-uart-xon-xoff-use-with-serial-uart-bl-500.ebl?raw=true 282

(Install USB-to-serial drivers for the device. Linux will load the driver automatically when you plug in; not sure about other OSes.)

Install a serial port communication app that supports X-MODEM. (debian/ubuntu: sudo apt-get install minicom) Run it and configure it (sudo minicom -s) to use the fake serial port (/dev/ttyUSB0) at 19200 baud 8N1. Disable both hardware (RTS/CTS) and software (XON/XOFF) flow control.

Type ‘AT’ , you should get OK in response. Now type ‘AT+BLOAD’ . The device will reboot into the bootloader.

Change the baud rate in the serial port communication app setting to 115200 baud. (exit minicom using CTRL-A Q, run ‘sudo minicom -s’ again)

Pressing enter in the terminal should now show you a three-option boot loader menu. Choose option 1.

‘C’ characters will start showing. Don’t wait for this to finish, but start an X-MODEM upload of the firmware you downloaded earlier (use CTRL-A S in minicom). You have 60 seconds to start the upload.

After the upload finished, you should return to the menu. Now select option 2, to reboot into the new firmware. You’re done.

SillyDay commented 4 years ago

@Hedda No, the bootloader in E180-ZG120 is not a UART XMODEM Bootloader, so you have to use SWD Interface at least once. BTW, the Bootloader is not included in my firmware.

MPM1107 commented 4 years ago

@SillyDay Yeah I checked the migration guide as well and it says somewhere in there that series 1 and 2 are not binary compatible. But you should still be able to run the exact same code/stack on them, so once implemented zigbee2mqtt shouldn't care which of the two it's using. That's why I specifically asked whether you could compile the firmware for that chip as well. It would also be interesting to then test the range/performance difference between the two series.

SillyDay commented 4 years ago

@MPM1107 Pls let me know the Pin definition of your device, especially TX&RX and you need a NCP firmware for EFR32MG21A020F768IM32 is that right?

MPM1107 commented 4 years ago

@SillyDay Yes, that is the right chip. I did some reversing of the circuitry today, I'm pretty sure I got all of the pins right. Getting hold of documentation for these chinese modules sure is a pain though.

TX: PB01 (connected to ESP8266 U0RXD) RX: PB00 (connected to ESP8266 U0TXD) PA00 connected to ESP8266 GPIO5 DRX: PA06 DTX: PA05 SWDIO: PA02 SWCLK: PA01

So long story short, TX is on PB01 and RX is on PB00. SWDIO and SWCLK are the default pins for them so that's no surprise. I'm not sure what DRX and DTX are for, these are just two pads that are broken out of the module as test pads and not actually used for communication with the main processor. Maybe as a debugging uart console? And then there's PA00, which is connected to one of the GPIO pins of the host processor. I'm not sure what that could be useful for, if you have any ideas feel free to think of a function for that pin.

stevew817 commented 4 years ago

@MPM1107 Do you have the possibility to attach a logic analyser to TX, RX and PA0? If there's only one GPIO signal between the two, it might be any of:

MPM1107 commented 4 years ago

Little update: No changes on that PA0 line in normal operation (even while sending/receiving data), so I'm pretty sure it's for entering the bootloader mode for over the air updates from the host mcu. This is further supported by the fact that there's a physical resistor in place that's pulling the data line up to Vcc which I didn't notice before. Thanks for the suggestions @stevew817

s-hadinger commented 4 years ago

@MPM1107 @SillyDay I did hooked a logic analyzer to the Sonoff Zigbee bridge and I confirm Sonoff does not use EmberZNet firmware. https://github.com/arendst/Tasmota/issues/8583

I will try to flash the firmware, as there are clearly 5 programming pads: Z_RX, Z_TX, SWDIO, SWCLK and Z_RST. IMG_0599

@MPM1107 I'm not familiar for EFR32 flashing, what do I need? Is JLink through RaspberyPI enough?

s-hadinger commented 4 years ago

Contrary to my previous message, the Sonoff Zigbee Bridge does come with Ember firmware already loaded, as it shows when listening to Z_TX

Reset info: 0x03 (EXT)
Extended Reset info: 0x0301 (PIN)
init pass
Build Time Apr 29 2020 17:12:29
EEPROM init: 0x00
EMBER_NETWORK_UP 0x0000
[Z3Coord_callbacks.c:emberAfStackStatusCallback]
status = 0x90
Send Data:
FE 0D 01 80 02 00 00 00 00 01 01 04 74 

Enter password:

InitNetworkEventHandler
PANID = D136
Hedda commented 4 years ago

@s-hadinger It would be absolutely great if that SONOFF ZBBridge (SONOFF Zigbee Bridge) already have Ember firmware already based on EmberZNet Zigbee Stack with their standard EZSP (EmberZNet Serial Protocol) serial protocol interface / API over UART bus.

If it already has Ember firmware with EZSP interfaces then will hopefully not need to worry about any firmware upgrade of in the first place and instead just focus on hacking its ESP8266 instead and maybe offering a serial server service (like Ser2Net) to offer remote access the Zigbee module.

However to answer the question. Yes, If the Zigbee SoC module used in the SONOFF ZBBridge (SONOFF Zigbee Bridge) is at all similar to other EFR32 modules, like the E180-ZG120B by Ebyte, then yes at least according to the Ebyte manual can flash the firmware on EFR32 module with J-Link

JTAG • 4-wire interface for programming and debugging, using the pins TCK, TMS, TDI, and TDO.

SWD (Serial Wire Debug)

In the Ebyte E180-ZG120B manual, the "burning program" sais that GPIO pins for J-Link interface is:

FYI, there's also a related discussion about flashing firmware on these type of EFR32 modules here:

If still are serious about flashing that firmware then checkout a Silicon Labs document called AN136

Maybe it is possible to just use the Command-Line Interface (CLI) of Simplicity Commander as it supports using Serial Wire Debug (SWD) or Joint Test Action Group (JTAG) as the target interface?

AN136 otherwise refers to other documents for many different methods, like SWD, JTAG, and C2:

MarkDing who work at Silicon Labs summarized their Serial Wire Debug (SWD) in their forum here:

The same @MarkDing also maintains a wiki about development with Silicon Labs IoT MCU SoCs:

Hedda commented 4 years ago

All, by the way, here are a few of related discussions specifically about hacking SONOFF ZBBridge:

s-hadinger commented 4 years ago

After more digging, the EFR32MG21 on the Sonoff Zigbee Bridge does have Ember software, but unfortunately it does not have EZSP. It looks like Sonoff implemented its own application on the EFR32 and has its proprietary protocol to the ESP. Trying to connect a non-SONOFF zigbee device to the coordinator what caught by the EFR, but was filtered out and never surfaced to the ESP8266.

I'm afraid we need to re-flash a complete EmberZNet + EZSP software anyways.

Hedda commented 4 years ago

@s-hadinger Good to know, but too bad. Maybe @SillyDay can build a firmware if get pin definitions?

Once the correct pins are routed, there should be a number of tools available to program EFx32 MCUs.

Hedda commented 4 years ago

Makes me wonder though if ITead developers really removed/disabled all the standard EZSP APIs in the firmware for the EFR32 MCU when they added proprietary app with its own protocol? As I understand it, the original EmberZNet app has several EZSP APIs, so maybe they just added an additional app and left the original with the EZSP APIs without even removing/disabling those.

s-hadinger commented 4 years ago

I tried to connect with a JLink but I'm getting an error:

 - Connecting via USB to probe/ programmer device 0
 - Probe/ Programmer firmware: J-Link EDU Mini V1 compiled May 26 2020 15:04:19
 - Device "EFR32MG21AXXXF1024" selected.
 - Target interface speed: 1000 kHz (Fixed)
 - VTarget = 3.293V
 - Found SW-DP with ID 0x6BA02477
 - DPIDR: 0x6BA02477
 - Scanning AP map to find all available APs
 - AP[3]: Stopped AP scan as end of AP map has been reached
 - AP[0]: AHB-AP (IDR: 0x84770001)
 - AP[1]: APB-AP (IDR: 0x54770002)
 - AP[2]: AHB-AP (IDR: 0x84770001)
 - Iterating through AP map to find AHB-AP to use
 - AP[0]: Skipped. Could not read CPUID register
 - AP[1]: Skipped. Not an AHB-AP
 - AP[2]: Skipped. Could not read CPUID register
 - Found SW-DP with ID 0x6BA02477
 - DPIDR: 0x6BA02477
 - Scanning AP map to find all available APs
 - AP[3]: Stopped AP scan as end of AP map has been reached
 - AP[0]: AHB-AP (IDR: 0x84770001)
 - AP[1]: APB-AP (IDR: 0x54770002)
 - AP[2]: AHB-AP (IDR: 0x84770001)
 - Iterating through AP map to find AHB-AP to use
 - AP[0]: Skipped. Could not read CPUID register
 - AP[1]: Skipped. Not an AHB-AP
 - AP[2]: Skipped. Could not read CPUID register
 - ERROR: Could not find core in Coresight setup
 - ERROR: Failed to connect.

It looks similar to error here: https://forum.segger.com/index.php/Thread/6521-SOLVED-Can-t-connect-to-Silicon-Labs-EFR32MG21A010F1024/

Any idea from someone who knows JLink?

stevew817 commented 4 years ago

Debug interface might be locked down.

s-hadinger commented 4 years ago

Confirmed, Simplicity Studio gives more details. The device is debug-locked, but not erased lock. It means that we can erase it completely and push a new firmware (secure boot is not enabled so firmware doesn't need to be signed):

EFR32
MPM1107 commented 4 years ago

I'm guessing that itead has implemented a way to run ota firmware updates, so the gecko bootloader is probably available, right? Correct me if I'm wrong but that should make it possible for anyone to flash new firmware over the esp8266 instead of a debugger probe.

s-hadinger commented 4 years ago

Yes probably. Interestingly there is a 24C512 chip attached to the ESP8266, which is a I2C 512KB EEPROM with 2 POGO pads to flash it. So I guess it's goal is to store a copy of the EFR32 firmware so that the ESP can flash it.

What I'm concerned with is that it requires an unlock key. If we erase the chip, does it erase the bootloader as well?

MPM1107 commented 4 years ago

Oh that's interesting. From what i can see from the pictures the second chip is a FM25Q16A, so flash for the esp8266. That would indeed mean that the EEPROM has soem zigbee function. Another guess would be that it stores information about all of the devices that are allowed to connect to the bridge. I don't know a lot about these chips, but from the screenshot you posted above I can see that there's no sign key written to the device. My guess is that this means we don't actually have to erase the chip, right? If there's no key to validate the firmware signature we should be able to just push new firmware using the bootloader, without erasing the chip first.

s-hadinger commented 4 years ago

512KB for Zigbee information would be huge, considering that the SPI Flash is already 4MB. It is surprising that Sonoff didn't use the 4MB Flash instead.

This document describes the Debug-lock behavior: https://www.silabs.com/documents/public/application-notes/an1190-efr32-secure-debug.pdf

Indeed they don't mention that debug-lock prevents flashing new software.

Also erasing the EFR32 does not erase the bootloader as described here: https://www.silabs.com/documents/public/application-notes/an1190-efr32-secure-debug.pdf

Looking at the PCB, there are only 4 wires connected between EFR32 and ESP8266:

MPM1107 commented 4 years ago

Looking at this doc, page 3: https://www.silabs.com/documents/public/application-notes/an706-ezsp-uart-host-interfacing-guide.pdf it's almost certain that the GPIO5 connection is for entering bootloader mode. That fact is confirmed even further by the pullup resistor that I can see on that line, and the fact that the only other way of entering bootloader mode is through EZSP communication as far as I can tell from the documentation, and we have already determined that EZSP is not used in this firmware. So itead has to have a data line for entering the bootloader if they want to execute remote firmware updates.

s-hadinger commented 4 years ago

It makes sense. Next step, I will flash a Tasmota on the ESP and try to enter bootloader mode.

stevew817 commented 4 years ago

Also erasing the EFR32 does not erase the bootloader as described here: https://www.silabs.com/documents/public/application-notes/an1190-efr32-secure-debug.pdf

Fair warning, erasing the EFR32 series 2 does also erase the bootloader, since the bootloader would be a part of main flash that is mass-erased as part of the 'standard debug unlock' or 'device erase' commands.

Hedda commented 4 years ago

So could also be possible unlock debug access on EFR32 via CLI from ESP8266 with custom firmware?

That is if debug unlock don't need a custom key to unlock or you can get that key from ITead/Sonoff?

Hedda commented 4 years ago

Do you think it will be possible to flash from a custom ESP8266 firmware over UART or OTA?

Do you think it's maybe possible to access the EFR32's SWD interface from ESP8266 over UART?

And are there any open-source tools that could flash EFx32 over UART or OTA from ESP8266?

I only found this discussion which talks about debugging and flashing with OpenOCD and GDB:

Guess I'm getting way ahead but nice to know if anyone has experience with EFR32 or EFM32. Just very excited to see a few DIY hackers interest in this ESP8266-based Zigbee bridge so soon!

s-hadinger commented 4 years ago

Do you think it will be possible to flash from a custom ESP8266 firmware over UART or OTA?

That's what we hope with the Gecko bootloader.

Do you think it's maybe possible to access the EFR32's SWD interface from ESP8266 over UART?

Definitely no. The SWD interface is not electrically connected to the ESP8266. It is only available through the pads.

And are there any open-source tools that could flash EFx32 over UART or OTA from ESP8266?

The protocol is XMODEM which is fairly simple. If the Gecko bootloader would allow us.

s-hadinger commented 4 years ago

Sadly I'm currently stuck. I wasn't able to flash Tasmota to the ESP8266EX. After many tentatives and trial/error, it looks like the Flash SPI chip has an issue with sector erase. Some bits are not properly erased and keep flipping. I didn't find anything in the data sheet that could explain it, beyond a defective chip.

I will probably need to unsolder it and replace it. Unless someone has a better idea.

MPM1107 commented 4 years ago

Are you getting any errors while erasing? Also, you say that it has issues with sector erase, did you try a full chip erase as well?

s-hadinger commented 4 years ago

I'm not getting any error when erasing, but reading back shows that although most bytes are 0xFF, some bits are 0. And among them some bits are flipping between reads.

When I dumped the original Sonoff firmware, I did it twice and got exactly the same result. This makes me think that the corruption is internal to the SPI Flash chip.

When I write only 0x00 to a sector, the result is good. This makes we think the erasing phase is not working well.

Full flash erase completes in 2.1 seconds without error and does nothing. The data sheet says it should last 7 seconds. This is weird too.