search

Koenkk / zigbee-herdsman

A Node.js Zigbee library
MIT License
485 stars 301 forks source link

[REQUEST] Support for Silicon Labs EFR32 (Mighty Gecko family) based Zigbee radio adapters using EZSP serial protocols? #168

Closed Hedda closed 4 years ago

Hedda commented 4 years ago

Will you consider adding support for Silicon Labs Zigbee cordinator dongles/sticks via EZSP (EmberZNet Serial Protocol) CLI/API?

It is maybe possible for you to port the EZSP (EmberZNet Serial Protocol) code from the bellows library to zigbee-herdsman (in a similar manner to how zigpy-cc which was ported from zigbee-herdsman)?

I understand that the bellows library itself was based Silabs official EZSP Reference Guides from Silicon Labs, like UG100:

PS: Silicon Labs EFR32 family replaces its old ETR35x however both are Ember NCP and support the same EmberZNet Zigbee stack.

MPM1107 commented 4 years ago

What about just trying to flash a "blank" bin file then? Stupid and hacky, but maybe the chip just has some weird ways to erase it that the flash tool doesn't quite understand? If you flash a 0xFF-filled bin and no bits are flipped that should be a good sign that at least the chip is able to hold data properly.

s-hadinger commented 4 years ago

I tried too. I flashed a 900KB file full of 0xFF. But the problem remains the same which is understandable. Flashing 0xFF means erasing and writing nothing. Only flashing 0x00 means actually writing bits after erasing.

I also tried flashing a 4KB file with only 0xFF. Since esptoool.py zips the content before sending to UART, I had the proof that it was not a UART data corruption during the file transfer.

Edit: I also issued a read_flash_status and both bytes are 0x00 which means that the chip is not write protected.

Issuing a flash_id commands gives the following, which are ids of the FM25Q16A: Manufacturer: a1 Device: 4015

MPM1107 commented 4 years ago

Ah, so it's a chinese clone of the Winbond flash modules used by many of the little esp8266 boards. Source: https://chromium.googlesource.com/chromiumos/third_party/flashrom/+/master/flashchips.h

I'm not quite sure what to say at this point. It would be weird if you got unlucky enough to receive a module with dead flash. It does seem like an error somewhere but who knows where it comes from.

larena1 commented 4 years ago

Maybe try asking Sonoff?

s-hadinger commented 4 years ago

The numbers do match with FM25Q16A data sheet: FM25Q16A

Hedda commented 4 years ago

Again, WeMos D1 (all models?) should have the same type of "ESP8266EX" as Sonoff ZBBridge so could possibly be the perfect hardware for creating alternative development kit platforms for such a project?

As for the Zigbee MCU SoC part it seems to currently be a bit harder to find inexpensive alternative hardware modules with Silicon Labs EFR32 Series 2 (EFR32MG21) MCU SoC if you do not want to buy Silabs official EFR32xG21 Wireless Gecko Starter Kit (SLWSTK6006A) for $479 US-dollars on your own:

Though three developers could co-purchase such a kit to split as it contains three complete development boards and EFR32MG21 Zigbee radio modules, plus a bonus is that purchasing the starter kit also it gives your Silabs registered account access to the Zigbee stack which is not publicly available otherwise.

SLWSTK6006A kit is sold worldwide by Farnell, Digikey, Mouser, Arrow/Verical, Newark, and Element14.

However, if it would be good enough to use a Silicon Labs EFR32 Series 1 (EFR32MG12) module in a DIY development kit instead, if so then check out this Ebyte E180-ZG120B module and the E180-ZG120B-TB (EFR32 Test Board) which can both be found relatively inexpensive from ex. Aliexpress or eBay.

At least Silicon Labs seem to use the same SDK and Zigbee stack for all EFR32; EFR32 Mighty Gecko Wireless SoC Series 1 = EFR32MG1/EFR32MG12/EFR32MG13/EFR32MG14, and EFR32 Wireless Gecko Series 2 =EFR32MG21 "Mighty Gecko". Same Zigbee API too however a newer version API.

Hardware-wise all Wireless Gecko Series 1 MCUs obviously has less RAM and FLASH memory as well as being based on Arm Cortex-M4 instead of an Arm Cortex M33 CPU that Wireless Gecko Series 2 has.

FYI, alternative hardware platforms for development platforms was also suggested/discussed here:

MPM1107 commented 4 years ago

I don't really see a need for an alternative development board, since we're not doing any hardware development here. Having access to the zigbee stack would be nice, but the price of that kit is really high and as far as I know we already have members here that have access to it.

Hedda commented 4 years ago

FYI, @mtx512 posted now in https://github.com/arendst/Tasmota/issues/8583 saying that he also has access to Silicon Labs SDK and EmberZNet Zigbee Stack and has offered his help to Tasmota developers to build compatible Zigbee coordinator firmware image for the EFR32MG21 NCP that is in the Sonoff ZBBridge hardware. Hopefully, he might also willing to publicly share those firmware images via GitHub repo like @SillyDay

s-hadinger commented 4 years ago

Thanks to @grobasoz we now have the complete pinout map (cross-posting from Tasmota issue):

The complete EFR32 module pinout and ESP8266 connection is as follows:

Additional pads for ESP8266:

Gamester17 commented 4 years ago

Maybe try asking Sonoff?

ITead (Sonoff) as a company has benefited a lot from alternative firmware from the community in the past and there are today more aware of that fact so would not hurt for you to ask.

Suggest that you or any other developers from here or Tasmota try to send polite but direct emails to "Jerry Shi" <jerry.shi 'at' itead.cc> and "Stan Li" <stan.li 'at' itead.cc> and explain that you are an open-source developer from the Tasmota community that is looking into developing an alternative firmware for the Sonoff ZBBridge and ask if they mind forwarding questions to their engineering department.

I have no affiliations to ITead/Sonoff myself however in the past I have sent a few questions or suggestions to them and got relatively quick replies from Jerry Shi and Stan Li. I think that Jerry is in marketing at ITead so he has to pass on the technical questions to the developers, however, I know he helped the community in the past and since he is in marketing I assume he would especially helpful if it directly or indirectly helps ITead sell more products, which is something that the Tasmota firmware surely already does.

s-hadinger commented 4 years ago

[Cross-posting] Back to business, I flashed a second Sonoff device successfully. The first had definitely a faulty flash memory.

And booting the EFR32 with PA00 low, here is what I see on the console output:


Gecko Bootloader v1.9.1.04
1. upload gbl
2. run
3. ebl info
BL >

Tada! There is a usable bootloader

MPM1107 commented 4 years ago

Man that faulty flash sure is unlucky. I hope so at least, let's not have every 3rd unit have a fault like that. Also hurray, that's great news! If you can upload an EZSP enabled firmware over the bootloader we're golden. My bridges are still on their way, hopefully it won't take much longer.

s-hadinger commented 4 years ago

No doubt we can flash a new firmware on the EFR32, and @mtx512 did one already. But I need now to pass XMODEM through the ESP8266. I need to study that.

MPM1107 commented 4 years ago

Maybe by flashing esp-link onto the esp8266 and using that as a serial over wifi bridge. Then you can just flash from your PC as if the EFR was connected to your PC over serial. This of course wouldn't be the way to do it in the final implementation, but maybe it's good enough for testing?

Hedda commented 4 years ago

Maybe by flashing esp-link onto the esp8266 and using that as a serial over wifi bridge. Then you can just flash from your PC as if the EFR was connected to your PC over serial. This of course wouldn't be the way to do it in the final implementation, but maybe it's good enough for testing?

Serial Bridge functionality (via a transparent serial network server like example ser2net and socat) for serial access to the Zigbee MCU over the network would be awesome to get working as soon as possible.

Serial Bridge function would be great if it could be prioritized as that would allow those developers more interested in only deep-diving into Zigbee MCU part for and not so much have to worry about the ESP8266 part, leaving that part to those developers more interested in that.

The reason I recommended esp-link over anything else is that it has the option to reset the zigbee chip and pull down the bootloader pin every time a new telnet connection is made to a special port instead of the default one. That allows for easy firmware upgrades of the zigbee chip without the need for a swd probe.

As @MPM1107 mentioned, esp-link by JeeLabs is a firmware for ESP8266 that has "Serial Bridge" (a.k.a. "Serial over IP") as its main function, though many other alternative firmware for ESP8266 do also have some kind of Serial Bridge implementation, (including Tasmota), however esp-link has the serial bridge feature as its core goal:

Request for esp-link specific discussion:

Also check out:

s-hadinger commented 4 years ago

Thanks, I actually added Serial bridge to Tasmota too, as it will be very useful for flashing later on. It will also greatly simplify trial/error.

s-hadinger commented 4 years ago

Quick update, huge thanks to @mtx512, I now have a device with Tasmota and EmberZNet flashed, with support for EZSP. Now the fun really begins.

We will need to find an easier way to flash the EFR32, I had to use the JTag interface.

@Koenkk happy to share the details of EZSP progress in the near future.

MPM1107 commented 4 years ago

@s-hadinger Just to clarify any confusion because of all the cross-posting going on, are you going to work on ezsp support on the tasmota or on the z2m side of things? Just so we don't have multiple people separately working on the same thing

s-hadinger commented 4 years ago

I will only work on Tasmota, not z2m. But I'm we will surely save time by sharing issues and information.

Also Tasmota now supports TCP Serial bridge so it can be used to tunnel efr32 to z2m, as an alternative to Z2T.

MPM1107 commented 4 years ago

Perfect! My plan was to only work on the z2m side so that means we won't be wasting resources having two people do one thing. I'm sure there are many tips and tricks about the protocol that will be interesting for both sides so I'll make sure to share anything interesting I come across.

Hedda commented 4 years ago

I actually added Serial bridge to Tasmota too, as it will be very useful for flashing later on. It will also greatly simplify trial/error.

@s-hadinger What is the status on your TCP Serial Bridge for this in Tasmota? Will that be merged into Tasmota's dev or master-branch and as such available as a standard feature in Tasmota mainline or?

MattWestb commented 4 years ago

@MPM1107 Little info on the cracy swedes module with MG1 https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Modul. Only need "disassembling" one brand new E27 bulb (or one On / Off Switch, wos a kit with both for 9.99 €) and asking @mtx512 for compiling the firmware.

Hedda commented 4 years ago

FYI; @s-hadinger now added that TCP/IP serial bridge to Tasmota firmware for remote access:

Description:

Add a new feature: Serial to TCP bridge (similar to esp-link). This allows to remotely communicate to a MCU through ESP8266. Needs #define USE_TCP_BRIDGE

It adds 2 GPIO types: TCP Tx (208) and TCP Rx (209) and can work with hardware or software serial.

Commands:

  • TCPBaudRate <x>: sets the baudrate for serial (only 8N1 mode), min 1200, max 115200 by 1200 increments.
  • TCPStart <port>: listens to port <port>. This features supports 2 parallel TCP connexions, which can be useful if you need a terminal + a specific protocol (like XMODEM). The 3rd connection will disconnect an previous connection. The number of parallel connections is a compile-time option.
  • TCPStart 0 or TCPStart: shuts down the TCP server and disconnects any existing connection.

For security reasons, the TCP bridge is not started at boot, and requires an explicit TCPStart command (can be automated with Rules).

@mtx512 also built a custom ZNet EZSP NCP firmware for the EFR32 module inside Sonoff ZBBridge:

Hedda commented 4 years ago

@MattWestb wrote:

@MPM1107 Little info on the crazy swedes module with MG1 https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Modul. Only need "disassembling" one brand new E27 bulb (or one On / Off Switch, wos a kit with both for 9.99 €) and asking @mtx512 for compiling the firmware.

Cool! I see that @mtx512 now also built a custom ZNet EZSP NCP firmware for ICC-A-1 module too:

FYI: @basilfx has more info on disassembling IKEA Trådfri bulbs to extract an "ICC-A-1" EFR32 module:

MattWestb commented 4 years ago

@MPM1107 I have finding one EFR32MG1 RPi scheeld that having a Ubuntu C-library package, Node.js NPM package and Python library package. Elelabs Zigbee Raspberry Pi Shield https://elelabs.com/products/elelabs-zigbee-shield.html. And the Elelabs Zigbee USB Adapter has a Mighty Gecko (EFR32MG1) https://elelabs.com/products/elelabs-usb-adapter.html with the same software pack.

MPM1107 commented 4 years ago

@MattWestb That's a really cool find! Their node library is published on npm and looks pretty useful. It uses the EZSP protocol so it's just what we need. That also means that this shield and USB dongle will work with zigbee2mqtt once I'm done implementing the new adapter. I'll see if this library can help reduce the effort required or at least help me with parts of the code.

grobasoz commented 4 years ago

@MPM1107 You mentioned "Their node library is published on npm and looks pretty useful"

Could you please let me know the package name? I can seem to locate it.

Thanks, Gary

MPM1107 commented 4 years ago

@grobasoz It's called node-ezsp: https://www.npmjs.com/package/node-ezsp

grobasoz commented 4 years ago

@MPM1107 Thanks - will try it now...

Hedda commented 4 years ago

@mtx512 Do you think these can be also flashed with later EZSP NCP firmware if you build it for them?

Elelabs Zigbee Raspberry Pi Shield => https://elelabs.com/products/elelabs-zigbee-shield.html

Elelabs Zigbee USB Adapter => https://elelabs.com/products/elelabs-usb-adapter.html

I have found one EFR32MG1 RPi shield that has a Ubuntu C-library package, Node.js NPM package and Python library package. Elelabs Zigbee Raspberry Pi Shield https://elelabs.com/products/elelabs-zigbee-shield.html And the Elelabs Zigbee USB Adapter has a Mighty Gecko (EFR32MG1) https://elelabs.com/products/elelabs-usb-adapter.html with the same software pack.

I thought that Elelabs had not published exactly which chip type they were using other than they were from Silicon Labs, so I had no idea that they are based on EFR32 MG1 (EFR32 Mighty Gecko Series 1).

I was sure that Elelabs only marketed those as alternatives Nortek GoControl QuickStick Combo Model HUSBZB-1 if you only want a Zigbee adapter compatible the bellows library as used in the ZHA integration component for Home Assistant. I know that the fact that HUSBZB-1 is not sold outside the United States was the main reason that Elelabs started making those, at least they at first have been targeting Home Assistant's ZHA users, and later also openHAB users.

Regardless Elelabs must only ship an old firmware version for them since it is compatible with the bellows library out-of-the-box and the fact is that the bellows library is currently only compatible with the older versions of EZSP protocols.

Hedda commented 4 years ago

FYI; @cdjackson maintains openHAB Zigbee bindings for "Ember" (EZSP NCP), inc. Elelabs adapters:

openHAB's implementation is based on Z-Smart Systems ZigBee Libraries from @zsmartsystems

MattWestb commented 4 years ago

Its verified that 6.6.4.0 its working in compatibility mode https://sprut.ai/client/article/2583 . If 6.6.4.0 its old you sud taking a look on the other firmware for example IT CCs. From 6.6.4.0s release note:
"These release notes cover SDK version(s): 6.6.4.0 released September 13, 2019." The 6..6.7.0 its made from Zigbee R22 specification and i think also 6.6.4.0.
Zigpy is also looking upgrading to support newer version of the stack but after 6.6.7.0 (zigbee R23) mutch old hardware support its going away so must have dual stack protocol for not braking all that have being made.

mtx512 commented 4 years ago

@mtx512 Do you think these can be also flashed with later EZSP NCP firmware if you build it for them?

Elelabs Zigbee Raspberry Pi Shield => https://elelabs.com/products/elelabs-zigbee-shield.html

Elelabs Zigbee USB Adapter => https://elelabs.com/products/elelabs-usb-adapter.html

Would need to know the EFR32 part number first.

Hedda commented 4 years ago

Noticed that Silabs just released a new $99 "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) which only contain one mainboard and two low-power radios (Zigbee Green Power compatible):

(Google search shows that relatively inexpensive kit can already be bought from resellers globally).

While such low-power radios are not really usable as Zigbee coordinators, for $99 that kit can be useful to developers if buying such an official kit also gives full access to the Zigbee Stack/SDK and libraries.

That fact is still unknown for that new $99 kit, however, we do know that buying the latest $479 "EFR32xG21 Wireless Gecko Starter Kit" (article: SLWSTK6006A) do include full Zigbee SDK access.

It contains three starter kits and includes full access Silicon Labs Zigbee SDK after register on Silabs.

jnicolson commented 4 years ago

Noticed that Silabs just released a new $99 "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) which only contain one mainboard and two low-power radios (Zigbee Green Power compatible):

(Google search shows that relatively inexpensive kit can already be bought from resellers globally).

While such low-power radios are not really usable as Zigbee coordinators, for $99 that kit can be useful to developers if buying such an official kit also gives you full access to the Zigbee SDK and libraries.

That fact is still unknown for that new $99 kit, however, we do know that buying the latest $479 "EFR32xG21 Wireless Gecko Starter Kit" (article: SLWSTK6006A) do include full Zigbee SDK access.

It contains three starter kits and includes full access Silicon Labs Zigbee SDK after register on Silabs.

I can confirm this kit does give access to the Zigbee Stack - I contacted SiLabs support specifically asking this question and got confirmation it does. I then purchased one and have access to the Zigbee stack.

If anyone needs a custom firmware compiled then i'm also happy to help - i'd love to see SiLabs support in zigbee2mqtt. I was actually going to ask if anyone is actively developing this and if there's any input i can provide - i'm not the greatest programmer but can normally make things work if given enough time so maybe i can be of some use.

grobasoz commented 4 years ago

@jnicolson - Which kit are you talking about exactly? I recently saw mention on the forum of the $499 kit being the only one that permits ZigBee access? It would be great if SL does offer a lower (than $499) cost entrance into EmberZnet development... I have already asked but will ask SL again just to make sure...

Hedda commented 4 years ago

Which kit are you talking about exactly? I recently saw mention on the forum of the $499 kit being the only one that permits ZigBee access? It would be great if SL does offer a lower (than $499) cost entrance into EmberZnet development... I have already asked but will ask SL again just to make sure...

@grobasoz Please see posts above yours; it's the new "SLWSTK6021A" low-power kit with only one motherboard for $99 versus existing "SLWSTK6006A" high-power kit with three motherboards for $479.

grobasoz commented 4 years ago

@Hedda - thanks for that. Seems like they are offering access to ZigBee with the SLWSTK6021A kit! That is good news.

jnicolson commented 4 years ago

I actually got interested in Zigbee for DIY client devices but I struggled to understand the NXP SDK for the JN5169 (used inside a lot of Xiaomi devices) and I read that the SiLabs app builder was much easier - it definitely is! The SLWSTK6021A kit has made it much more accessible and the Thunderboard Sense 2 also has a zigbee capable chip on it and it's even cheaper once you have SDK access.

The elelabs raspberry pi board looks really interesting though..

MPM1107 commented 4 years ago

@jnicolson Yeah I've been working on z2m support for EZSP in the little free time that I have over the past few days. I've got the frame builder/parser mostly done, just working on implementing the protocol stuff now. Your willingness to help is certainly appreciated. If nothing else having people with access to the zigbee stack available here is awesome, as my current financial situation doesn't really allow me to throw out 100 bucks for a dev board just to get access to the stack.

mtx512 commented 4 years ago

The SLWSTK6021A ($99) comes with EFR32MG22 modules so it only EmberZNet 6.7.3.0 or greater. BTW EFR32MG22 parts will be priced competitively so should see them in end devices this year or early next . Downside these parts support secure boot so bootloaders may not be easily hacked.

Adminiuga commented 4 years ago

@mtx512 but do you get access to older sdk versions?

jnicolson commented 4 years ago

@mtx512 but do you get access to older sdk versions?

Yes - I see 6.7.6.0 as the latest and going back to 5.7.2.0 as the oldest, with many in between. This is from registering the SLWSTK6021A kit ($99).

Hedda commented 4 years ago

So new EFR32MG22 require Zigbee EmberZNet SDK 6.7.4.0 or later, but EFR32MG21 support 6.5 to 6.7?

EFR32MG12 supports EmberZNet 6.0 to 6.7 or higher but has more limited storage, RAM, and CPU?

jnicolson commented 4 years ago

@Hedda The release notes show that EFR32MG22 have been supported since 6.7.3.0, so essentially yes. EFR32xG21 are listed as supported in 6.5.3.0.

EFR32MG12 seems to have been added in 5.9.0.0 so quite a bit longer support. From looking at the specs, the EFM32MG12 seems to have the highest ARM core (M4 + FPU), the most flash (1024kB) and the most memory (up to 256kB) but doesn't have some security features (like Secure Boot) and i would guess the power consumption is probably worse (didn't look in great detail in the power consumption though).

I'm just a hobbyist so am by no means an expert and hopefully someone with more knowledge can confirm/correct.

s-hadinger commented 4 years ago

I'm currently running 6.7.6.0 of EFR32MG21, using protocol v8. The stack is correctly initialized, based on the work from bellows, the network is up and EFR32 is acting as coordinator. But I'm struggling to have devices pairing to the coordinator.

My first attempts showed a tentative of devices (Xiaomi and OSRAM) to pair, and unpairing because of a TC key issue. Now I'm not receiving any message when trying to pair. I'm stuck...

NilsBohr commented 4 years ago

Hello guys,

That's Kit from Elelabs.

I thought that Elelabs had not published exactly which chip type they were using other than they were from Silicon Labs, so I had no idea that they are based on EFR32 MG1 (EFR32 Mighty Gecko Series 1).

Jumping here in the discussion, if you don't mind.

We had sold:

Yes, we are shipping it by default with EZSP protocol v6. But we are really happy to provide a v7 or v8 firmware update if you are interested.

mtx512 commented 4 years ago

@NilsBohr few questions about the ELR023:

  1. Are JTAG or GPIO pins available on the 10 pin RPI connector.
  2. EFR32MG1 64Kb Ram?
NilsBohr commented 4 years ago

@mtx512 The Raspberry shield also had different models:

The JTAG or GPIO pins are not available on the RPI connector. Only UART.

But the Firmware update can be done using UART, (in case of USB ELU013 through Serial port) so no need to solder the programmer.

mtx512 commented 4 years ago

@NilsBohr shame no jtag or gpio as the board would have been useful to flash customised firmware for other use cases since EFR32MG13P732 supports BLE/Zigee/Thread.