search

xoseperez / espurna

Home automation firmware for ESP8266-based devices
http://tinkerman.cat
GNU General Public License v3.0
2.98k stars 635 forks source link

[REQUEST] ESPurna firmware on Sonoff ZBBridge (Sonoff Zigbee Bridge) #2224

Open Hedda opened 4 years ago

Hedda commented 4 years ago

Please consider porting ESPurna to the new Sonoff Zigbee Bridge (Sonoff ZBBridge)

Itead has just launched Sonoff ZBBridge as an inexpensive Sonoff Zigbee Bridge which is somewhat based on a similar concept/idea as the Sonoff RF Bridge (which I believe your firmware already has support for?).

According to the teardown on notenoughtech.com it sounds as if it is based on Silicon Labs EFR32 (Mighty Gecko) for Zigbee 3.0 radio module support and ESP8266 ("ESP8266EX") for WiFi and bridge/gateway/controller software

As you probably already know, Tasmota does something similar with Zigbee2Tasmota but by connecting an ESP8266 to a Texas Instrument CC2530 module instead however it too is using a serial communication protocol, see:

From the images in notenoughtech.com teardown, it even looks like Sonoff are reusing the same injection moulds as for the Sonoff RF Bridge housing/enclosure which Tasmota already supports

EZSP serial protocol interface that Silicon Labs uses is also well documented and already used by open source projects, see example the bellows radio library written in Python (which is used by Home Assistant's ZHA integration component via the zigpy library):

https://www.silabs.com/documents/public/user-guides/ug100-ezsp-reference-guide.pdf

Z Smart System also has a Java device driver for Ember based serial dongles (among other dongles):

For reference, Z Smart System also has a sniffer library written in Java which uses same serial interface:

Hedda commented 4 years ago

FYI, The alternative today is to build your own Zigbee2Tasmota bridge with an ESP8266 and a TI CC2530 module, but that is not a complete hardware solution and the Zigbee module included in Sonoff ZBBridge is much more powerful than TI CC2530, plus the EFR32 based module in the Sonoff ZBBridge supports Zigbee 3.0 while TI CC253x only supports Zigbee Home Automation (HA) 1.2

xoseperez commented 4 years ago

This is an interesting device, very much in the line we should go (less WiFi devices cluttering our home networks). My current approach is using zigbee2mqtt with a CC2531 which provides the same functionality used with NodeRED or alike.

Hedda commented 4 years ago

@xoseperez Have you by any chance tested Zigbee2Tasmota (a.k.a. Z2T) as a Zigbee to MQTT bridge?

https://tasmota.github.io/docs/Zigbee/

It is similar to Zigbee2mqtt but you connect a CC2530 module to an ESP8266 via serial instead.

Problem with Zigbee2Tasmota is that device translation is not as complete as it is in Zigbee2mqtt.

Using Zigbee2Tasmota with example Home Assistant is for example not as simple as Zigbee2mqtt.

Hedda commented 4 years ago

One workaround for Home Assistant users could be to simply present a Zigbee radio module as a remote adapter to Home Assistant's ZHA (their Zigbee Home Automation integration component)?

You could use something like ser2net to allow ZHA to remote connect to it via serial over network, like:

Then you need a radio library for zigpy that works with the serial protocol used. For example with that Sonoff ZBBridge you would need a zigpy radio library for EZSP protocol used by Silicon Labs EFR32

Hopefully, their bellows radio library should already be compatible or could be made to be if updated

zigpy devs already do something similar to connect remotely with ESP8266 based "ZiGate Pack WiFi"

As proof-of-concept you could do this with a CC2530 remote to zigpy-cc from a ESP8266 using ser2net

https://github.com/zigpy/zigpy-cc

xoseperez commented 4 years ago

Haven't tested Zigbee2Tasmota, my setup is zigbee2mqtt running in a docker container on a server. I'm trying to move more and more things to higher level layers in my setup (keeping them in my network, that is). The dumber the devices the better.

Hedda commented 4 years ago

So then it might be a smart idea to have a Sonoff ZBBridge with ESPurna running some kind of basic serial server (like ser2net or similar) in order to allow remote connection from bellows and zigpy or?

Ex: ZHA component (Home Assistant) -> zigpy -> bellows -> ESPurna with ser2net on Sonoff ZBBridge

At least from an end-user perspective it could be made easy to just enter IP and port in ZHA config.

Again, that is kind of what is needed from an end-user perspective to setup the "ZiGate Pack WiFi".

Benefit with Sonoff ZBBridge is its nicley packaged and has a much more powerful Zigbee 3.0 radio

Hedda commented 4 years ago

Do you think that you could maybe use the serial bridge from esp-link for a serial forwarding feature?

Hedda commented 4 years ago

ESPEasy (ESP Easy) can otherwise be used as a generic Ser2Net gateway (serial port wrapper)

Hedda commented 4 years ago

oxan slook to have a serial server solutions for ESPHome firmware in https://github.com/esphome/feature-requests/issues/660

mcspr commented 4 years ago

Also note that we probably need these if we want to implement EZSP UART protocol: https://www.silabs.com/documents/public/user-guides/ug100-ezsp-reference-guide.pdf https://www.silabs.com/documents/public/application-notes/an706-ezsp-uart-host-interfacing-guide.pdf (Assuming that Itead used Silabs SDK and built apps based on the things there, reviews of the zibgee bridge only describe EWLink side. We don't want that :) Hardware is missing, still. Need to wait until May / June?

If following the serial bridge route... note of the serial<->tcp library at #1314. idk how extended was the testing though, we do need to make sure it does not break when networking goes down / up unexpectedly and there are no issues with missing packets (that were mentioned somewhere upstream) We don't have the v2 model too, I think the way to go is to move Serial / Serial1 / SWSerial management into a separate entity. That way we can configure serial once and 'attach' it to any consumer that wants it as generic Arduino Stream ('thing' that we can read and write into). That would help with GPIO locking too.

Hedda commented 4 years ago

Almost any ESP8266 development board (like a NodeMCU or a Wemos D1 Mini) with a Silicon Labs EFR32MG based Zigbee module like the Ebyte E180-ZG120B would just about be about the equivalent to having a hacked Sonoff ZBBridge, so such a setup could be used as a development environment.

E180-ZG120B (and the older E180-ZG120A) modules are sold on eBay and Aliexpress at low prices.

Example:

Ebyte also makes an inexpensive development board called "E180-ZG120B-TB" made for testing it.

This Zigbee 3.0 capable module has a powerful MCU and great radio specifications for its price:

This development is sold for less than $9 US-dollar on Aliexpress or about twice that on eBay UK:

Note! Ebyte is now making two EFR32MG12 based modules called E180-ZG120A and E180-ZG120B

This Zigbee 3.0 capable module has some very powerful MCU and radio specifications for its price:

The downside to using such setup instead of Sonoff ZBBridge is that the E180-ZG120B module will likely not come preloaded with firmware so have to build EmberZNet Zigbee Stack firmware with coordinator device type config and flash firmware to the E180-ZG120B module first.

Sonoff ZBBridge could be the perfect hardware if it was not for the case that it needs to be hacked.

Hedda commented 4 years ago

The idea with Sonoff ZBBridge would be to replace the complete firmware the ESP8266 MCU SoC with ESPurna FW but hopefully just leave the EFR32 Zigbee MCU SoC Zigbee coordinator firmware as it is.

@mcspr FYI, believe that bellows library can already handle serial connection disconnect & reconnect.

mcspr commented 4 years ago

@hedda My comment regarding connectivity & library was referring to the esp8266 networking side, nvm that.

I am still slightly confused by the terminology used by Silabs / Zigbee docs.

Hedda commented 4 years ago

Yes Silicon Labs terminology is a bit confusing to me too but after research I understand a little more.

You can say that "EmberZNet PRO" is the firmware on the EFR32 MCU and not the driver for it that you would run on ESP8266. The "driver" (like bellows) is the code that talks to the chip using the different SPI (Serial Peripheral Interface) and UART protocol APIs for direct communication using TTY-like software interface to the EmberZNet firmware support and those serial APIs are called and referred to as "EZSP" which is short for "EmberZNet Serial Protocol".

You do not need any proprietary stuff from Silicon Labs running on the ESP8266 as the "driver" side because EZSP uses open APIs, which are basically only serial commands (similar to AT Commands for old style analog modems if you remember those). So the driver side can be completely open-source code, like bellows is.

bellows library for example currently implemented EZSP features are:

In reality "EmberZNet PRO" is Silicon Labs name for their whole Zigbee Protocol Stack Software program (or an application framework if you will) for the Ember compatible MCUs that supports Zigbee, and those are both Silabs MCU chips made for Zigbee belonging to their new EFR32 family as well as to their old EM35x family of MCU chips). Both the old EM35x based MCU chips and the new EFR32 based MCU chips support the EmberZNet PRO Zigbee stacks, but slightly different versions of it, as EM35x based MCU chips is only compatible with a Zigbee PRO stack while the newer EFR32 based MCU chips support a full Zigbee 3.0 stack.

The newer EFR32 based MCU chips also runs EmberZNet PRO Zigbee stack that uses newer versions of the EZSP protocol with 16-bit Command IDs, while the older EM35x based MCU chips uses older versions of the EZSP protocol with 8-bit Command IDs. It should, however, be possible to all versions support both, though it would mean more work on the driver side, (for example https://github.com/zsmartsystems/com.zsmartsystems.zigbee does support all versions of the EZSP protocol).

With the exception of a bootloader, which is a separate firmware on its own (that are pre-installed from the factory on these), you can think of EmberZNet PRO Zigbee Stack as an application software program that contains everything you need to run on an MCU with a compatible radio for it be a Zigbee device.

To make an EmberZNet firmware for an EFR32 or an EM35x chip you have to build it in their Simplicity Studio, there add the EmberZNet PRO Zigbee Stack as an app and set its configuration if it should be a Zigbee Coordinator (Controller), Zigbee Router (like a mains connected plug or light), or a Zigbee End-Device (like a battery-operated sensor), and you then there finally make a hex/bin file of the firmware which you can also flash with other tools. Technically since EFR32 and EM35x chips are MCUs you could depending on available resources also install additional apps or scripts and such that you can run on an MCU.

Think these might be a couple of easier how-to guide for building EmberZNet for EFR32 and EM35x:

Another is to checkout "Zigbee Boot Camp Course" wiki for Silicon Labs that MarkDing has put on GitHub :

MarkDing also put together this PDF with Silicon Labs ZigBee Onboarding Roadmap for beginners:

Regarding understanding Zigbee itself you should checkout Silicon Labs training videos of ZigBee Concepts: Architecture basics, MAC/PHY, node types, and application profiles:

Those videos are also available as a video playlist on YouTube:

Disclaimer: I do not own a Silicon Labs chip or device myself so this information is only from research.

Hedda commented 4 years ago

https://github.com/zsmartsystems/com.zsmartsystems.zigbee has this summery on its Ember support:

Silicon Labs Ember EM35x / EFR32

The library supports the Silicon Labs EZSP protocol using ASH or SPI protocols over a serial interface. The implementation of the SPI protocol assumes that the SPI provides a TTY-like software interface to the application, or is otherwise abstracted via the ZigBeePort interface.

It is worth noting that EM3588 devices that have an embedded USB core will likely work with any baud rate, where dongles using external USB interface (eg CP2102 used with an EM3581) will likely require a specific baud rate.

Currently there are two main NCP images - one that supports hardware flow control with a baud rate of 115200, and one that supports software flow control with a rate of 57600.

Ember NCP configuration

The library provide a standard set of configuration constants to configure the NCP for use as a coordinator. There are two methods available in the Ember driver to manipulate the configuration maps updateDefaultConfiguration and updateDefaultPolicy. The configuration is sent to the NCP during the initialisation sequence which is performed when calling the ZigBeeNetworkManager.initialize() method, so any changes to configuration must be performed prior to this.

The Ember dongle driver includes a public method getEmberNcp() which returns a EmberNcp class. This class provides high level methods for interacting directly with the NCP.

Ember Reset

By default the library uses the ASH Reset command to reset the NCP when the framework starts. Silabs have stated that this is not reliable due to possible communication problems between the NCP and the host. Where possible the user should implement a hardware reset to reset the NCP. Since this is application specific, the framework provides an interface for the application to implement this reset. The user should implement the EmberNcpResetProvider interface, and set this during NCP initialisation with the ZigBeeDongleEzsp.setEmberNcpResetProvider() method.

Ember MfgLib use

The library provides access to the mfglib functions in the NCP to facilitate device testing. To use this function, create the ZigBeeDongleEzsp and then call the getEmberMfglib(EmberMfglibListener) method to get the EmberMfglib class and also set the callback listener. The EmberMfglib class provides access to the mfglib functions.

Note that this can only be used if the dongle is not configured for use in the network (ie initialize has not been called).

The com.zsmartsystems.zigbee.sniffer project is an example of the use of these features to provide a network sniffer to route frames to Wireshark.

Hedda commented 4 years ago
  • we can assume that Ebyte evaluation board simply installs a UART<->USB chip and connects it to the EFR32? Meaning, we can read data stream directly?

The specification of that Ebyte evaluation board does indeed say have a UART<->USB chip so that will likely present it just like a standard serial port in the operating system if you plug it in via USB to a normal computer (for serial/UART communication), however for connecting it to an ESP8266 then you would probably want to use those jumpers on the that Ebyte evaluation board to disable the UART<->USB chip (so only use the USB to power the module) as that way you can instead connect it directly via UART/serial using its TXD and RXD ports for talking to an ESP8266 (for serial/UART communication and not have to go through a USB converter). That would be almost like only having a stand-alone E180-ZG120B SMD module but with the convenience of USB-power and header-pins on a GPIO-staplinglist.

The benefit of that Ebyte evaluation board is really only that it makes it easy to connect to a normal computer and use JTAG for debugging (and moving between a connection to a normal computer and an ESP8266 during development). After development is done and you wanted to make a production device yourself for an end-user then you would only use an ESP8266 with an E180-ZG120B SMD module (and not that Ebyte evaluation board) as that would save cost and space.

mcspr commented 4 years ago

Thanks! Latest link to the https://github.com/zsmartsystems/com.zsmartsystems.zigbee/tree/master/com.zsmartsystems.zigbee.dongle.ember looks like a great example of the protocol implementation. (although, I am not yet sure why it reads as if data transport is implemented as SPI over UART)

Hedda commented 4 years ago

FYI @SillyDay don't own a E180-ZG120B but has tried to build a firmware for it as per discussion here:

https://github.com/Koenkk/zigbee-herdsman/issues/168

SillyDay GitHub repo for EFR32 firmware:

https://github.com/SillyDay/EFR32

Hedda commented 4 years ago

FYI, as per related request https://github.com/zigpy/zigpy/issues/405 I just learned that there is an existing ESP8266-based networked-attached Zigbee-adapter called "ZiGate Pack WiFi adapter" which has a new v2.0 firmware that archives this requested function of UART-to-TCP/IP (for Serial-port to WiFi-bridge function) using "ESP-LINK from Jeelab" software, in addition, using ESP-LINK also adds mDNS zeroconf to allow automatic network discovery and configuration:

Description of functions that using ESP-LINK will add to v2.0 firmware for ZiGate Pack WiFi adapter:

https://translate.google.com/translate?sl=fr&tl=en&u=https%3A%2F%2Fzigate.fr%2Fdocumentation%2Fdescription-du-firmware-v2-xx%2F

As I understand it, all ZiGate hardware look to be modular in design and the "ZiGate Pack WiFi adapter" is really just an optional ESP8266 based "dumb" UART-to-TCP/IP (for Serial-port to WiFi-bridge function) for the standard "ZiGate TTL adapter" that allows users to connect to it remotely using TCP/IP over your home LAN (Local Area Network) instead of plugging it directly to your computer via the optional USB adapter.

Specifically, please see the picture of "ZiGate Pack WiFi adapter" https://zigate.fr/produit/zigate-pack-wifi-v1-3/ compared to the picture of "ZiGate TTL USB adapter" https://zigate.fr/produit/zigate-ttl/

Thus "ZiGate Pack WiFi adapter" allows ZHA users to have a networked Zigbee adapter setup like this:

ZHA <–> zigpy/zigpy-zigate <–> TCP/IP over LAN <–> ZiGate-WiFi <–> UART <–> ZiGate Radio

Suggesting this now as I just learned from @doudz there that the new v2.0 version of the ZiGate Pack WiFi adapter firmware contains "ESP-LINK from Jeelab" software which among other things adds mDNS and UART WiFi Bridge support over TCP. As I understand, version v1.x of the firmware for the ZiGate Pack WiFi adapter basically only contained a simple UART/serial-port server forwarding service ( serial server software that just acts as a dumb Zigbee to WiFi bridge for zigpy-zigate), while the new version v2.x also has more advanced features (which does not need to used) it still also contain a simple UART/serial-port server forwarding service, but now mDNS also makes it easier to discover the adapter on your local network.

Now it would be awesome if the ZHA integration component for Home Assistant from an end-user perspective supported just as an easy detection and configuration of network networked-attached Zigbee coordinator adapters, like the Sonoff ZBBridge and the ZiGate Pack WiFi adapter.

I would therefore also suggest using some kind of Zero-configuration networking (zeroconf) method, like for example mDNS, (as mDNS is already in use in Home Assistant Core), to make the ZHA integration component for Home Assistant automatically detect, connect, and configure compatible networked-attached Zigbee coordinator adapters like the "ZiGate Pack WiFi adapter" as that is otherwise already supported by the zigpy-zigate radio library for zigpy.

Hedda commented 4 years ago

FYI, @s-hadinger has now got one and started review https://github.com/arendst/Tasmota/issues/8583

First signs it is the EFR32 Zigbee module it has inside it has a EmberZNet / Ember based firmware on it.

Hedda commented 4 years ago

There is now also a deep dive follow-up discussion here in parallel about hacking or flashing its EFR32:

Hedda commented 4 years ago

FYI; @s-hadinger is making great progress hacking the Zigbee module in Sonoff ZBBridge for Tasmota:

@mtx512 has now built a custom EZSP NCP firmware for the EFR32 module inside Sonoff ZBBridge:

Interestingly @s-hadinger also added a TCP serial bridge to Tasmota firmware for remote access:

Description:

Add a new feature: Serial to TCP bridge (similar to esp-link). This allows to remotely communicate to a MCU through ESP8266. Needs #define USE_TCP_BRIDGE

It adds 2 GPIO types: TCP Tx (208) and TCP Rx (209) and can work with hardware or software serial.

Commands:

  • TCPBaudRate <x>: sets the baudrate for serial (only 8N1 mode), min 1200, max 115200 by 1200 increments.
  • TCPStart <port>: listens to port <port>. This features supports 2 parallel TCP connexions, which can be useful if you need a terminal + a specific protocol (like XMODEM). The 3rd connection will disconnect an previous connection. The number of parallel connections is a compile-time option.
  • TCPStart 0 or TCPStart: shuts down the TCP server and disconnects any existing connection.

For security reasons, the TCP bridge is not started at boot, and requires an explicit TCPStart command (can be automated with Rules).

Hedda commented 4 years ago

FYI, Silicon Labs recently just released a less expensive "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) development kit for $99 which only contain one mainboard/development board and a couple of EFR32MG22 modules which are low-power Zigbee Green Power compatible Mightly Gecko Series 2 radios.

https://www.silabs.com/products/development-tools/wireless/efr32xg22-wireless-starter-kit

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it jas been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

Hedda commented 4 years ago

FYI, Silicon Labs recently just released a less expensive "EFR32xG22 Wireless Gecko Starter Kit" (SLWSTK6021A) development kit for $99 which only contain one mainboard/development board and a couple of EFR32MG22 modules which are low-power Zigbee Green Power compatible Mightly Gecko Series 2 radios.

https://www.silabs.com/products/development-tools/wireless/efr32xg22-wireless-starter-kit

While the EFR32MG22 low-power radios it comes with are not really usable as Zigbee coordinators, for $99 that kit can be very useful to developers since it has been confirmed that buying that kit also gives full official access to Silicon Labs Zigbee Stack, Zigbee SDK, and libraries (current, future, and old versions).

Those who are interested in that official $99 SLWSTK6021A started kit might also be interested in buying article SLWRB4180A for around $49 at the same time as that is the matching EFR32MG21 based high-power (+20 dBm) radio board for that dev board kit.

https://www.silabs.com/products/development-tools/wireless/slwrb4180a-efr32-wireless-gecko-radio-board

(Google search shows that SLWRB4180A can be bought from many of resellers globally who also stock the SLWSTK6021A kit).

Hedda commented 4 years ago

FYI, @s-hadinger has after the last pull request posted saying that he now feels Tasmota ready for testing of EZSP v8 interface:

https://github.com/arendst/Tasmota/issues/8583

https://github.com/arendst/Tasmota/pull/8839

Not for everyone yet since still need to flash its EFR32 chip with some type of J-Link Debug Probe (might always be required?).

Hedda commented 3 years ago

Any further thoughts on Sonoff ZBBridge support now that this is fully supported and already very stable/mature in Tasmota?

https://github.com/arendst/Tasmota/issues/8583

That Sonoff ZBBridge is inexpensive but contain great Zigbee hardware with potential for different types of Zigbee controller uses, as inside it has an ESP8266 WiFi SoC/MCU connected via UART to a very powerful Silicon Labs EFR32MG2 (Mighty Gecko Series 2) Zigbee SoC/MCU on the same board.

https://sonoff.tech/product/smart-home-security/zbbridge

Tasmota allows using its ser2net like TCP serial-server inside ESP8266 firmware to take direct control of the Zigbee chip that it is connected to via UART/serial and then connect to Zigbee devices without any further involvement on Tasmota (which via its TCP serial-server only will only act as a dumb pass-through WiFi-to-serial bridge without any knowledge of what the data traffic that is going back and forth between the remote Zigbee chip AND different home automation software like example Home Assistant).

You can even make your own DIY WiFi bridge with an ESP8266 board + an ICC-A-1 module torn out of an IKEA Tradfri device:

https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module

Example with ZHA integration component in Home Assistant which add native support for Zigbee adapters, including the ZBBridge:

https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html

Sure, it could have been even greater with ESP32 but still geat ESP8266 hardware with Silabs EFR32 MG2 Zigbee coordinator!

mcspr commented 3 years ago

I don't have any use-case for the zbridge hardware, so idk when I'll have time to write something to test / make it work.

At this point, I am not sure I remember everything that happened in this issue / referenced Tasmota implementation:

In case we really want the custom blob, upload mechanism need to be there too. But, at least that part is really clear - download the blob into the device's free space, reboot efr32 into bootloader by flipping a pin, slowly transfer the data from flash to the bootloader via xmodem protocol.

ref. https://github.com/1technophile/OpenMQTTGateway/issues/205#issuecomment-691032277

... the best way would be to externalize it into a Zigbee library. This way it will benefit the different projects (OMG, ESPurna, ESPEasy, ESPHome...).

I'd agree with that comment. Main issue was also mentioned in the other discussion - referenced integration (and pretty much every other, afaik) is quite large and requires substantial OS resources.

Hedda commented 3 years ago

I think first PoC use-case is a TCP serial-server bridge feature that has transparent pass-through serial-to-IP bridge, e.g. like ser2net

https://linux.die.net/man/8/ser2net

That would make it possible to act as a remote proxy for any serial module which can be access remotely by other software, e.g.:

https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html

Hedda commented 3 years ago
  • stock efr32 fiwmware is unusable? or simply that was never explored?

Firmware that ITead ships with their Sonoff ZBBridge is not the standard EmberZNet NCP application so needs to be reflashed with a other firmware image so Tasmota has compiled a generic version of SiLabs EmberZNet NCP application with a EZSP serial interface:

https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html

Tasmota provide a couple firmware image versions of the standard EmberZNet NCP application with EZSP serial interface:

https://github.com/arendst/Tasmota/tree/development/tools/fw_zbbridge

  • provided efr32 blob does not have much of the description besides "upload this to make this work". is this the generic code from silabs or some custom fw compiled specifically for Tasmota?

It is the generic EmberZNet NCP application with EZSP serial interface from Silicon Labs but it has been signed for Sonoff ZBBridge.

https://www.silabs.com/documents/public/application-notes/an1010-customized-ncp.pdf

https://www.silabs.com/documents/public/user-guides/ug100-ezsp-reference-guide.pdf

That is the standard EmberZNet NCP application with EZSP serial interface from Silicon Labs that is used other manufacturers, e.g.

Reason for it having to be signed it that ITead has encrypted so that their firmware can not be read or hacked via the GPIO pins and that means that you have to erase their firmware and write over it with another firmware image so that is what Tasmota does.

You can use the same EmberZNet NCP application with EZSP serial interface from Silicon Lab on other EFR32 Zigbee module without signing it, (e.g. an IKEA TRADFRI ICC-A-1 Module), to make your own DIY Zigbee to WiFi bridge with any ESP8266/ESP32.

https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module

In case we really want the custom blob, upload mechanism need to be there too. But, at least that part is really clear - download the blob into the device's free space, reboot efr32 into bootloader by flipping a pin, slowly transfer the data from flash to the bootloader via xmodem protocol.

Yes that is exactly what Tasmota has integrated in their Tasmota zbbridge firmware to make the EFR32 flashable from Tasmota.

Hedda commented 3 years ago
  • provided efr32 blob does not have much of the description besides "upload this to make this work". is this the generic code from silabs or some custom fw compiled specifically for Tasmota?

It is the generic EmberZNet NCP application with EZSP serial interface from Silicon Labs but it has been signed for Sonoff ZBBridge.

https://www.silabs.com/documents/public/application-notes/an1010-customized-ncp.pdf

https://www.silabs.com/documents/public/user-guides/ug100-ezsp-reference-guide.pdf

FYI, you can compile such standard (or custom) EmberZNet NCP EZSP firmware images for any EFR32 SoCs/MCUs hardware yourself if you buy a Silicon Labs $99 EFR32xG22 Wireless Gecko Starter Kit which once you register to your account gives you access to the EmberZNet PRO Zigbee stack via Silabs Simplicity Studio (buying a dev kit is required to get access to their Zigbee SDK):

https://www.cnx-software.com/2020/06/25/efr32xg22-wireless-gecko-starter-kit-targets-bluetooth-zigbee-and-proprietary-wireless-development/

https://www.silabs.com/development-tools/wireless/efr32xg22-wireless-starter-kit

https://www.silabs.com/products/development-tools/software/simplicity-studio

If have that development kit you can buy additional SLWRB4180A for around $49 which is a EFR32MG2 (EFR32 Mighty Gecko Series 2) based high-power (+20 dBm) radio board for that dev board kit, this is the same EFR32 MG2 chip that is inside Sonoff ZBBridge:

https://www.silabs.com/products/development-tools/wireless/slwrb4180a-efr32-wireless-gecko-radio-board

If you want to inexpensive alternatives then there is $9 Ebyte E180-ZG120B-TB Evaluation Board or IKEA TRADFRI ICC-A-1 Modules, however, note that those have the previous generation EFR32, that is the EFR32 Mighty Gecko Series 1 (EFR32MG1 / EFR32 MG1):

https://www.cnx-software.com/2020/04/27/ebyte-e180-zg120b-tb-zigbee-3-0-evaluation-board-features-silicon-labs-efr32mg1b-zigbee-thread-soc/

https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module

You can find precompiled firmware images for EByte E180-Z120B here

https://github.com/zha-ng/EZSP-Firmware

PS: Actually recommend IKEA TRÅDFRI ICC-A-1 Modules if want to make a DIY bridge because of ease of access to those modules:

https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module

Hedda commented 3 years ago

By the way, @grobasoz wrote that he's making a more alternative to Sonoff ZBBridge that he plan on selling before end of the year.

https://github.com/zigpy/bellows/issues/243

Hedda commented 3 years ago

FYI, HW tip now is to instead use Tube's Zigbee Gateways open-source hardware by @tube0013 (which is tested with ESPHome).

It is based on WT32-ETH01 ESP32 board from Wireless-Tag which also has wired Ethernet.

He designed two variants; one has a Silicon Labs EFR32 Series 2 module and one that has a Texas Instruments CC2652P module.

https://github.com/tube0013/tube_gateways

https://github.com/zigpy/zigpy/discussions/584

https://www.tubeszb.com/shop/coordinators/2

image

image

Hedda commented 3 years ago

I guess that best would probably be if you could get some informal advice about their API/CLI protocols from s-hadinger as he has experience of both since he implemented ZNP (TI Z-Stack) and EZSP (Silabs EmberZNet) support in Tasmota Z2T (Zigbee2Tasmota):

https://tasmota.github.io/docs/Zigbee/

https://github.com/arendst/Tasmota/pulls?q=s-hadinger

If only look at availability of dev-tools + modules available to buy then Texas Instruments CC2652P and/or CC1352P are best today:

https://www.szrfstar.com/product/list-196-en.html

https://www.ebyte.com/en/product-view-news.aspx?id=766

https://www.ebyte.com/en/product-view-news.html?id=1002

Silabs require buying their official starter kit to get access to firmware SDK + availability of EFR32MG21 modules is currently low:

https://www.silabs.com/development-tools/wireless/efr32xg22-wireless-starter-kit

https://www.silabs.com/development-tools/wireless/slwrb4180b-efr32xg21-wireless-gecko-radio-board

https://github.com/zigpy/zigpy/discussions/586

But low available of EFR32MG21 modules might not be a factor if could figure out to utilize the new ITead Zigbee 3.0 Dongle(?):

https://www.itead.cc/zigbee-3-0-usb-dongle.html

The older EFR32MG12 is another option available and it is even possible to extract such modules from any IKEA Trådfri device:

https://www.ebyte.com/en/product-view-news.html?id=842

https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module

Texas Instruments currently also have more open source tools for things like firmware flashing + NVRAM backup and restore:

https://github.com/JelmerT/cc2538-bsl

https://github.com/zigpy/open-coordinator-backup

https://github.com/zigpy/zigpy-znp/blob/dev/TOOLS.md#backup-and-restore

https://github.com/zigpy/zigpy-znp

I don't know about Texas Instruments ZNP serial protocol but if you want to offer an EZSP serial-server bridge option in ESPurna to enable Zigbee-serial-to-IP then know that really want to use a wired Ethernet and not WiFi to get a stable connection.

https://github.com/zigpy/bellows#warning-about-zigbee-to-wifi-bridges

PS: I believe tube0013 and grobasoz (who both make DIY gateway hardware) mention they prefer TI CC chips, tools, FW, APIs?

Hedda commented 3 years ago

CC1352 is more expensive but allow Sub-1 GHz frequencies used by Zigbee Smart Energy (ZSE) profile which isn't part of Zigbee 3.0

https://www.ebyte.com/en/product-view-news.aspx?id=766

https://www.szrfstar.com/product/235-en.html

https://www.szrfstar.com/product/188-en.html

Zigbee Smart Energy is the world standard used by electricity, water, and gas meters that all utility companies install in your house.

https://zigbeealliance.org/solution/smart-energy/

https://zigbeealliance.org/wp-content/uploads/2019/11/docs-07-5356-19-0zse-zigbee-smart-energy-profile-specification.pdf

https://training.ti.com/overview-zigbee-smart-energy-internet-things

https://www.ti.com/lit/slaa467

https://www.ti.com/lit/pdf/tidu213

https://www.ti.com/lit/pdf/swra467

https://www.nxp.com/pages/jn516x-zigbee-smart-energy:ZIGBEE-SMART-ENERGY

https://www.nxp.com/docs/en/supporting-information/MAXSECZBNETART.pdf

"Zigbee Smart Energy (Zigbee SE) is the world's leading standard for interoperable wireless products that monitor, control and automate the delivery and use of energy (and other resources, such as water). A wireless network is implemented at the consumer's premises using the ZigBee PRO protocol with the Smart Energy application profile. This Home Area Network (HAN) contains devices such as a Metering Device, In-Premise Display (IPD) and Load Control Device, and is connected to the energy provider using an Energy Service Interface (ESI) via a backhaul network."

PS: Note that Z-Stack 3 and SE share ZCL cluster library but the encryption of Z-Stack 3.0 and SE profiles are different:

https://e2e.ti.com/support/wireless-connectivity/zigbee-and-thread/f/zigbee-thread-forum/661416/cc2538-using-smart-energy-devices-in-z-stack-3-0

Hedda commented 2 years ago

FYI, thegroove made a simple to use Serial-to-IP bridge appliance firmware for Sonoff Zigbee Bridge running ESPHome firmware:

https://github.com/thegroove/esphome-zbbridge

oxan and thegroove also look to have made two alternative serial server code component solutions for ESPHome firmware:

https://github.com/oxan/esphome-stream-server

https://github.com/thegroove/esphome-serial-server

thegroove even made an experimental version with Zeroconf support for automatic network discovery by ZHA in Home Assistant:

https://github.com/thegroove/esphome-zha-ezsp-zeroconf

https://www.home-assistant.io/integrations/zha/#discovery-via-usb-or-zeroconf

Note that Zigbee2MQTT is another popular alternative to Home Assistant's ZHA for use with this type of Serial-to-IP bridge:

https://www.digiblur.com/2021/03/zigbee2mqtt-with-sonoff-zigbee-bridge.html

https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html

Hedda commented 2 years ago

FYI, ZB-GW03 eWeLink Ethernet Zigbee Gateway is based on an ESP32 and Silabs EFR32 Zigbee and has been hacked with Tasmota.

https://github.com/arendst/Tasmota/discussions/12764

ZB-GW03 has been hacked with Tasmota32 and look to support the same Tasmota and Silabs firmware as ITead Sonoff ZBBridge, however since the ZB-GW03 has wired Ethernet port instead of WiFi it should be a better option compared to ITead Sonoff ZBBridge.

https://templates.blakadder.com/ewelink_ZB-GW03

image

image

image

ZB-GW03 v1.0 and ZB-GW03-V1.2 is apparently sold rebranded under many names, including SmartWise and EACHEN brands:

https://ewelink.eachen.cc/product/eachen-ewelink-zigbee-bridge-pro-zbbridge-pro/

https://ewelinkcommunity.net/device-lists/zigbee/eachen-zb-gw03/

https://www.okosabbotthon.hu/smartwise-zigbee-bridge-pro

https://www.amazon.de/APP-Fernbedienung-Sprachsteuerung-Funktioniert-Verbindung-Smart-Produkten/dp/B094JKVLNR/

image

image

Looks like is it is based on the same "SM-011 V1.0" module by CoolKit as ITead uses in their Sonoff ZBBridge Zigbee Bridge:

https://github.com/zigpy/zigpy/discussions/586

https://github.com/xsp1989/zigbeeFirmware/

https://www.coolkit.cn/product/sm-011/

https://github.com/CoolKit-Technologies/DevDocs/tree/master/Zigbee

https://github.com/CoolKit-Technologies/DevDocs/blob/master/Zigbee/SM-011%E5%BA%94%E7%94%A8%E6%8C%87%E5%AF%BC%E4%B9%A6.md

https://translate.google.com/translate?hl=&sl=auto&tl=en&u=https%3A%2F%2Fgithub.com%2FCoolKit-Technologies%2FDevDocs%2Fblob%2Fmaster%2FZigbee%2FSM-011%E5%BA%94%E7%94%A8%E6%8C%87%E5%AF%BC%E4%B9%A6.md

Hedda commented 2 years ago

FYI, there's now a detailed step-by-step instruction about flashing Tasmota on ZB-GW03 eWeLink Ethernet Zigbee Gateway (wired network bridge) for Zigbee2Tasmota (also using with Home Assistant's ZHA):

https://thehelpfulidiot.com/a-wired-sonoff-zigbee-alternative

Again, basic info for experienced Tasmota and Z2T users here :

https://templates.blakadder.com/ewelink_ZB-GW03

and again there is also a discussion about it in Home Assistant community forum here:

https://community.home-assistant.io/t/zb-gw03-ewelink-ethernet-zigbee-gateway-now-hacked-with-tasmota-zbbridge-so-can-be-used-via-mqtt-or-as-a-remote-zigbee-adapter-with-home-assistant-zha/341223

UPDATE! The ESP32 inside ZB-GW03 has now also been hacked with ESPHome for use as a remote Zigbee Coordinator adapter:

https://github.com/syssi/esphome-zb-gw03

https://github.com/syssi/esphome-zb-gw03/blob/main/docs/flashing.md

Hedda commented 2 years ago

FYI, ESP32 and CC2652P based "Itead SONOFF Zigbee Bridge Pro" (ZBBridge-P, a.k.a. "ZB Bridge-P") is now available for purchase:

https://itead.cc/product/sonoff-zigbee-bridge-pro/

FCC leak from earlier in January this year showed inside with ESP32 + CC2652P based Sonoff ZBBridge-P v1.1 Zigbee Bridge:

https://fccid.io/2APN5ZBBRIDGEP

Both NotEnoughTech and digiblur have received theirs and while neither of them has tried to flash/hack it yet they posted some early thoughts on the original eWeLink hub firmware and hardware board, confirming that the first retail version ITead's SONOFF Zigbee Bridge Pro with PCB board labelled "ZBBridge-P V1.1 2021.11.12" is indeed based on Espressiff ESP32-DOWD-V3 (that ESP32 variant by the way also has integrated support for Bluetooth 4.2 including BLE), Texas Instruments CC2652P Zigbee module labelled "SM-031 V1.1 2021.07.28" (so probably manufactured by CoolKit Technologies like the Zigbee module for previous Sonoff bridge), with a cell-battery for the RTC (Real-Time-Clock), a built-in buzzer for chime or alarm-siren, and also an Espressiff branded PSRAM (ESP-PSRAM16H) with 2MB/16Mbit capacity.

digiblurDIY also mention that he managed to backup original firmware from ESP32 without issue so suspect that it is not locked:

https://www.youtube.com/watch?v=P5op57y5i6k?t=920&ab_channel=digiblurDIY

https://notenoughtech.com/home-automation/sonoff-zigbee-bridge-pro/

https://www.youtube.com/watch?v=fsIjFxCPJLA&ab_channel=NotEnoughTECH

Internal pictures posted from FCC show using an unannounced Texas Intruments CC2652P (CC2652P1) MCU chip based “SM-031 v1.1” radio module which must be made by CoolKit (company that also makes their EFR32MG21 based “SM-011 V1.0” module).

The new module based on Texas Instruments CC2652 probably come preloaded with Zigbee Coordinator firmware from Koenkk:

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.x.0/bin

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator

https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_3.x.0

Again the main downside the this new and the old Sonoff ZBBridge is that they only have WiFi and not a wired Ethernet LAN port.