search

esphome / feature-requests

ESPHome Feature Request Tracker
https://esphome.io/
406 stars 26 forks source link

Zigbee/Thread support on the new upcoming ESP32-H2 variant? #1397

Open levihb opened 2 years ago

levihb commented 2 years ago

Describe the problem you have/What new integration you would like

A new variant of the ESP32 was announced last month, the ESP32-H2. It supports 802.15.4 and therefore Zigbee and Thread, as well as BLE, and now runs on a RISC-V architecture which should be a lot more efficient. It also still supports WiFi.

This looks super interesting because it will finally give home users a way to easily build Zigbee/Thread versions instead of WiFi ones. I think ESPHome supporting this would be a brilliant step forward.

Please describe your use case for this integration and alternatives you've tried:

There are no alternatives to ESPHome for Zigbee as far as I know? Or really any simple development environments for home users.

Additional context

Of course the H2 isn't out yet. But I think it'd be good to get a discussion on this going before it is released. Zigbee is definitely a lot more friendly to larger home installations in many ways.

Support for battery operated end Zigbee devices would also be a huge leap forward. And I would think that should be possible on the new H2.

oxan commented 2 years ago

The big open question here is whether some group will port a Zigbee stack to the H2, as that's likely a too big task to do for ESPHome by itself.

levihb commented 2 years ago

The big open question here is whether some group will port a Zigbee stack to the H2, as that's likely a too big task to do for ESPHome by itself.

I asked on the ESP forums. Looks like they haven't released any information publicly yet.

But there will be an OpenThread implementation available. What about potential support for that?

levihb commented 2 years ago

@oxan Excellent news. They have confirmed that there will be a Zigbee stack available!

Hopefully this means we can have a Zigbee ESPHome.

DelusionalAI commented 2 years ago

With thread being IP based, would it be possible to connect it to a ESPHome thread device to a thread-border router and then control it over existing tools like MQTT and the Esphome API?
I believe the only "issue" would be that Thread is IPv6 only, which is being worked on

Controlling my devices over the existing APIs but without worrying about wifi would be amazing, and then you wouldn't be limited to the zigbee protocols.

oxan commented 2 years ago

I looked a tiny bit into this, and getting the API to run on Thread seems like it'll be fairly straighforward. Once devices start becoming available that's probably something we'll work to support.

As for Zigbee, examples using the Z-Boss 3.0 stack have started appearing in the IDF repository last week, so I bet that's what they're going with. There's actually an example of how to use an ESP as a lightbulb. It doesn't seem like it'll be too hard to integrate that into ESPHome as well, in case anyone is interested. I'm happy to help people implementing this, but probably won't work on it myself.

Drizzt321 commented 2 years ago

Really interested in this myself! Would love to create a few battery sipping things for myself, particularly Zigbee 3.0.

Hedda commented 2 years ago

With thread being IP based, would it be possible to connect it to a ESPHome thread device to a thread-border router and then control it over existing tools like MQTT and the Esphome AP

That could indeed be possible if Espressif releases a documented CLI or open API for Zigbee NCP or a Zigbee SDK for RCP, and that is something that Tasmota already feature support for but only for Silicon Labs EZSP and Texas Instruments ZNP, see: https://tasmota.github.io/docs/Zigbee/

There is by the way a related similar request here for Silabs EZSP (EmberZNet Serial Protocol) -> https://github.com/esphome/feature-requests/issues/681

There is also a separate request for just pass-through serial stream server (ser2net tunnel) support for Zigbee MCU here -> https://github.com/esphome/feature-requests/issues/688

kaystrobach commented 2 years ago

Related?

https://community.home-assistant.io/t/announcement-esphome-running-on-adafruit-52840-over-thread-mesh-networking-using-zephyrrtos/363712

HarvsG commented 2 years ago

I would very happy to contribute to the esphome team getting an engineering sample ASAP (if it is a matter of $) - Apparently espressif are hoping to get them out in May 2022 according to this reddit post

Hedda commented 2 years ago

Espressif now looks to have released ESP ZBOSS 3.0 binary libraries supporting the Zboss Zigbee 3.0 stack for ESP32 series chips:

https://components.espressif.com/component/espressif/esp-zboss-lib

https://www.esp32.com/viewtopic.php?t=24624

FYI, Espressif has joined "ZBOSS Open Initiative (ZOI)" so chances are they will base is Zigbee stack on ZOI's ZBOSS stack by DRS:

https://dsr-iot.com/news/78

https://www.prweb.com/releases/espressif_joins_zboss_open_initiative_zoi_a_community_for_royalty_free_zigbee_pro_stack_software/prweb18263163.htm

ZOI (ZBOSS Open Initiative) ZBOSS stack look to be available as royalty-free for ZOI members like but not as open source code:

https://dsr-zoi.com/frequently-asked-questions/

https://www.prweb.com/releases/announcing_launch_of_zboss_open_initiative_zoi_for_royalty_free_zigbee_pro_stack_software/prweb16814352.htm

https://dsr-zboss.com/#!/updates/dsr-joins-cmgc-partnership-s2link

All support for it will need be handled though Espressif as implementer and not directly to ZOI or DRS.

There now also looks to be Zigbee examples in Espressif’s IoT Development Framework (ESP-IDF) but no news on SDK or docs?

https://components.espressif.com/component/espressif/esp-zboss-lib

ESP ZBOSS 3.0 Libraries

This repository contains binary libraries supporting the Zboss Zigbee 3.0 stack for ESP32 series chips.

Packages from this repository are uploaded to Espressif’s component serivce. You can add them to your project via idf.py add-dependency

More information about idf-component-manager can be found in Espressif API guide or PyPi registry.

There now looks to be some Zigbee examples in Espressif’s IoT Development Framework (ESP-IDF):

https://components.espressif.com/component/espressif/esp-zboss-lib

https://www.espressif.com/en/products/sdks/esp-idf

https://github.com/espressif/esp-idf

https://github.com/espressif/esp-idf/tree/master/examples/zigbee

Those examples include; Zigbee coordinator ("light coordinator"), "Zigbee RCP (radio co-processor"), and "Zigbee gateway":

https://github.com/espressif/esp-idf/tree/master/examples/zigbee/light_sample/light_coordinator

"_lightcoordinator is a light coordinator example demonstrating Zigbee Coordinator role. It provides a formation of the Zigbee network. It runs on an 802.15.4 SoC like ESP32-H2. For more details to see the example readme file."

https://github.com/espressif/esp-idf/tree/master/examples/zigbee/esp_zigbee_rcp

https://github.com/espressif/esp-idf/tree/master/examples/zigbee/esp_zigbee_gateway

Wondering if "RPC" works similar to Silicon Labs RPC which works like a dumb radio requiring external ESP ZBOSS 3.0 libraries?

https://components.espressif.com/component/espressif/esp-zboss-lib

"_This test code shows how to configure Zigbee rcp (radio co-processor) device. Rcp doesn't function alone, it needs to work together with Zigbee gateway (see esp_zigbeegateway example)"

"After rcp starts up, it will send its own MAC ieee address and Zigbee stack version number to the Zigbee gateway and start working together with Zigbee gateway via UART communication"

ESP32-H2 has apparently now also already been certified as a “Zigbee Compliant Platform” by CSA:

https://www.espressif.com/en/news/ESP32-H2_Thread_and_Zigbee_certified

https://www.espressif.com/sites/default/files/ESP32-H2%20Zigbee%20Compliant%20Platform%20Certification.pdf

Type of Device: Zigbee Compliant Platform Zigbee PRO Feature Set (2017) Manufacturer: Espressif Systems (Shanghai) Co., Ltd. Model Identification: ESP32-H2 Firmware Version: V1.0 Hardware Version: V1.0 Certification Date: October 20, 2021 Certification ID Number: ZIG21030ZCP27315-24

PS: ESP32-H2 has also passed "Thread Certified Component” certification and will surely also be prioritizing Matter certification:

https://youtu.be/bS9Ch7k_GK0

PPS: They also say Espressif’s IoT Development Framework (ESP-IDF) feature a Thread SDK and that OpenThread is now supported:

https://github.com/espressif/esp-idf/tree/master/examples/openthread

HarvsG commented 2 years ago

@Hedda thanks for that information dump. Could I be bold and ask what the implications for esphome are? Sounds like many of the software stacks are already quite mature. Do you think that the team have enough to work with or do they need the SDK first?

Hedda commented 1 year ago

FYI, ESP-IDF (Espressif IoT Development Framework) look to now have posted an OpenThread RCP (Radio Co-Processor) example:

https://github.com/espressif/esp-idf/tree/master/examples/openthread/ot_rcp

I think that is needed for ESPHome to support OpenThread and Matter over Thread on ESP32-H2 with its IEEE 802.15.4 radio?

https://www.espressif.com/en/news/ESP32_H2

At least if that could be built so firmware provides OpenThread RCP (Radio Co-Processor) mode with SPI interface over the Spinel protocol (spinel+hdlc+uart serial protocol) and tunnel that serial stream to agners OTBR addon (OpenThread Border Router add-on via OpenThread adapter in RCP mode) from Home Assistant addons-development repository for testing Thread/Matter support inside Home Assistant:

https://community.home-assistant.io/t/chip-matter-support-including-thread/305633/88

https://github.com/home-assistant/addons-development/tree/master/openthread_border_router

https://github.com/openthread/ot-br-posix/

https://openthread.io/platforms/co-processor

https://github.com/agners

https://community.home-assistant.io/u/agners/summary

Matter upstream SDK also has support for ESP32 (but not sure if that is only over WiFi?):

https://github.com/project-chip/connectedhomeip/tree/master/examples/lighting-app/esp32

The main reason for OpenThread RCP (Radio Co-Processor) support in an "RPC" firmware with OTBR (OpenThread Border Router) support will make ESP32-H2 with its IEEE 802.15.4 radio compatible with the upcoming Thread stack / IEEE 802.15.4 radio based "Matter" (Project CHIP / Connected Home over IP) devices if used in Home Assistant with their other add-ons for the that is also in development. That addon by agners requires that the IEEE 802.15.4 radio have a OpenThread firmware in "RPC" mode instead of the no traditional "NCP" mode of OpenThread stack running on the IEEE 802.15.4 radio:

https://github.com/home-assistant/addons-development/tree/master/chip_controller_repl

https://github.com/home-assistant/addons-development/tree/master/chip_tool

https://github.com/project-chip/connectedhomeip

https://buildwithmatter.com

https://csa-iot.org/all-solutions/matter/

Also having optional OpenThread "NCP" border router firmware would allow users to alternatively use other existing OpenThread applications that use "NCP" mode instead of the newer "RPC" mode which require additional component running on the host. Ex:

https://github.com/openthread/wpantund

Note that as far as I read so far agners has only worked with Silicon Labs based adapter with OpenThread "RPC" firmware for Thread / EEE 802.15.4 radio based Matter (as well as ESP32-C3 based devkit for Matter over WiFi) and that is only because it is a Silabs EFR32MG21 chip based adapter that will ship inside the official Home Assistant Yellow (formerly Home Assistant Amber) hardware as well as the upcoming official Home Assistant SkyConnect USB Stick dongle:

https://github.com/zigpy/zigpy/discussions/894

https://github.com/home-assistant/addons-development/tree/master/silabs-multiprotocol

https://www.home-assistant.io/blog/2021/09/13/home-assistant-yellow/

https://www.crowdsupply.com/nabu-casa/home-assistant-yellow

https://community.home-assistant.io/t/home-assistant-skyconnect-usb-stick-announced-will-be-compatible-with-both-zigbee-and-thread-including-matter-chip-over-thread/433594

Hedda commented 1 year ago

FYI, see they now also released ESP ZBOSS 3.0 binary libraries (esp_zboss_lib) to support Zboss Zigbee 3.0 stack on chips like ESP32-H2:

https://github.com/espressif/esp-zboss-lib

Again that could be used with their "ESP" Zigbee RPC + Gateway examples with their ESP32-H2 DevKitC (ESP32-H2-DevKitC-1 V2.1) board:

https://github.com/espressif/esp-idf/tree/master/examples/zigbee/

https://github.com/espressif/esp-idf/blob/master/examples/zigbee/esp_zigbee_rcp

https://github.com/espressif/esp-idf/tree/master/examples/zigbee/esp_zigbee_gateway

Home Assistant's ZHA integration would need a ZBOSS (e.g. zigpy-zboss) radio library to be able to use it as a remote Zigbee Coordinator in serial pass-through mode.

https://www.home-assistant.io/integrations/zha

https://github.com/zigpy/zigpy

Compatible zigpy radio libraries

Hedda commented 1 year ago

ESP32-C6 SoC, module and development board has now been launched with WiFi 6, BLE 5.0, and 802.15.4, ink. OpenThread and Zigbee! ...so think these could be perfect for ESPHome as either a Thread Board Router (for Matter) or as a Zigbee Router device.

https://www.espressif.com.cn/en/news/ESP32-C6_Available

https://espressif-docs.readthedocs-hosted.com/projects/espressif-esp-dev-kits/en/latest/esp32c6/esp32-c6-devkitc-1/index.html

https://github.com/espressif/esp-idf/issues/10423

Espressif launched the ESP32-C6 WiFi 6, Bluetooth 5.0 LE, and 802.15.4 IoT microcontroller by making the ESP32-C6-WROOM-1 module:

https://www.cnx-software.com/2021/04/11/esp32-c6-wifi-6-and-bluetooth-le-risc-v-soc-for-iot-devices/

image

image

_Espressif Systems has finally launched the ESP32-C6 WiFi 6, Bluetooth 5.0 LE, and 802.15.4 IoT microcontroller by making the ESP32-C6-WROOM-1 module available on Aliexpress for $14.25 for a pack of 5 modules as well as the ESP32-C6-DevKitC-1 development board that’s out of stock on the official store at the time of writing. but somehow listed in another store for $16.22 plus shipping._

Besides the availability announcement, another surprise is the 802.15.4 radio for Zigbee and Thread that was not part of the original ESP32-C6 announcement in April 2021. The ESP32-C6-DevKitC-1 development board comes with a module with 8MB flash, exposes most of the I/O’s from the ESP32-C3, and comes with two USB-C ports, two buttons, and an RGB LED.

ESP32-C6-DevKitC-1 development board specifications:

image

image

The new ESP32-C6 module and development board work with the ESP-IDF v5.1, currently under development (see progress report), and some basic documentation can be found on read the docs.

_If you can’t purchase an ESP32-C6 module or board just yet due to limited supplies, don’t despair as more are coming very soon as Espressif tells us to expect more products on Adafruit, Akizuki, Digikey, and Mouser._

MiguelAngelLV commented 1 year ago

The C6 board can be a perfect board for esphome. Wifi, BLE, Zigbee and Thread in only one board!

Hedda commented 1 year ago

The C6 board can be a perfect board for esphome. Wifi, BLE, Zigbee and Thread in only one board!

Well almost. I understand Zigbee and Thread are currently mutually exclusive, so can not use Zigbee and Thread at the same time.

At least it should allow having many separate ESPHome-based devices with different functions utilizing either Thread or Zigbee.

I especially think that ESPHome devices could be perfect as a "Thread Board Router" (for Matter) or as a "Zigbee Router" device.

linkedupbits commented 1 year ago

I am starting to work on at least getting native esphome support for the esp32-c6-DevKitC-1. I have created a pull request to add support to espressive's esptool-js for this board - this library is used by esp-web-tools to flash the board. I have successfuly uploaded a zigbee espressive sample firmware to the esp32-c6-DevKitC-1 via my local build of esptool-js (but binaries built via esp-idf). Then I a local tar.gz of the esp-web-tools library, and dropped it into my local esp-web-tools , and was able to connect to the serial log output, but not able to upload a firmware built by ESPHome.

Where would I find an esp-idf project file for any of the "standard" new device firmwares eg esphome-web-esp32c3.bin or esphome-web-esp32s2.bin for bootstrapping into ESPHOme? I could then adapt this to build a base new device firmware for this board.

Hedda commented 1 year ago

Where would I find an esp-idf project file for any of the "standard" new device firmwares eg esphome-web-esp32c3.bin or esphome-web-esp32s2.bin for bootstrapping into ESPHOme?

Probably best if you start a new dedicated issue for that question

https://github.com/esphome/feature-requests/

https://github.com/esphome/esp-web-tools/

https://esphome.github.io/esp-web-tools/

https://web.esphome.io/

rotilho commented 1 year ago

I am starting to work on at least getting native esphome support for the esp32-c6-DevKitC-1. I have created a pull request to add support to espressive's esptool-js for this board - this library is used by esp-web-tools to flash the board. I have successfuly uploaded a zigbee espressive sample firmware to the esp32-c6-DevKitC-1 via my local build of esptool-js (but binaries built via esp-idf). Then I a local tar.gz of the esp-web-tools library, and dropped it into my local esp-web-tools , and was able to connect to the serial log output, but not able to upload a firmware built by ESPHome.

Where would I find an esp-idf project file for any of the "standard" new device firmwares eg esphome-web-esp32c3.bin or esphome-web-esp32s2.bin for bootstrapping into ESPHOme? I could then adapt this to build a base new device firmware for this board.

You may be interested in this fork. @stintel managed to flash a C6.

https://github.com/stintel/esphome/tree/esp32c6

Hedda commented 1 year ago

FYI, 01Space is sell a tiny ESP32-C6 IoT dev board based on ESP32-C6-WROOM-1 wireless module with two 10-pin headers + USB-C for power:

https://www.aliexpress.us/item/3256805225446433.html

image

ESP32-C6 Arduino support will require ESP IDF 5.1 framework which is still under development:

https://github.com/espressif/arduino-esp32/issues/7713

https://github.com/espressif/arduino-esp32/issues/7852

sandervandegeijn commented 1 year ago
  • ZHA integration

Don't forget zigbee2mqtt, this is one of the most mature solutions when it comes to interface with zigbeestuff through mqtt :)

Hedda commented 1 year ago

Don't forget zigbee2mqtt

Zigbee2MQTT support is off-topic here as zboss adapter (esp-zboss) support would have to be implemented into the zigbee-herdsman library it depends on and that has nothing to do with ESPHome or Home Assistant, so you would instead have to post a request there instead -> https://github.com/Koenkk/zigbee-herdsman

Hedda commented 1 year ago

Espressif now looks to have released ESP ZBOSS 3.0 binary libraries supporting the Zboss Zigbee 3.0 stack for ESP32 series chips

FYI, indirectly related; you can now use a Nordic Semi nRF52840 dongle as a Zigbee Coordinator inside ZHA via zigpy-zboss:

https://github.com/kardia-as/zigpy-zboss

This ZBOSS radio library for zigpy is unofficial and still experimental so testers need to manually it to ZHA in Home Assistant:

https://github.com/zigpy/zigpy/issues/394

That should for now at least make it usable to use either nRF52840 USB dongle or nRF52840 dev kits with the correct firmware.

https://www.nordicsemi.com/Products/Development-hardware/nrf52840-dongle

nRF52840 Dongle only costs about $10 USD so great value for people looking to test Zigbee. nRF52840 DK cost about $60 USD:

https://www.nordicsemi.com/Products/Development-hardware/nrf52840-dk

Perhaps more interesting for ESPHome developers is that if and when this zigpy-zboss library gets developed further then it could possibly enable users to use an ESP32-C6 (or ESP32-H2) module with ESPHome as a remote Zigbee Coordinator adapter for ZHA.

https://www.home-assistant.io/integrations/zha

wombat70 commented 1 year ago

Espressif announced their ESP Thread BR dev platform: https://www.hackster.io/news/espressif-launches-a-kit-to-turn-its-esp32-modules-into-certified-thread-border-routers-for-matter-c61db0fce050

Hedda commented 1 year ago

Espressif announced their ESP Thread BR dev platform: https://www.hackster.io/news/espressif-launches-a-kit-to-turn-its-esp32-modules-into-certified-thread-border-routers-for-matter-c61db0fce050

Nice! That is a new ESP32-H2 + ESP32-C3 combo development board kit and ESP Thread Boarder Router SDK which as been certified as a Thread Border Router (TBR) / OpenThread Border Router (OTBR) which can be used for Matter based on the Thread 1.3 standard, alternatively be used as an as Zigbee gateway/hub or bridge, and they also got an optional Ethernet Sub-Board

https://www.cnx-software.com/2023/06/20/espressif-esp-thread-border-router-board-combines-esp32-h2-esp32-c3-wireless-chips/

https://github.com/espressif/esp-thread-br

https://docs.espressif.com/projects/esp-thread-br/en/latest/esp32/

https://www.aliexpress.us/item/1005005688193617.html

PS: An earlier two-board ESP Thread Border Router solution was previously available as an example project for reference:

https://github.com/espressif/esp-idf/tree/master/examples/openthread/ot_br

https://openthread.io/guides/border-router/espressif-esp32

image

and optional Optional Ethernet Sub Board

image

Hedda commented 1 year ago

Espressif ESP-IDF v5.1 with ESP32-C6 and ESP32-H2 has now been released

https://github.com/espressif/esp-idf/releases/tag/v5.1

HarvsG commented 1 year ago

Are there anymore requirements before development can start on ESP Home Matter / Zigbee devices?

Would we still need this?:

The big open question here is whether some group will port a Zigbee stack to the H2, as that's likely a too big task to do for ESPHome by itself.

Hedda commented 11 months ago

Off-topic but FYI as related; just noticed that grobasoz added some Zigbee RCP and CLI firmware images for ESP32-C6. Note that is not ESPHome build, however, could serve as a partial proof-of-concept in a two-chip setup (similar to Espressif's new ESP32-H2 + ESP32-C3 combo development board kit where the ESP32-S3 would be flashed with ESPHome and the ESP32-C6 would only run Zigbee/OpenThread firmware to simply act as a Zigbee/Thread radio connected via serial using CLI or RCP interface). No idea how or if these firmware images work, (but grobasoz is at least well known in Home Assisistant's ZHA community for providing good unofficial Zigbee NCP firmware images for a wide range of Silicon Labs based Zigbee Coordinator adapters):

PS: DamKast recently also announced "zigpy-zboss" as an (unofficial and experimental) ZBOSS radio library for zigpy -> https://github.com/kardia-as/zigpy-zboss and that could in theory make zigpy compatible with ESP32-C6/ESP32-H6 if their firmware use the standard ZBOSS API/CLI which if it is the case should make it usable as a remote network-attatched Zigbee Coordinator for Home Assistant's ZHA integration if configure via serial server pass-through as proxy using socat (similar to ser2net) -> https://github.com/zigpy/zigpy/issues/1052 (More zigpy-zboss discussion related to zigpy here -> https://github.com/zigpy/zigpy/issues/394)

Hedda commented 11 months ago

FYI, vidplace7's & stintel's pull request with support for ESP32-C2 & ESP32-C6 has now been merged -> https://github.com/esphome/esphome/pull/4377 and there is also an additional PR by stintel adding support for Espressif's ESP32-C6-DevKitC-1 too -> https://github.com/esphome/esphome/pull/5289

Note! Those does not include Zigbee or Thread (or Matter) support so I think this feature request is still in place to discuss that.

knallt commented 11 months ago

Is the Zigbee support with the new ESP-32 e.g. C6 variant already on the official roadmap? It would be a complete gamechanger to use Zigbee ZHA connection instead of Wifi / BT for esphome integrations.

sorryusernameisalreadytaken commented 10 months ago

I would love to know that too. But I cant find information on that....

Hedda commented 10 months ago

Is the Zigbee support with the new ESP-32 e.g. C6 variant already on the official roadmap? It would be a complete gamechanger to use Zigbee ZHA connection instead of Wifi / BT for esphome integrations.

I would love to know that too. But I cant find information on that.

Questions like that and "+1" posts are better asked in esphome discussions -> https://github.com/esphome/esphome/discussions

I do not think that there is a need to just post new "+1" comments to this issue as they do not actually add anything constructive to this feature request that could perhaps have the opposite effect and annoy developers instead of encouraging as each post in issues generates an e-mail, spamming everyone following this.

If you only want to add a simple "+1" to show that you are interest in this then suggest that you consider just click the symbolic thumbs-up emoji at the bottom of the original post above as that will not generate an e-mail to the followers of this request:

image

vidplace7 commented 9 months ago

I've submitted another PR to add initial support for ESP32-H2 https://github.com/esphome/esphome/pull/5498

Doing this in hopes that someone smarter than me will develop a Thread/Zigbee implementation in ESPHome, once board support is available.

Hedda commented 8 months ago

Not sure if any use to help acheive Zigbee on ESPHome but FYI, as an alternative to the ESP-IDF Zigbee library, but FYI, it looks like @P-R-O-C-H-Y is planning on porting ESP-IDF Zigbee library and ZBOSS libraries to be available as Arduino libraries for ESP32:

https://github.com/espressif/arduino-esp32/issues/8807

https://github.com/espressif/esp32-arduino-lib-builder/issues/142

https://github.com/espressif/arduino-esp32/pull/8844

https://github.com/espressif/esp32-arduino-lib-builder/pull/146

micfogas commented 8 months ago

This is good to hear. I recently picked up the ESP-THREAD-BR which is a dual SoC board that uses an ESP32-C3-WROOM as the host/main SoC, and the ESP32H2-MINI-1 as the 802.15.4 RCP. The board is named Espressif OpenThread Border Router Gateway/ZigBee. I also snatched up the Ethernet Sub Board module for it because something important like a OTBR should have a steady and solid connection to the network, IMO. My H2 firmware is good, but I'm still working on getting a functional firmware loaded on the C3 side. The one that did compile kernel panics on bootup and bootloops, and I'm trying to figure out why my current sdkconfig is failing on the compile. I think it might have something to do with my effort to disable the WiFi on the C3. The error seems to indicate something in the STA defines, which I need to recheck. Just going to wipe and repull the esp-thread-br repo and start over on it, better documenting my changes this time. I think I need to manually edit the sdkconfig file as there are things in there that aren't available in the menuconfig, even with experimental turned on. I reached out to Espressif and I'm waiting on them to reply.

Ideally, we would be able to create a firmware based on ESP-IDF and esp-thread-br that would allow HA to transparently control the device across IP. I don't think it would require much on the firmware side. All of this is still somewhat experimental from Espressif.

I also bought 3 ESP32-C6 boards for now (I'll need more but ESPHome integration is what I'm really waiting on, so I can use my current config and simply change the wifi to 802.15.4 settings and have it function seamlessly) to use as endpoints for projects. At $10USD per board, it's cost effective enough for me to replace all my ESP8266 and ESP32 wifi projects with these boards. I live in attached condos and have had one or two people over the past 20 years run deauth attacks (I have kismet set up as a WIDS). An 8266 with an 18650 pack and deauth firmware on a waterproof case discretely placed outside their units for a couple days seemed to send the proper message ;) low power TX that my APs couldn't hear it, but the SSID name changes indicated the frustration and and surrender. Got to love a world where we can communicate with our tech savvy neighbors using SSIDs.

I'll be following this thread... get it? Thread? 🤦 and see if I can't contribute. I was a patreon supporter of ESPHome for a long time before it was taken over, and used to be a regular on their discord. I might be able to contribute on their end too.

Looking forward to what the future brings!

Now if we can get nRFHome... 😈

spusuf commented 8 months ago

Ideally, we would be able to create a firmware based on ESP-IDF and esp-thread-br

I don't think the br (border router) functionality is required. There are many home assistant compatible border routers, its just a matter of esphome equivalent end device (I don't know if that's the correct Thread terminology, but that's what ZigBee calls it). And the using that endpoint to test which existing/new border routers have functioning discovery and connectivity. Whether or not the communication uses matter or an esp/ZigBee specific communication protocol is up to the Devs. I'm sure the tinkerer won't care as long as it works

I'll be following this thread... get it? Thread?

😭

spusuf commented 8 months ago

Whether or not the communication uses matter or an esp/ZigBee specific communication protocol is up to the Devs.

Wait is Matter over ZigBee a thing (or non-matter protocols over Thread) or does the issue of ZigBee and Thread functionality need to be split into two different issues.

Hedda commented 8 months ago

Wait is Matter over ZigBee a thing (or non-matter protocols over Thread) or does the issue of ZigBee and Thread functionality need to be split into two different issues.

Thread and Zigbee should really be two separate issues, and Matter support should also be its own separate issue too.

To summerize:

Valid native Matter paths:

Matter bridges enable more of a one-way path since Matter devices can now be accessed in other IoT networks, like for example:

Below is a topology diagram of Matter fabric that has a Zigbee to Matter bridge in it:

image

PS: Recommend read this -> https://www.theverge.com/22832127/matter-smart-home-products-thread-wifi-explainer

Hedda commented 8 months ago

I don't think the br (border router) functionality is required.

Thread Border Router support is definitely of interest as Thread Border Routers works similarly to existing Bluetooth Proxy feature.

https://esphome.io/components/bluetooth_proxy.html

Thread radios have very short range, their signals have poor building material penetration, and is also extremely sensitive to EMF/EMI/RMI interference, so having the option to enable native Thread Border Router support in ESP32-C6 with ESPHome would be great as would allow users can then place such ESP32-C6 based devices that are within Ethernet or WiFi reach close to their Thread devices that would otherwise be out of reach from other Thread radios on the same network. And you can have multiple Thread Border Routers on the same Matter network.

Below is a diagram of typical Matter topology:

image

Dodoooh commented 8 months ago

For my part, I just want something that extends the range of ESPHome devices and has a better range than WiFi.

Hedda commented 8 months ago

For my part, I just want something that extends the range of ESPHome devices and has a better range than WiFi.

If looking for better range than WiFi then Thread and Zigbee is not for you, (suggest look at LoRa and LoRaWAN instead -> https://github.com/esphome/feature-requests/issues/124).

Thread and Zigbee have a much shorter range than WiFi since the physical layer of Thread and Zigbee relies on IEEE 802.15.4 radios that are low-rate low-energy, while in comparison Wi-Fi uses high-rate high-energy radios. Thread and Zigbee transmit low-energy signals that have very poor penetration of building materials and they also send very short messages, (e.i. "turn device xyx on"), so their signals can easily be drowned out in the environment noise from other electronics.

https://en.wikipedia.org/wiki/IEEE_802.15.4

So the benefit with Thread and Zigbee is that they support their own mesh networking so practically any mains-powered Thread and Zigbee device can act as a router that forwards messages within the network that they belong to (e.i. almost all mains-powered Thread and Zigbee devices will act as signal-repeaters/range-extenders within their own network).

Thread also has the bonus feature of supporting multiple "Thread Border Routers" in the same Thread network, compared to Zigbee which only supports a single Zigbee Coordinator on a Zigbee network. So if you can have several Thread Border Router devices on the same network then you both have redundancy and multiple ways to bridge the Matter protocol traffic to other non-Thread networks (e.i. your LAN network via either WiFi or Ethernet).

PS: Off-topic but if you looking for an IoT protocol where each individual device has longer range without going a mesh network with routers/repeaters/extenders then suggest that you instead check out LoRa/LoRaWAN, (or alternatively if ESPHome and ESP32 is not a requirement then maybe check out Z-Wave 700 and Z-Wave 800 which even has a "Z-Wave Long Range" standard).

micfogas commented 8 months ago

Hedda, great information for people. I think these 2 explanations are a little off though:

  • You can not use Zigbee over Thread, (Thread only supports IPv6 so it can support implementation that support IPv6).

It should be noted that Thread is an alternative to ZigBee and Z-Wave. They are all built on top of 802.15.4 hardware. Bluetooth uses 802.15.1 hardware. And of course WiFi is 802.11 hardware. These IEEE standards define essentially layers 1 and 2 of the OSI model for these devices: the physical layer and the link layer. An 802.15.4 radio should be capable of working with ZigBee or Thread if flashed with appropriate firmware that implements the protocols fully. So ZigBee and Thread cannot run on top of each other, because they essentially operate at the same level. 802.15.4 defines the WPAN standard, or Wireless Personal Area Network. Thread specifically is built using 6LoWPAN, or IPv6 Low Power Wireless Personal Area Network. I'll address this more on the next section.

  • Matter and Thread are directly related as can use Matter over Thread, (though Matter is also supported over WiFi and Ethernet, without Thread, so Thread support is not a requirement to support Thread, however it is required to support Thread-based Matter devices).

Matter is fully compatible with Thread because Matter is built on IPv6. However, Matter is not exclusively 6LoWPAN. Like you said, Matter supports Ethernet, Wifi and more. Matter is specifically build as a Smart Home universal solution. It was originally known as CHIP, or Connected Home over IP. It's designed to be a universal solution and set of standards that allow all smart home devices (everything Tuya, ZigBee, Thread, SmartThings, etc) and hubs (such as Amazon Echo devices, Home Assistant, Google/Nest Hubs, etc) to integrate with each other seamlessly. The idea is to eliminate the need for, for example, an Aqara hub to use Aqara devices, a Hue hub for Philips, etc. Even though Home Assistant has been developed to overcome many of these proprietary systems, remember that Matter is not something designed or developed for Home Assistant. It's universal and meant to allow all smart home devices to integrate seamlessly without the need of something like Home Assistant. That doesn't mean that Home Assistant is pointless; the advanced level of routines, automations, sensor tracking, states, and the ease of implementing your own homemade smart devices with things like ESPHome, are all reasons why Home Assistant is amazing. Not everyone is tech savvy enough or wants the advanced abilities of Home Assistant. Matter is meant to simplify the connected home for everyone, from Home Assistant gurus to Donna with an Echo Dot who wants to add smart bulbs all around her house and has no idea what Tuya, ZigBee, Thread, or SmartThings is. The idea is that you don't have to do a ton of research to ensure compatibility of your smart devices with your hub; you can have an Echo Dot or a Nest Home Mini and buy 5 different smart bulbs, screw them in, and they just work.

The advantage of using Thread over ZigBee is that it natively supports IPv6, which Matter uses. This means you don't need a ZigBee bridge. However, Thread with Matter devices still need to bridge from 802.15.4 to communicate with WiFi and Ethernet Matter devices. Think of WiFi and the need for an Access Point or WiFi router to bridge your wireless packets to Ethernet so that they can communicate with non-wifi connections, such as the internet.

Thread Border Routers are that solution. With ZigBee, you can only have one Coordinator. In fact, you must have exactly 1 coordinator.

Most people using Home Assistant do this through the use of a USB dongle running ConBee, or that have a generic Silicon Labs board that was programmed with a debugger tool, flashed with an older coordinator firmware. I have one of these. I also have an Echo Plus. It's an older Echo device from around 2016 or 2017 I believe. But it has a ZigBee coordinator built into it, which allows it to connect with non-proprietary ZigBee devices like SIGLED. Tuya ZigBee devices require a Tuya ZigBee hub, Aqara ZigBee devices require an Aqara hub, etc. That's what Thread and Matter are meant to eliminate. Thread can have multiple Thread Border Routers and allows for multiple isolated groups of Thread devices to exist. Matter runs on top of Thread, allowing these isolated groups to be part of the same network. They do have to be bridged of course using Wifi or Ethernet. Matter does use a controller as well. Home Assistant has one available. The big tech smart home hubs such as Echo and Nest Hubs take on this role as well. Think of it like your home router that runs the DHCP server, assigns addresses, and is your centralized gateway for all devices in your network to communicate with each other across WiFi and Ethernet, out to the internet. Matter doesn't require the internet, though a Matter network can be accessed through the internet. Your internet router acts as a controller, sort of, for your home network. You could operate a LAN without a central router, but you'd have to assign IP addresses manually, and most Internet providers only give one public IP address, which means that only one device could connect to the Internet at a time.

In an Ethernet network, your home router is generally referred to as an edge router or border router. It sits at the edge or border of your LAN and passes traffic in and out of your LAN to a WAN, which is typically the internet.

In order to do that, there are very big tech companies behind alliances, and simply put, if a company wants its smart home devices to be used in people's homes, they will adopt the Matter standard or end up isolated to being controllable only through their own app. Even Tuya, as big as they are, doesn't want to be cut off from working with Alexa, Google Nest/Home, or HomeKit.

So to summarize: Thread is based off the same technology as ZigBee and is a replacement for ZigBee. Matter is a new standard designed for interoperability of smart home devices. Thread was designed with Matter in mind so that Thread devices could easily carry Thread on it. Thread Border Routers are necessary and sit on the edge or border of Thread network segments and to connect these segments to your smart home networks. The idea is that Matter will be running on top of the Thread segments as well as on your Wifi and other smart home devices. Home Assistant's Border Router implementation bridges Thread devices to Home Assistant to be controllable by Home Assistant. I'm not sure if they are using Matter on top of it or not. It's not necessary right now with Home Assistant acting as the Border Router by directly controlling the 802.15.4 hardware to interface with Thread end devices. Eventually, Matter will be running on top of all of these things, including Home Assistant, to integrate smart home devices together, and Home Assistant will be the Matter coordinator.

I hope that helps people better understand the differences and what Matter and Thread are, and aren't.

djorr5 commented 7 months ago

Hi all, I have been deep diving lately into the ESP32C6/H2 as I have been playing around with remotely switching off electric fences via a Zigbee power point. Subtle but it works for me. Currently I have a Sonoff Dongle plugged directly into my Pi which is running HA. This acts as the coordinator and then I enrol a Zigbee GPO into HA which I use my companion app when out in the field to turn it off. However now I have another paddock not on my home network so I am thinking to myself weather it might work for me to play around with the new C6 and see if it can be setup as an Zigbee2MQTT device that subscribes back to my home network HA and uses the Zigbee protocol to turn off the GPO.

I am adequately enough skilled with Kicad to design a board with 2 ESP SoCs to achieve this but what I am unsure about is if this is achievable from a software perspective. I thought I would ask here as I noticed @Hedda and others in this area seem quite well versed in the technology.

Hedda commented 7 months ago

@djorr5 before reading my replies below you should note that this is still only a feature request. There is not yet any Zigbee or Thread support in ESPHome, and as far as I know, no developer(s) have even started working on any of this as of yet.

Off-topic; I also want to add that in my humble personal opinion, ESPHome is a good platform if you are targeting users of open-source home automation platforms, (that is, ESPHome is not a broad platform that can easily be used with commercial ecosystems like Google Home or Amazon Alexa if the user does have not also implemented an DIY bridge solution to translate/convert to other protocol or service). However, I believe that having native Zigbee (and Thread) support in ESPHome would change that since many commercial ecosystems do now have their own Zigbee and/or Thread gateway.

  • Can I run Zigbee2MQTT on an ESP32 SoC?

No, you can not run Zigbee2MQTT natively on ESP32 SoC, because Zigbee2MQTT and it dependencies does not have a small footprint and requires more resources (partially because it is coded in JavaScript and TypeScript), so requires a more powerful SoC (e.g. something with at least as much resources as the original Raspberry Pi) + are designed to run on Linux OS on a computer and not a RTOS on a microcontroller.

If want a Zigbee to MQTT gateway that has a small enough footprint to run natively on an ESP32 SoC then instead check out the Zigbee2Tasmota project by Tasmota, note that is a very barebone Zigbee gateway solution, however, you should not expect to run anything more advanced on ESP32-C6 (or ESP32-H2) which have single-core CPU and do not have much RAM or Flash Storage:

https://tasmota.github.io/docs/Zigbee/

Existing Tasmota developers have however not yet expressed an interest in native ZBOSS support for ESP32-C6/ESP32-H2, see:

https://github.com/arendst/Tasmota/discussions/20045

  • Does the Zigbee support in a C6 support publicly commercial end devices?
  • Will ESPHOME support remote instances of Zigbee ESP board?

First of all, understand that native Zigbee support on ESP32-C6/ESP32-H2 is primarily meant for them to be used as an Zigbee End Device (like for example a Zigbee sensor or actuator), not as a Zigbee gateway application with a Zigbee Coordinator than you in turn connect third-party Zigbee devices to (since running a Zigbee gateway application takes more resources and a more interactive user interface + a lot of custom conversion/translation for all non-standard Zigbee devices).

Regardless, if you want to have a proper Zigbee gateway solution for commercial use then you should really not use just a single ESP32 SoC, though you could integrate an ESP32 as a co-processor and use it in combination with an ARM-based single-board-computer (like a Raspberry Pi).

Espressif allows you to use its libraries for/from their esp-zigbee-sdk and esp-zboss-lib, as well as ESP-IDF SDK/libs and esp32-arduino SDK/libs in commercial products if that is what you are asking, (though you really always have to check the license for each and every library and piece of code used in your project before releasing a commercial product).

https://github.com/espressif/esp-zigbee-sdk

https://github.com/espressif/esp-zboss-lib

ESPHome firmware can also be used in commercial end devices, (but note that support for ESP32-C6 and ESP32-H2 is very new and not even fully supported yet, so should absolutely not be considered "stable", and it also not a high performace variant of ESP32). For reference, Everything Smart Technology makes and sells a couple of nice products based on ESPHome:

https://shop.everythingsmart.io

  • Will ESPHOME support remote instances of Zigbee ESP board?

That is what this feature request is partially asking for, this is a feature request, but again, no developer(s) are working on it yet.

I thought I would ask here as I noticed @Hedda and others in this area seem quite well versed in the technology.

FYI, I might be well read up on the subject but I do not have the coding skills to implement any of the things requested here.

nstadigs commented 7 months ago

I also want to add that in my humble opinion, ESPHome is only a good platform if you are targeting Home Assistant users, (that is, ESPHome is not a broad platform that can easily be used in commercial ecosystems if the user does not also have Home Assistant).

I don't agree. I'm using esphome with mqtt, without home-assistant and I still find the declarative approach of esphome is very beneficial.

crbn60 commented 7 months ago

I second that: I do not use HA at all, but heavily use ESPhome to simply connect MCUs to MQTT.

A.

On Thu, Nov 23, 2023 at 12:58 Niklas Stadigs @.***> wrote:

I also want to add that in my humble opinion, ESPHome is only a good platform if you are targeting Home Assistant users, (that is, ESPHome is not a broad platform that can easily be used in commercial ecosystems if the user does not also have Home Assistant).

I don't agree. I'm using esphome with mqtt, without home-assistant and I still find the declarative approach of esphome is very beneficial.

— Reply to this email directly, view it on GitHub https://github.com/esphome/feature-requests/issues/1397#issuecomment-1824782810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALABGRMJPIGCEKSXIYTXVDYF6FDFAVCNFSM5DMGOLPKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGQ3TQMRYGEYA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

MiguelAngelLV commented 7 months ago

Esphome can be used without HA and without connection. You have screen support so you can create easily a device to show data.

For me, before get support Zigbee and Thread, I think is more important get a Matter support over WiFi. So, any device create with EspHome can be used with any domotic system (HA, Google, Alexa....)

Hedda commented 7 months ago

@djorr5 to not spam this with off-topic discussions it is probably better if you start a new discussion instead, please see here:

https://github.com/esphome/esphome/discussions

Alternatively join the discord server for support:

https://discord.gg/KhAMKrd

Questions on whether or not ESPHome is the right platform for your unique project do not really belong in this feature request here is this specific feature request which is specifically asking for Zigbee and Thread support on ESP32-C6/ESP32-H2.

For me, before get support Zigbee and Thread, I think is more important get a Matter support over WiFi. So, any device create with EspHome can be used with any domotic system (HA, Google, Alexa....)

I agree, however, that is off-topic, and so is also all arguments if ESPHome is a good platform for non-HA / commercial use or not.

This specific feature request is asking for Zigbee and Thread support, both of which do not require Matter support, (and Matter does not require Thead or Zigbee since Matter since can be done with only Wi-Fi or Ethernet) + there is already an existing separate dedicated feature request specifically for Matter, see -> https://github.com/esphome/feature-requests/issues/1430

Updated my comment above to clarify why this comment is off-topic and why it in my personal opinion does not belong here.

AllardKatan commented 7 months ago

So, just to so I understand things correctly: this H2 board and the C6 board will make it possible to make ESPhome devices that communicate via Zigbee (as end devices or routers) instead of WiFi? That would be great. I don't use any ESPhome devices because my wifi isn't very reliable, all my sensors etc communicate through Zigbee. But I've looked at ESPhome before and it looks like a nice ecosystem. But apparently it's still a long way to go until I can expect to buy one of these and pull a few useful projects off the internet to flash onto them? Any rough ball-park estimates of that timeframe would be appreciated. Are there people actively working on it, or is this still waiting for someone to pick it up?

Hedda commented 7 months ago

So, just to so I understand things correctly: this H2 board and the C6 board will make it possible to make ESPhome devices that communicate via Zigbee (as end devices or routers) instead of WiFi?

Yes. As proof-of-concept it's possible using Espressif's ESP-Zigbee-SDK and example code with ESP-IDF (not via ESPHome though).

https://github.com/espressif/esp-zigbee-sdk

https://docs.espressif.com/projects/esp-zigbee-sdk/en/latest/esp32/

https://github.com/espressif/esp-zigbee-sdk/tree/main/examples

https://components.espressif.com/components/espressif/esp-zigbee-lib

https://github.com/espressif/esp-zboss-lib

https://github.com/espressif/esp-idf/tree/master/examples/zigbee/

Again, those examples only demonstrate how to do this using ESP-IDF, (find details on ESP-IDF website), not on ESPHome.

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/

I believe could get esp-zigbee-lib and esp-zboss-lib working on Arduino IDE now that ESP32-C6 and ESP32-H2 are supported?

https://github.com/espressif/arduino-esp32/issues/8807

https://github.com/espressif/arduino-esp32/pull/8844

https://github.com/espressif/esp32-arduino-lib-builder/issues/142

Any rough ball-park estimates of that timeframe would be appreciated. Are there people actively working on it, or is this still waiting for someone to pick it up?

As far as I know no one is working on this for ESPHome as of yet, (at least not what I've seen/heard so far, and I have searched).

So to clearify; this is just a feature request that is waiting for some skilled developer(s) to pick it up for ESPHome integration(s).