Poor connections to Home Assistant Yellow (TZ-972)

blaubaer commented 3 months ago

Answers checklist.

[X] I have read the documentation ESP Zigbee SDK Programming Guide and tried the debugging tips, the issue is not addressed there.
[X] I have updated ESP Zigbee libs (esp-zboss-lib and esp-zigbee-lib) to the latest version, with corresponding IDF version, and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.4-dev-1205-ge7070e777a

esp-zigbee-lib version.

1.3.2

esp-zboss-lib version.

1.3.2

Espressif SoC revision.

ESP32C6 and ESP32H2

What is the expected behavior?

Be able to join a Zigbee network reliable

What is the actual behavior?

Lot of luck to join a Zigbee network. See usually only the following outout:

BESP-ROM:esp32c6-20220919
Build:Sep 19 2022
rst:0x1 (POWERON),boot:0xc (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:2
load:0x40875720,len:0x1804
load:0x4086c110,len:0xe74
load:0x4086e610,len:0x2e7c
entry 0x4086c11a
I (23) boot: ESP-IDF v5.4-dev-1205-ge7070e777a 2nd stage bootloader
I (24) boot: compile time Jun 27 2024 16:31:16
I (24) boot: chip revision: v0.0
I (28) boot.esp32c6: SPI Speed      : 80MHz
I (33) boot.esp32c6: SPI Mode       : DIO
I (38) boot.esp32c6: SPI Flash Size : 4MB
I (42) boot: Enabling RNG early entropy source...
I (48) boot: Partition Table:
I (51) boot: ## Label            Usage          Type ST Offset   Length
I (59) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (66) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (73) boot:  2 factory          factory app      00 00 00010000 000e1000
I (81) boot:  3 zb_storage       Unknown data     01 81 000f1000 00004000
I (88) boot:  4 zb_fct           Unknown data     01 81 000f5000 00000400
I (96) boot: End of partition table
I (100) esp_image: segment 0: paddr=00010020 vaddr=420a0020 size=32c64h (207972) map
I (152) esp_image: segment 1: paddr=00042c8c vaddr=40800000 size=0d38ch ( 54156) load
I (165) esp_image: segment 2: paddr=00050020 vaddr=42000020 size=99294h (627348) map
I (295) esp_image: segment 3: paddr=000e92bc vaddr=4080d38c size=0281ch ( 10268) load
I (298) esp_image: segment 4: paddr=000ebae0 vaddr=4080fbb0 size=02434h (  9268) load
I (307) boot: Loaded app from partition at offset 0x10000
I (307) boot: Disabling RNG early entropy source...
I (324) cpu_start: Unicore app
I (333) cpu_start: Pro cpu start user code
I (333) cpu_start: cpu freq: 160000000 Hz
I (334) app_init: Application information:
I (336) app_init: Project name:     on_off_light_bulb
I (342) app_init: App version:      6d288fb-dirty
I (347) app_init: Compile time:     Jun 27 2024 16:30:56
I (353) app_init: ELF file SHA256:  XXXXXXXXXX...
I (358) app_init: ESP-IDF:          v5.4-dev-1205-ge7070e777a
I (365) efuse_init: Min chip rev:     v0.0
I (370) efuse_init: Max chip rev:     v0.99
I (375) efuse_init: Chip rev:         v0.0
I (379) heap_init: Initializing. RAM available for dynamic allocation:
I (387) heap_init: At 408166E0 len 00065F30 (407 KiB): RAM
I (393) heap_init: At 4087C610 len 00002F54 (11 KiB): RAM
I (399) heap_init: At 50000000 len 00003FE8 (15 KiB): RTCRAM
I (406) spi_flash: detected chip: generic
I (410) spi_flash: flash io: dio
W (414) spi_flash: Detected size(8192k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
I (427) sleep: Configure to isolate all GPIO pins in sleep state
I (434) sleep: Enable automatic switching of GPIO sleep configuration
I (441) coexist: coex firmware version: 66616e60c
I (455) coexist: coexist rom version 5b8dcfa
I (456) main_task: Started on CPU0
I (456) main_task: Calling app_main()
I (466) phy_init: phy_version 310,dde1ba9,Jun  4 2024,16:38:11
W (466) phy_init: failed to load RF calibration data (0x1102), falling back to full calibration
W (516) phy_init: saving new calibration data because of checksum failure, mode(2)
I (526) phy: libbtbb version: 04952fd, Jun  4 2024, 16:38:26
ZBOSS: common/zb_init_default.c:181   ED build
ZBOSS: common/zb_init_default.c:187   sizes: g_zb 7408 sched 224 bpool 16 nwk 912 aps 3040 addr 284 zdo 840
ZBOSS: common/zb_init_default.c:196   sec 2
ZBOSS: common/zb_init_default.c:198   zcl 600
ZBOSS: common/zb_init_default.c:201   zll 960
ZBOSS: common/zb_init_default.c:207   nvram 140
ZBOSS: common/zb_init_default.c:210   buttons 200
ZBOSS: common/zb_init_default.c:212   err_ind 4
ZBOSS: common/zb_init_default.c:218   scheduler q size 80
ZBOSS: common/zb_init_default.c:221   g_mac 816 g_imac 28
ZBOSS: common/zb_init_default.c:225   Configurable mem build, use ZBOSS lib defaults = 0
ZBOSS: common/zb_init_default.c:231   ZB_IOBUF_POOL_SIZE 80 ZB_NWK_IN_Q_SIZE 40 ZB_MAC_PENDING_QUEUE_SIZE 20 ZB_APS_DST_BINDING_TABLE_SIZE 16 ZB_APS_BIND_TRANS_TABLE_SIZE 20 ZB_N_APS_RETRANS_ENTRIES 27
ZBOSS: common/zb_init_default.c:261   ZB_N_APS_KEY_PAIR_ARR_MAX_SIZE 8 ZB_IEEE_ADDR_TABLE_SIZE 64 ZB_NEIGHBOR_TABLE_SIZE 64 ZB_NWK_ROUTING_TABLE_SIZE 0 ZB_APS_DUPS_TABLE_SIZE 32
ZBOSS: zdo/af_descriptor.c:357   ep_count 1
I (616) main_task: Returned from app_main()
ZBOSS: zdo/zdo_app_prod_conf.c:144   no production config block found
ZBOSS: zdo/zdo_app_prod_conf.c:781   no valid production configuration found, signaling to application
I (746) ESP_ZB_ON_OFF_LIGHT: ZDO signal: ZDO Config Ready (0x17), status: ESP_FAIL
I (756) ESP_ZB_ON_OFF_LIGHT: Initialize Zigbee stack
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:549   COMMISSIONING_STOP: app signal 5 comm status 0
I (776) gpio: GPIO[8]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (776) ESP_ZB_ON_OFF_LIGHT: Deferred driver initialization successful
I (796) ESP_ZB_ON_OFF_LIGHT: Device started up in  factory-reset mode
I (806) ESP_ZB_ON_OFF_LIGHT: Start network steering
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
ZBOSS: mac/mac.c:1039   Oops - error beacon payload is too small buf_sz 15 - drop it
ZBOSS: zdo/zdo_commissioning.c:210   Can't find PAN to join to! param 25
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:549   COMMISSIONING_STOP: app signal 10 comm status 3
I (3066) ESP_ZB_ON_OFF_LIGHT: Network steering was not successful (status: ESP_FAIL)
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
ZBOSS: mac/mac.c:1039   Oops - error beacon payload is too small buf_sz 15 - drop it
ZBOSS: zdo/zdo_commissioning.c:210   Can't find PAN to join to! param 15
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:549   COMMISSIONING_STOP: app signal 10 comm status 3
I (6336) ESP_ZB_ON_OFF_LIGHT: Network steering was not successful (status: ESP_FAIL)
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
ZBOSS: zdo/zdo_commissioning.c:210   Can't find PAN to join to! param 6
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:549   COMMISSIONING_STOP: app signal 10 comm status 3
I (9606) ESP_ZB_ON_OFF_LIGHT: Network steering was not successful (status: ESP_FAIL)
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
ZBOSS: mac/mac.c:1039   Oops - error beacon payload is too small buf_sz 15 - drop it
ZBOSS: zdo/zdo_commissioning.c:210   Can't find PAN to join to! param 6
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:549   COMMISSIONING_STOP: app signal 10 comm status 3
I (12876) ESP_ZB_ON_OFF_LIGHT: Network steering was not successful (status: ESP_FAIL)
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
ZBOSS: mac/mac.c:1039   Oops - error beacon payload is too small buf_sz 15 - drop it
ZBOSS: zdo/zdo_commissioning.c:210   Can't find PAN to join to! param 6
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:549   COMMISSIONING_STOP: app signal 10 comm status 3
I (16146) ESP_ZB_ON_OFF_LIGHT: Network steering was not successful (status: ESP_FAIL)
ZBOSS: commissioning/bdb/zdo_commissioning_bdb.c:349   dev type 2, joined 0, ext_pan_id 0, authenticated 0, tclk_valid 1
[..]

Steps to reproduce.

Have a ESP32C6 or ESP32H2
Was tested with the following hardware:
- Waveshare ESP32-C6-Dev Kit N8
- Waveshare ESP32 H2 Dev Kit N4
- Seeed Studio XIAO ESP32C6 with internal and external antenna (See pin 14)
Have a Home Assistant Yellow in the regular configuration with Zigbee enabled. This also means NO multiprotocol support - Zigbee only.
Checkout esp-zigbee-sdk in revision #4383011.
Go to examples/esp_zigbee_HA_sample/HA_on_off_light.
Run:
```
idf.py set-target esp32c6
```
Enable Zigbee debugging of ESP32 with:
```
idf.py menuconfig
```
Start searching for new Zigbee devices in Home Assistant (Default: http://homeassistant.local:8123/config/zha/add).

Run:

idf.py -p <PORT> erase-flash build flash monitor

More Information.

This was tested with several devices.
If the Home Assistant Yellow is in range of 50cm to 1m it always works.
If another Zigbee device (tested with Philips LOM007 Hue Smart Plug) is in mode Router is in range of 50cm to 1m it always works. Also if the distance between Home Assistant Yellow and the ESP32 is more than 10m.
If another Zigbee device (tested with Philips LOM007 Hue Smart Plug) is in mode Router is 5m away it does also not work.
Another Zigbee End Device (tested with Philips RWL021 Hue Dimmer Switch) which is in the same location as the ESP32 this device always works. Also if the ESP32C6 fails to work. Many other devices teste and all are working as expected.

lpy4105 commented 3 months ago

Could you please provide the pcap file if you have a sniffer?

BTW, did all the three ESP boards fail to work?

xieqinan commented 2 months ago

@blaubaer ,

This was tested with several devices. If the Home Assistant Yellow is in range of 50cm to 1m it always works. If another Zigbee device (tested with Philips LOM007 Hue Smart Plug) >is in mode Router is in range of 50cm to 1m it always works. Also if the distance between Home Assistant Yellow and the ESP32 is more than 10m. If another Zigbee device (tested with Philips LOM007 Hue Smart Plug) is in mode Router is 5m away it does also not work. Another Zigbee End Device (tested with Philips RWL021 Hue Dimmer Switch) which is in the same location as the ESP32 this device always works. Also > if the ESP32C6 fails to work. Many other devices teste and all are working as expected.

According to the above description, the issue seems to be related to the energy of joining the network. The ESP device will determines whether to join the network based on the network's LQI. But the threshold value for joining LQI can be set using the esp_zb_secur_network_min_join_lqi_set() API. The default joining LQI threshold is 32. Could you set it to a smaller value and test it again?

blaubaer commented 2 months ago

Hi @xieqinan

In steps of ~5 I lowered it down to 10. There the connection works reliable, also in the same ranges as the Philips devices.

I would recommend to either change the default value or at least improve the documentation (for example the FAQ section).

xieqinan commented 2 months ago

@blaubaer ,

I would recommend to either change the default value or at least improve the documentation (for example the FAQ section).

No problem, I will consider adding the minimum joining LQI issue to the FAQ. Thanks for your testing and suggestion.

espressif / esp-zigbee-sdk