search

zigpy / zha

Zigbee Home Automation
Apache License 2.0
28 stars 26 forks source link

Challenges with Integrating Zigbee Devices in HomeAssistant via Zigbee (ZHA) - Pairing uncompleted #257

Closed Snabelt closed 4 weeks ago

Snabelt commented 1 month ago

Hello everyone,

I am new to the Zigbee environment and have been diving into the details. I am an IT specialist with a focus on system integration and a specialization in cybersecurity.

For my SmartHome setup, I tried integrating a module for my electricity meter into HomeAssistant. I chose the product: Electricity Meter Interface 2, LED by frient. Here is the link: https://frient.com/products/electricity-meter-interface-2-led/ The product is officially compatible with ZHA. See: https://frient.com/compatible-hubs/ Since I didn’t have any Zigbee devices before (except for Philips Hue, which operates in its own ecosystem), I purchased an officially supported Zigbee antenna. Based on this document: https://www.home-assistant.io/integrations/zha/, I decided on the ConBee 3. See: https://phoscon.de/de/conbee3

I managed to get the ConBee 3 running with ZHA, and it seems to work flawlessly. As a next step, I tried connecting the frient Electricity Meter to the network. This also worked perfectly at first. The device was detected, and real values were initially displayed in the entities. However, after some time, I noticed that the entities were no longer updating. LQI and RSSI are displayed as unknown after a while. It seems that the device is not fully integrated into the network or the connection is lost. When I delete and re-add the device, the same issue occurs. The error is therefore reproducible.

I also noticed that after the supposed pairing, the Electricity Meter remains in pairing mode (the LED continues to blink). The pairing mode is maintained for 15 minutes, during which I can reproduce the issue. After these 15 minutes, I need to restart the pairing process on the device.

To rule out whether the issue lies with the end device, I removed a motion sensor from my Hue system and tried to pair it with the Zigbee network. The same thing happens here. I can find the device during the initialization process, and it displays the correct values for a moment, but immediately after initialization, the connection seems to drop. This error is also reproducible.

I have attached a debug log of the connection to the frient device. home-assistant_zha_2024-10-23T09-51-29.577Z.log Unfortunately, I am not yet knowledgeable enough to fully understand the log or the Zigbee functionality to further narrow down the issue. Therefore, I am hoping for your help.

Here are some technical details about the setup: HomeAssistant is running on a virtual machine, installed using this standard: https://www.home-assistant.io/installation/alternative Core: 2024.10.3 Supervisor: 2024.10.2 Operating System: 13.2 Frontend: 20241002.3

Best regards

Hedda commented 1 month ago

For my SmartHome setup, I tried integrating a module for my electricity meter into HomeAssistant. I chose the product: Electricity Meter Interface 2, LED by frient. Here is the link: https://frient.com/products/electricity-meter-interface-2-led/ The product is officially compatible with ZHA. See: https://frient.com/compatible-hubs/ Since I didn’t have any Zigbee devices before (except for Philips Hue, which operates in its own ecosystem), I purchased an officially supported Zigbee antenna. Based on this document: https://www.home-assistant.io/integrations/zha/, I decided on the ConBee 3. See: https://phoscon.de/de/conbee3

Before troubleshooting further can you answer if have connected Zigbee Coordinator via a USB-extension-cable to USB 2.0 port (not USB 3.0 port) and also if you first added any Zigbee Router devices (mains-powered) to the same ZHA Zigbee network as well?

Not saying one or both of those are the root causes in this case for you but they are two very common issues and best practice is therefore to always use Zigbee Coordinator connected via a long USB-extension-cable and place it a little bit away from anything else to get it away from possible sources of EMI/RMI/EMF + connect it to a USB 2.0 port and not USB 3.0 port (as USB 3.0 is a known source of EMI/RMI/EMF unless use a long shielded USB 2.0 extension-cable). Then another best practice it that you normally will first want to add a few mains-powered Zigbee Router devices as those acts as Zigbee repeaters to extent the range of ZHA's Zigbee network mesh before you add any Zigbee End Devices (non-routers) such as example battery-powered Zigbee products, as those usually do not have the signal range to get good reception without connectiing via other Zigbee devices on the same Zigbee network mesh.

Anyway, this specific repository is probably not correct support channel, as normally you post to the forum or Discord and then submit bug to Home Assistant core if get no help, at least if you are not a Python developer that planing to look deeper and possibly fix the problem yourself.

Unfortunately, I am not yet knowledgeable enough to fully understand the log or the Zigbee functionality to further narrow down the issue. Therefore, I am hoping for your help.

There is by the way a very extensive/detailed guide and discussion regarding Zigbee best practices in Home Assistant's community forum that recommend you try following regardless as doing so usually catch a lot of common root causes that can be avoided :

Also suggest you check out these additional community guides on Zigbee too as all of those can help you get up to speed:

Snabelt commented 1 month ago

Hi @Hedda,

Thank you for your quick reply. I reviewed the links but couldn't find any information directly related to my problem. However, they did help me understand the issue a bit better.

Before troubleshooting further can you answer if have connected Zigbee Coordinator via a USB-extension-cable to USB 2.0 port (not USB 3.0 port) and also if you first added any Zigbee Router devices (mains-powered) to the same ZHA Zigbee network as well?

Currently, the Zigbee Coordinator is connected to the front ports of my server. Technically, it is an extension, but I have already ordered a 5-meter extension cable, which should arrive tomorrow. I will update you as soon as I've installed it. Like you mentioned, I also don’t think that this is the root cause. I don't have any Zigbee Routers in my network. The End Device is just 3 meters away from the Coordinator.

Anyway, this specific repository is probably not correct support channel, as normally you post to the forum or Discord and then submit bug to Home Assistant core if get no help, at least if you are not a Python developer that planing to look deeper and possibly fix the problem yourself.

I apologize if I'm using the wrong channel. I'm quite new to this, so I rely heavily on Google searches to find my way. Every problem that seemed close to mine led me to this GitHub issue page, so I decided to open a ticket here. Another reason is that I believe this might be a bug.

Hedda commented 1 month ago

I apologize if I'm using the wrong channel. I'm quite new to this, so I rely heavily on Google searches to find my way. Every problem that seemed close to mine led me to this GitHub issue page, so I decided to open a ticket here. Another reason is that I believe this might be a bug.

It might be a bug, however, the symptoms you describe do also fit with symptoms commonly seen when you have some sources of EMI/RMI/EMF interference too close to either the Zigbee Coordinator and/or the Zigbee device, so first see if you can reproduce/replicate the same symptoms after you are following the best practices needed to avoid EMI/RMI/EMF interference (i.e. USB extension cable and connected to USB 2.0 port). Even so, if it looks like bugs in ZHA then it should first be reported to as an issue Home Assistant's core repository here -> https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+zha

Currently, the Zigbee Coordinator is connected to the front ports of my server. Technically, it is an extension, but I have already ordered a 5-meter extension cable, which should arrive tomorrow. I will update you as soon as I've installed it.

Again, do check and verify that it is either connected to a USB 2.0 port (and not a USB 3.0 port) or/and use a good quality shielded USB 2.0 extension cable (and not and not a USB 3.0 extension cable to a USB 3.0 port as that combination is known to cause EMI/RMI/EMF interference problems). An alternative if computer/server/NAS only has USB 3.0 ports is to buy a USB 2.0 port (with its own external power-supply) and connect via that instead.

Regardless, not using a USB extension cable and connecting to a USB 3.0 port can both be the root cause(s) of the problem, as can be seen in this short but precise demo video clearly showing how EMI/RMI/EMF interference will jam communication and prevent Zigbee Coordinator from receiving signals -> https://www.youtube.com/watch?v=tHqZhNcFEvA

As you can see and understand from that video EMI/RMI/EMF interference can jam Zigbee communications such that it will block or corrupt the signal reception which can cause symptoms like you describe, and such EMI/RMI/EMF interference is more common than you might think, especially for Zigbee which is a low-energy (i.e. weak signals) IoT protocol (.ie. short messages) that uses the 2.4GHz frequency range which has poor radio propagation (bad at penetrating building materials + crappy at bouncing signals) and is the most saturated frequency range when it comes to EMI/RMI/EMF interference from many different sources. Suggest read -> https://community.home-assistant.io/t/guide-for-zigbee-interference-avoidance-and-network-range-coverage-optimization/515752

As evidence see Intel's white paper on how USB 3.0 radio frequency interference impacts low-energy 2.4GHz wireless IoT devices based on ex. IEEE 802.15.4 standard (e.g. Zigbee and Thread) as reference -> https://www.usb.org/sites/default/files/327216.pdf

Hence you always need to get both the Zigbee Coordinator and devices a little bit away from any electricity that might emit interfering EMF (electromagnetic fields) as radio EMI/RMI interference, (at least a couple of feet in distance but preferably more), which includes the computer/server that runs Home Assistant, and you should always also begin by adding mains-powered products that act as Zigbee Router devices before adding any battery-powered devices.

Snabelt commented 1 month ago

Hi @Hedda,

My server only has USB 2.0 ports, and I have now connected a 5-meter extension cable. With this setup, the antenna and the end device were only about 30 cm apart. Unfortunately, the exact same issue persists. The nature of the device requires that it be attached to an electricity meter, but for testing purposes, I also tried it in my office to ensure there were no interference issues. Of course, I didn’t receive valid data in this setting, but even here, it seems that the pairing process is not properly completed. The end device remains in pairing mode even after the supposed completion, and no further communication occurs. When I manually try to read out, for example, the manufacturer information, I receive no response and get an error indicating that the devices are not communicating (even though they are right next to each other).

Summarized, the issue remains the same: During pairing, I receive valid data once, but after that, it seems no further connection is established, and the end device stays in pairing mode.

Even so, if it looks like bugs in ZHA then it should first be reported to as an issue Home Assistant's core repository here -> https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+zha

Should I now post this problem in the Home Assistant's core repository, or can we continue to work through it here?

Snabelt commented 4 weeks ago

Update: After updating Home Assistant Core from version 2024.10.03 to 2024.10.04, the issue still persists.

I also spoke with a friend who has been using ZHA for some time. He’s working with a different antenna, a significantly larger number of Zigbee devices, and is running Core version 2024.10.04. When he tries to add new Zigbee devices, he encounters the same issue as I do. This strongly suggests it may be a software bug.

puddly commented 4 weeks ago

Let's track this in the home assistant repository instead.