Eve Energy is no longer available after 5 minutes [Matter Firmware]

DerRuedi0 commented 1 year ago

The problem

As soon as I add the Eve Energy (in my case "Nachttischlampe") via the Home Assistant Companion IOS app, it is only usable for a short time. After that, it changes its status to "unavailable" every 5 minutes.

Here's what I tried:

Deleted device in HA and added it in HomeKit, then put it in pairing mode and added it again in HA.
Deleted device from HA and HomeKit and added it only in HA.
restarted HA several times
reinstalled the Matter integration in HA

The device is always reliably responsive via HomeKit, in HA unfortunately only the first 5 minutes after that it remains unavailable.

I used Home Assistant OS with the SkyConnect stick (USB extension is used!).

I could not find any abnormalities in the log.

What version of Home Assistant Core has the issue?

core-2023.9.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Matter

Link to integration documentation on our website

https://www.home-assistant.io/integrations/matter/

Diagnostics information

matter-01402d1c238bba7aaa9b8655b7973567-Eve Energy 20EBO8301-6f2872aa6755313cd1cc3cc2927734ba.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Eve Energy 20EBO8301 von Eve Systems Firmware: 3.2.1 Hardware: 1.1

home-assistant[bot] commented 1 year ago

Hey there @home-assistant/matter, mind taking a look at this issue as it has been labeled with an integration (matter) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `matter` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign matter` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

matter documentation matter source _{^{(message by IssueLinks)}}

DerRuedi0 commented 1 year ago

Also today I tried again to solve the problem somehow, without success. After multiple remove and re-add I add here once the entry from the logbook and the Matter log.

core_matter_server_2023-10-08T18-37-15.556Z.log

forsethc commented 1 year ago

I was able to "solve" this by removing the outlet from HA via HomeKit (I had also added it as a matter device there). Then I could wait a bit before adding it back as a new matter device to HA.

I say "solved" in quotes because the unavailable outlet is still in HA, but the same re-added outlet is also there, so disabling the broken one hides it and makes things work again.

Certainly not ideal, but it at least got me up and running.

DerRuedi0 commented 1 year ago

I finally found a way to solve the problem, I uninstalled the Matter addon from my HA and then added all the devices again. The whole process took me 3 1/2 hours, I hope in the future the problem does not arise again.

marcelveldt commented 1 year ago

@DerRuedi0 if you are in the Apple ecosystem and have one or more Apple border routers (like the Homepod mini or ATV 4K), you shouldn't be using the SkyConnect as Thread radio as that will actually limit the connectivity and better use your existing Apple border routers instead which probably have a much better coverage. At some point in the future we will allow to add the HA SkyConnect as additional Thread border router in the network.

If you use the HA Companion app on iOS it should automatically use the Apple border routers.

@forsethc please try to not hijack an issue report with a different problem. Next time delete the device first in HA (in the device page you can click the delete button, it might be hidden behind the 3 dots action button)

marcelveldt commented 1 year ago

I finally found a way to solve the problem, I uninstalled the Matter addon from my HA and then added all the devices again. The whole process took me 3 1/2 hours, I hope in the future the problem does not arise again.

It could happen again, that is why we still mark Matter as BETA ;-) While the standard (and devices) are making huge leaps forward I will not yet call it ready for primetime just yet and you're getting the side effects of being an early adopter :-)

marcelveldt commented 1 year ago

I think we can then close this issue as you re-did the setup so most probably the devices are now using the Apple border routers ?

forsethc commented 1 year ago

@forsethc please try to not hijack an issue report with a different problem. Next time delete the device first in HA (in the device page you can click the delete button, it might be hidden behind the 3 dots action button)

No intention to hijack anything. When matter devices become unavailable there is no option to delete in HA (the delete button on the device is greyed out), which is why I had to delete it from HA via HomeKit instead. This has happened twice with two different Eve Energy devices and the symptoms are substantially similar to the reported issue.

Just tried to offer a suggestion for what worked for me as a resolution, however imperfect.

marcelveldt commented 1 year ago

No intention to hijack anything.

Yeah, I was in fact talking about your issue report that you couldn't delete the device. If a device becomes unresponsive that is network related (or signal in case of thread).

When matter devices become unavailable there is no option to delete in HA (the delete button on the device is greyed out), which is why I had to delete it from HA via HomeKit instead. This has happened twice with two different Eve Energy devices and the symptoms are substantially similar to the reported issue.

Ah, I agree that our UI is really bad at this point... The button to delete the device is a bit hidden away.

Schermafbeelding 2023-10-13 om 15 14 16

If you delete the device that way, it will also be removed from the Matter controller (and from HA).

forsethc commented 1 year ago

No intention to hijack anything.

Yeah, I was in fact talking about your issue report that you couldn't delete the device. If a device becomes unresponsive that is network related (or signal in case of thread).

I disagree. The device was still available and useable in HomeKit while simultaneously being unavailable in HA.

When matter devices become unavailable there is no option to delete in HA (the delete button on the device is greyed out), which is why I had to delete it from HA via HomeKit instead. This has happened twice with two different Eve Energy devices and the symptoms are substantially similar to the reported issue.

Ah, I agree that our UI is really bad at this point... The button to delete the device is a bit hidden away.

If you delete the device that way, it will also be removed from the Matter controller (and from HA).

Ah, interesting! I do see that delete button now (I was looking at the entity delete button before 🤦). Unfortunately when I re-added the device to HA, it showed up as a new entity on the same device which still cannot be deleted:

I'm happy to report a new issue to not derail the discussion on this one, just not sure where to start as I can't readily reproduce it and the debug logs show nothing about the unavailable device (perhaps because I enabled debug logging after it went unavailable).

marcelveldt commented 1 year ago

If the device goes unavailable in HA while it can be controlled in Apple Home (controlled is key here as Apple Home does not mark the device unavailable realtime) make sure that both HA and Apple are using the same Thread Border Router(s), do NOT use the SkyConnect or HA Yellow in this case as the Apple network will provide better coverage and reliability (until we manage to make the SkyConnect part of the Apple created thread network to act as an additional route).

Make sure that both HA and your Matter devices, including Matter bridges and Thread Border routers as well as your phone is on THE SAME LAYER 2 NETWORK. That is really important. Some people try to get this working with separate IoT vLANS and mDNS forwarding and whatnot, it will simply not work or you will have a ton of issues.

Things to try/consider:

Enable (or disable) IGMP snooping on your switches, it may help or hinder (varies per vendor)
Use Home Assistant OS on bare metal hardware, even hypervisors can mess up the routes.
Using Thread based devices with docker based HA set-up is not possible, WiFi based matter devices should in theory work but in this case an issue with the host network stack can cause severe issues.
Check your WiFi gear and disable stuff like multicast enhancements and the likes (like with IGMP snooping, it may help or hinder per vendor)
If your router has some option called mDNS proxy, or "multicast dns", disable it.
Restart your Apple Border router(s) and/or Home Assistant OS

If it works with Apple and HA it is 100% a network issue you can solve yourself by looking at the above tips.

If both Apple and HA have issues reaching the device it could still be network issues but it could also be just mesh issues or distortion of the radio signal, move the device closer to the border router.

DerRuedi0 commented 1 year ago

If the device goes unavailable in HA while it can be controlled in Apple Home (controlled is key here as Apple Home does not mark the device unavailable realtime) make sure that both HA and Apple are using the same Thread Border Router(s), do NOT use the SkyConnect or HA Yellow in this case as the Apple network will provide better coverage and reliability (until we manage to make the SkyConnect part of the Apple created thread network to act as an additional route).

Make sure that both HA and your Matter devices, including Matter bridges and Thread Border routers as well as your phone is on THE SAME LAYER 2 NETWORK. That is really important. Some people try to get this working with separate IoT vLANS and mDNS forwarding and whatnot, it will simply not work or you will have a ton of issues.

Things to try/consider:

Enable (or disable) IGMP snooping on your switches, it may help or hinder (varies per vendor)

Use Home Assistant OS on bare metal hardware, even hypervisors can mess up the routes.

Using Thread based devices with docker based HA set-up is not possible, WiFi based matter devices should in theory work but in this case an issue with the host network stack can cause severe issues.

Check your WiFi gear and disable stuff like multicast enhancements and the likes (like with IGMP snooping, it may help or hinder per vendor)

If your router has some option called mDNS proxy, or "multicast dns", disable it.

Restart your Apple Border router(s) and/or Home Assistant OS

If it works with Apple and HA it is 100% a network issue you can solve yourself by looking at the above tips.

If both Apple and HA have issues reaching the device it could still be network issues but it could also be just mesh issues or distortion of the radio signal, move the device closer to the border router.

Hi @marcelveldt,

I added all Thread/Matter devices to HA via the HA Companion app. The SkyConnect was meanwhile plugged in and also configured for Matter. How can I now determine which Thread router my endpoints are connected to? How can I set these devices to use one of my HomePod Minis as a Thread router?

marcelveldt commented 1 year ago

Just unplug the SkyConnect and see what happens. If you used the iOS HA Companion app, the devices should be on the Apple Thread network. If you used an Android device, the devices will be connected to the SkyConnect.

DerRuedi0 commented 1 year ago

You are absolutely right, all Matter devices still work. Only my Zigbee devices are no longer accessible since I pulled the SkyConnect. Is there a way to switch the SkyConnect to only Zigbee?

marcelveldt commented 1 year ago

As far as I know this is just a matter of disabling the "multiprotocol" checkbox in the hardware tab. Just to be safe make sure to create a backup first.

austwhite commented 1 year ago

The issue of Zigbee devices unavailable is different to the Eve Energy being unavailable and is to do with the SkyConnect and Multiprotocol. I can't comment on how to fix that as I configured my SkyConnect a dedicated Thread radio and I use a separate Zigbee radio to avoid the issues of losing connections.

Now Eve devices in Matter - I didn't have the 5 minute issue, but every time Home Assistant lost power unexpectedly (which unfortunately happens here sometimes) and sometimes even just with a restart, the EVE Energy devices I use go unavailable. I have to reset them and repair them to get them back. This is repeatable. I do not use Apple Homekit, this is purely within the HA Matter integration and using HA as the Thread Border Router, so I feel there is still an issue with Matter in HA. I know it is still labelled Beta, but still good to have this reported so it can be checked. It won't get out of beta if people don't discuss the bugs and fix them :)

marcelveldt commented 12 months ago

Is there any reason this issue report still needs to be open ? I've lost track due to the various discussions going on here

Rob-Abbott commented 12 months ago

I have the same issue as OP. Devices are an Eve Motion Sensor and a Smartwings window shade, both Matter over Thread. Using SONOFF ZBDongle-E with multiprotocol. ZigBee is running reasonably well with 40 devices but the Matter devices are difficult to add and when added go in and out of availability. For the Eve sensor it was at the same 5 minutes available though after removing it I've had bad luck adding it back.

ZigBee is running on channel 11, while I think Thread is on channel 15. I'd try changing Thread to 11 too if I knew where.

I have no other border routers or controllers. The ZigBee network does have some router nodes.

austwhite commented 12 months ago

If you are using multi protocol, you should have zigbee and thread both on the same channel. They are using the same radio and as such having them on different channels can cause problems. If I recall correctly, this is actually documented in the multi protocol documentation.

Is there a reason you are using channel 11 for zigbee? If not, I would change zigbee to channel 15 and have both on channel 15. This would also take the zigbee network outside of the Wi-Fi channels. You should be able to do this without needing to pair your devices again as they should automatically move to the new channel providing everything is powered on when you change the channel and you wake up the battery devices

Rob-Abbott commented 12 months ago

Thanks, I'll try updating it to see if that helps. Might be few days before I can report back. The channel changing difficulty and solutions are described here.

austwhite commented 12 months ago

Just as a note, if you need to change the thread channel, you can do that through the thread integration in the integrations page. There is an option under the thread configuration to change channel. Go to the thread integration, click configure, click the three dots. There should be an option there to select change channel

Rob-Abbott commented 11 months ago

Things are working better for me. I did change a few other things but I suspect aligning the channels solved it and probably was the same problem OP had. To change align the channels I changed ZigBee to channel 15 to match Thread. I had to redo that network, but that's okay. Luckily when re-pairing devices they mostly retained their names. The curl command in the other issue didn't work and I didn't feel like trying to figure it out.

@austwhite , that would make sense but it doesn't work for the multi-protocol on Sonoff. 🫤

marcelveldt commented 11 months ago

Please be aware that Multi Protocol can still be unstable from time to time and is only recommended for small (test) setups, say like a total of 30 till 40 devices on both zigbee and thread. Above that amount I would really use a separate radio for thread and Zigbee.

home-assistant / core