search

zigbee2mqtt / hassio-zigbee2mqtt

Official Zigbee2MQTT Home Assistant add-on
https://www.zigbee2mqtt.io
Apache License 2.0
1.11k stars 390 forks source link

Empty devices list #594

Open JohannCR opened 5 months ago

JohannCR commented 5 months ago

Description of the issue

All devices disappeared from the addon's devices list. They are still present in the integration page, and still work. Since 1.36.1-1 I've been uninstalling and reinstalling switching between the stable and edge versions to try to fix some ghost bindings that broke a couple of devices. Since then, I don't know why, the devices list is now empty. As if the database.db was corrupted in some way... When I open it, it looks good and contains all devices though.

Tried clearing cache, rebooting addon, reloading integration, rebooting HA, no change.

Is there a way to fix this ? Like removing the database, reboot and then putting it back or something like that ?

Addon version

edge. Can't find commit hash, but it's the same on stable anyway

Platform

Core 2024.4.1 Supervisor 2024.04.0 Operating System 12.1 Interface utilisateur 20240404.1

Logs of the issue (if applicable)

Nothing in the logs

JohannCR commented 5 months ago

For those wondering, I have not found any solution other than deleting everything and restarting from scratch repairing every device... That was a pain 🤣

nrehn commented 4 months ago

Hi. I have the same issue now. Device list is empty but everything works. It does not work to join devices either. When pressing the button in GUI nothing happens.

martintm86 commented 3 months ago

Hello, for me it happens today after Update 2024.6.0. Everything works but i cant configure anything in the Zigbee2MQTT menu. All empty and without function...

Reboot and clearing cache does not fix that...

Astromechdroide commented 3 months ago

Hello, for me it happens today after Update 2024.6.0. Everything works but i cant configure anything in the Zigbee2MQTT menu. All empty and without function...

Reboot and clearing cache does not fix that...

I have the same problem. Is there a solution?

JohannCR commented 3 months ago

I have the same problem. Is there a solution?

Only one I tested successfully was to uninstall and delete all devices, reinstall and repair everything... Painful. Plus I don't know why (may not be linked to this issue) but I have more devices disconnecting from the network since. Unfortunately I also changed my coordinator so I don't know what caused those disconnections. Again, may not be linked to z2m. Try repairing all if your network isn't too big and don't hesitate to report back 😉

martintm86 commented 3 months ago

After update to 2024.6.1 everything is fixed for me....

Operrr commented 3 months ago

My version is XA 2024.6.1, version Z2M 1.38.0-1 the list of devices is empty, but everything seems to work. Connecting new devices does not work, there are no logs, no information. Help me please!

1

booroondook-eng commented 3 months ago

The same problem, but I see empty devices list (and also empty Dashboard) in ingress mode only. When I switch to normal mode (open http://xxx.xxx.xxx.xxx:8099 in the new browser window) I see completed list of my devices and working Dashboard). In other words: The problem with empty devices list appears in the ingress mode only.

Operrr commented 3 months ago

I can't open the page directly

Operrr commented 3 months ago

Is it really possible that no one else has such a problem and there will be no help?

martintm86 commented 3 months ago

For me the Problem comes with my Apache Reverse Proxy. In the Browser Console i have seen that it can't open a special WebSocket URL. After Adding it to reverse Proxy config anything is ok.... Before if i connect directly to HA by IP the Plugin was OK

ProxyPass /api/hassio_ingress ws://192.168.2.4:80/api/hassio_ingress ProxyPassReverse /api/hassio_ingress ws://192.168.2.4:80/api/hassio_ingress

pdm0 commented 2 months ago

Is it really possible that no one else has such a problem and there will be no help?

You are not alone in this. Unfortunately I am in the same situation and despite considerable efforts have not found any fix.

Operrr commented 2 months ago

I solved the problem. If you clear the "devices" section directly in "/homeassistant/zigbee2mqtt/configuration.yaml" then all device names and descriptions will disappear, but the device list will work correctly. After everything worked, I manually entered device descriptions using the old backup. Now everything is fine.

booroondook-eng commented 2 months ago

I found another way to resolve this problem. My configuration.yaml contains the following string: devices: devices.yaml ...and all devices' configuration stores in the devices.yaml file. I noticed that this file has very large size: about 5 MB. The reason for this size was that this file contains icons of all my devices in base64 format. For example:

'0xa4c1380d5fXXXXXX':
  friendly_name: Zigbee Lamp 2
  icon: >-
    data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAJYAAACWCAYAAAA8.... etc

I removed all the lines with icons, the file was reduced to about 4 kilobytes, and the problem with the empty device list completely disappeared!

Netpoet commented 1 month ago

@booroondook-eng Thanks a lot! I had the same issue, and your hint helped! I cleared all the image information lines from the config, restarted the coordinator and Home Assistant, and voilà, everything's back! <3

ahamro commented 1 month ago

@booroondook-eng

I found another way to resolve this problem. My configuration.yaml contains the following string: devices: devices.yaml ...and all devices' configuration stores in the devices.yaml file. I noticed that this file has very large size: about 5 MB. The reason for this size was that this file contains icons of all my devices in base64 format. For example:

'0xa4c1380d5fXXXXXX':
  friendly_name: Zigbee Lamp 2
  icon: >-
    data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAJYAAACWCAYAAAA8.... etc

I removed all the lines with icons, the file was reduced to about 4 kilobytes, and the problem with the empty device list completely disappeared!

You're the best! In my case the devices are in zigbee2mqtt/configuration.yaml and after weeding out the icon lines the devices list and the dashboard were populated again. Including the icons. Great! Thanks for sharing your solution!

xelemorf commented 1 week ago

Great finding, I was getting nuts by bumping into this bug all the time!

I have identified that the "Localise device images" feature is doing this, flooding the Z2M configuration.yaml with embedded Base64 images to a point where Z2M just cannot handle.

6eaceabb-0867-44a0-9375-5798a016fb8e

In my case it also was not able to finish the localization of images as it was already too much with my 147 devices, failed around 100. I was also reconsidering my life choices while clearing the config manually by having a file lenght of 4 318 294 characters (4.11 MB), and the cleaned up file is 43 540 character (42.51k). Anyway, Notepad++ was a great help in this by collapsing the "icon: >-" segments and deleting each from the start of the next line.

@Koenkk, can you please remove this "Localise device images" feature completely from Z2M for now or add a warning/confirmation that things can go badly when having numerous devices? I think embedding images and adding substantial amount of data into the main config file will always end up badly, maybe having them externally stored as jpg/png files in an dedicated z2m_img folder and referencing them by device id would be more fruitful. Thank you.

Koenkk commented 1 week ago

@xelemorf interesting finding, I guess z2m didn't startup anymore in your case?

xelemorf commented 1 week ago

@xelemorf interesting finding, I guess z2m didn't startup anymore in your case?

@Koenkk Z2M could still start without showing any errmsg but none of my devices were present on the default UI, while they were still working as expected in all HA automations, scripts and on Lovelace. Had to enable the front-end tcp-listen port 8099 to be able to see my devices in Z2M there (sidenote for the front-end 8099: it does not have any protection and could login without auth).

After the fix I've mentioned everything works as expected. Using currently available latest Z2M stable with latest HAOS.

Btw once had this issue, I also could not get the file from "request z2m backup" or "download state", and "network map" also never finished - works after the fix.

Koenkk commented 1 week ago

Interesting, do you still have the debug log of this? Could also be a frontend issue

See this on how to enable debug logging.

xelemorf commented 1 week ago

I'll try to find some time over the weekend to collect debug logs.

silfax commented 1 week ago

@xelemorf interesting finding, I guess z2m didn't startup anymore in your case?

@Koenkk Z2M could still start without showing any errmsg but none of my devices were present on the default UI, while they were still working as expected in all HA automations, scripts and on Lovelace. Had to enable the front-end tcp-listen port 8099 to be able to see my devices in Z2M there (sidenote for the front-end 8099: it does not have any protection and could login without auth).

After the fix I've mentioned everything works as expected. Using currently available latest Z2M stable with latest HAOS.

Btw once had this issue, I also could not get the file from "request z2m backup" or "download state", and "network map" also never finished - works after the fix.

I can confirm the same behavior with probably the same cause ('Localize device images'). Z2M appears to be working normally, device list only available by accessing it via port 8099. Didn't have the time to try cleaning devices.yaml (currently >5 MB). Will do it during weekend. Will also try to collect some logs before as well.