Light states reported by diyHue are consistently incorrect

[Flavien]

Describe the bug

When connected via either MQTT or Home Assistant, the lights on the app connected via diyHue don't reflect the correct state (while Home Assistant does).

Steps to Reproduce

1. Connect diyHue via Home Assistant or MQTT (in my case Home Assistant is itself managing lights via MQTT).
2. Link any app to the diyHue bridge (I have tried HueEssentials and All4Hue).
3. Observe the state of lights as indicated on the app.
4. Try turning lights on/off.

Expected behavior

The state shown on the app should correctly reflect the state on Home Assistant.

Actual behavior

The lights on the app reflect an incorrect state: sometimes off when the light is on, and sometimes on when the light is off. After turning a light on or off, the app reverts to an incorrect state after a few seconds.

See screenshot below, all the lights are actually off, yet they show as on on the app:

Logs

diyhue           | 2023-01-25 08:54:09,747 - werkzeug - INFO - 192.168.1.92 - - [25/Jan/2023 08:54:09] "GET /api/v7JN5q-7QXL-RHj95jYR8YJ-yCbkH0MMT1BVYX-5/lights HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:54:25,762 - werkzeug - INFO - 192.168.1.131 - - [25/Jan/2023 08:54:25] "GET /api/v7JN5q-7QXL-RHj95jYR8YJ-yCbkH0MMT1BVYX-5/lights HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:54:56,453 - services.stateFetch - INFO - start lights sync
diyhue           | 2023-01-25 08:55:09,747 - werkzeug - INFO - 192.168.1.92 - - [25/Jan/2023 08:55:09] "GET /api/v7JN5q-7QXL-RHj95jYR8YJ-yCbkH0MMT1BVYX-5/lights HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:25,759 - werkzeug - INFO - 192.168.1.131 - - [25/Jan/2023 08:55:25] "GET /api/v7JN5q-7QXL-RHj95jYR8YJ-yCbkH0MMT1BVYX-5/lights HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:28,645 - werkzeug - INFO - 192.168.1.82 - - [25/Jan/2023 08:55:28] "GET /api/hueess9e9c4111edbfea0242ac140005 HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:30,484 - services.stateFetch - INFO - start lights sync
diyhue           | 2023-01-25 08:55:38,699 - werkzeug - INFO - 192.168.1.82 - - [25/Jan/2023 08:55:38] "GET /api/hueess9e9c4111edbfea0242ac140005 HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:42,283 - werkzeug - INFO - 192.168.1.82 - - [25/Jan/2023 08:55:42] "PUT /api/hueess9e9c4111edbfea0242ac140005/lights/29/state HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:43,498 - services.stateFetch - INFO - start lights sync
diyhue           | 2023-01-25 08:55:45,394 - werkzeug - INFO - 192.168.1.82 - - [25/Jan/2023 08:55:45] "PUT /api/hueess9e9c4111edbfea0242ac140005/lights/29/state HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:48,818 - werkzeug - INFO - 192.168.1.82 - - [25/Jan/2023 08:55:48] "GET /api/hueess9e9c4111edbfea0242ac140005 HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:55:58,874 - werkzeug - INFO - 192.168.1.82 - - [25/Jan/2023 08:55:58] "GET /api/hueess9e9c4111edbfea0242ac140005 HTTP/1.1" 200 -
diyhue           | 2023-01-25 08:56:00,513 - services.stateFetch - INFO - start lights sync

Docker Info (please complete the following information):

• ARM64
• Docker image version: diyhue/core:877
• OS: Raspberry Pi OS (Bullseye)
• Hardware: Raspberry Pi 4

Checklist

• [X] I confirm that this is a bug report
• [X] I confirm that I have completed this template to the best of my ability

[fanaticDavid]

I'm observing the same issue. I just installed diyHue and connected it to MQTT (Zigbee2MQTT). At this very moment, 1 lamp is on which is confirmed by both Z2M and Home Assistant (connected to Z2M). But diyHue shows the lamp as off, and of course then the Hue app is showing that also.

[devinpitcher]

Not a Python developer, but I have been working on getting a local development environment set up so I can troubleshoot and fix this bug, as this affected me as well and forced me to move back to the Hue hub.

[YellowNest]

Any news on this issue? I also have zigbee2mqtt configured with diyhue. The light states are all over the place

[gregorij89]

Hi, I´ve looked into the mqtt service implementation. It completely ignores the state topic from the autodiscovery message. So state has to be reported using topic zigbee2mqtt/. It has to be exactly like this and in no other format.

Here is the code snippet that shows the topics mqtt client is subscribing to: https://github.com/diyhue/diyHue/blob/fe1805d7d7f19c4a9edb7b074c27fb5acbb3483a/BridgeEmulator/services/mqtt.py#L400-L410

Then if the message is in the expected format, the parsing of the device name happens from the topic itself. It is done by searching for the first occurrence of "/" char and taking everything on the right-hand side as a device name: https://github.com/diyhue/diyHue/blob/fe1805d7d7f19c4a9edb7b074c27fb5acbb3483a/BridgeEmulator/services/mqtt.py#L307-L308

Now the service is searching for a device with that name. It is doing so by probing all devices in the configuration and if the device has mqtt protocol, then searching for a property friendly_name (you have it from zigbee2mqtt device message) or if friendly_name doesn´t exist in the config, then comparing with state_topic configuration. https://github.com/diyhue/diyHue/blob/fe1805d7d7f19c4a9edb7b074c27fb5acbb3483a/BridgeEmulator/services/mqtt.py#L195-L214

So if one is using mqtt lights following homeassistant discovery protocol, but not zigbee2mqtt itself, the thing will simply won´t work. Or you have to report the state of the light exactly in the zigbee2mqtt topic. The same happens if you configure your zigbee2mqtt to publish states on a non-default topic.

But, even if you accommodate to the hardcoded topic, only on/off and brightness will be reported. Color temperature or color is not implemented. https://github.com/diyhue/diyHue/blob/fe1805d7d7f19c4a9edb7b074c27fb5acbb3483a/BridgeEmulator/services/mqtt.py#L361-L376

Unfortunately, I have to say, that MQTT implementation is done in a really bad way. Right now, I don´t have much time to try to fix it. Basically, I know nothing about the diyHue architecture and interfaces, so it would be really hard for me to fix this. Even though MQTT parts are simple. Maybe this analysis helps someone else, who has a better understanding of diyHue, to implement the fixes and do the MQTT integration in the right way.