Scene.create and Scene.turn_on not correctly restoring MQTT RGBW / RGBWW bulb state

[stevenjev]

Home Assistant release with the issue:

0.103.6

Last working Home Assistant release (if known):

Operating environment (Hass.io/Docker/Windows/etc.):

Hass.io system through Docker

Integration:

Description of problem: The new scene snapshot does not properly work with my RGB+W bulbs (running 6.5.0(release-sonoff), through MQTT). After creating the snapshot (of the bulb at a color_temp using only the white LED) and then restoring, it attempts to set the bulb back to white. However, the white that it sets it to uses only the RGB LEDs, when really it needs to restore to using the W LED.

Therefore, the scene snapshot feature is not correctly resetting RGB+W bulb color temperature. I successfully call scene.create and scene.turn_on.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

Traceback (if applicable):

Additional information:

[emontnemery]

@ASNNetworks Can you please share the discovery message for the RGBW light? The discovery message is what zigbee2mqtt publishes to add the light in Home Assistant.

You can probably find it by using this command: mosquitto_sub -v -h <your_mqtt_server> -t "homeassistant/light/#" | grep "zigbee2mqtt/Raam"

Or through HA frontend: Configuration -> Devices -> Name of the light -> MQTT Info

[ASNNetworks]

@ASNNetworks Can you please share the discovery message for the RGBW light? The discovery message is what zigbee2mqtt publishes to add the light in Home Assistant.

You can probably find it by using this command: mosquitto_sub -v -h <your_mqtt_server> -t "homeassistant/light/#" | grep "zigbee2mqtt/Raam"

Or through HA frontend: Configuration -> Devices -> Name of the light -> MQTT Info

Do you mean this?

Received 18:59:13
QoS: 0
Payload: brightness: 204
color:
  h: 14.824
  s: 100
  x: 0.399310247889814
  'y': 0.387626553413704
color_temp: 277
last_seen: '2020-10-26T18:59:13+01:00'
linkquality: 102
state: 'ON'
update:
  state: idle
update_available: false

This was after pushing the service call Kelvin: 3600

[emontnemery]

In that dialog you should be able to find also "MQTT discovery data", can you paste that please?

[ASNNetworks]

availability_topic: zigbee2mqtt/bridge/state
brightness: true
brightness_scale: 254
color_temp: true
command_topic: zigbee2mqtt/Raam/set
device:
  identifiers:
    - zigbee2mqtt_0x7cb03eaa0a0749ed
  manufacturer: OSRAM
  model: LIGHTIFY LED CLA60 E27 RGBW (AC03645)
  name: Raam
  sw_version: Zigbee2MQTT 1.15.0
hs: true
json_attributes_topic: zigbee2mqtt/Raam
name: Raam_light
schema: json
state_topic: zigbee2mqtt/Raam
unique_id: 0x7cb03eaa0a0749ed_light_zigbee2mqtt
platform: mqtt

[emontnemery]

Another question, after you set the light to a color temperature, does it go to white-only mode, meaning after you send the "Kelvin" command it goes from color to white?

[ASNNetworks]

How do you mean? When I change the color temperature, it changes the bulb to the temp I choose in the slider (or with service call, like Kelvin). But the RGB values don’t get removed from the attributes. HA even still colors the icon as the previous color (which is why restoring the snapshot doesn’t work).

This is what I see after changing color temp:

This is when using RGB again:

Basically it only updates attributes that are changed. If you switch back and forth between an RGB value and color temp, the attribute list is identical. I think the restoring snapshots prioritises RGB value if they’re there.

[emontnemery]

@asnnetworks thanks, it's clear now!

For HA to know that the light is in white mode, zigbee2mqtt must:

1. Set the flag https://www.home-assistant.io/integrations/light.mqtt/#white_value to True in the discovery message
2. Set white_value to same as brightness when the light is in white mode and to 0 if it's in color mode This can't be fixed in the MQTT integration, it needs to be changed in zigbee2mqtt, please open a ticket there.

@koenkk HA is a bit messy with RGBWW support, let me know if it's not clear. Here's a PR on the documentation attempting to make it clearer: https://github.com/home-assistant/developers.home-assistant/pull/626/files

@ASNNetworks The screenshots are huge, I'll hide your comment, you can edit the comment with smaller screenshots if you want.

[emontnemery]

I think this ticket can be closed now, since the Tasmota problem is fixed in Tasmota and the zigbee2mqtt problem is confirmed to be a problem with zigbee2mqtt.

[Koenkk]

I didn't read the entire topic, but if there is still a bug in Zigbee2MQTT please open an issue here: https://github.com/Koenkk/zigbee2mqtt/issues

[FransTwisk]

Updating the bulb(s) to the latest Tasmota firmware fixed this issue for me.

For me it did not. I still have this issue (or at least very much the same behaviour) on RGBWW lights running Tasmota 8.5.1. Not sure if there in an issue on Tasmota Github about this or what to do to resolve this. Do I need to reset settings / reset MQTT device?

[emontnemery]

@FransTwisk Can you please share the MQTT discovery message for your RGBWW light, as well as an example of:

• How you configure the light
• What is stored in the scene
• What HA sends to the light when you do scene.turn_on

[FransTwisk]

I have a bunch of different RGBWW/CW lights, all (now) running Tasmota 8.5.1. Home Assistant 0.117.2 with Mosquitto Broken installed via Supervisor. When saving a scene and reloading that scene all these lights get a different color then when the scene was saved. They all revert their color settings it seems. All lights are configured with Home Assistant over MQTT and manual control works fine.

This is the state reported in Home Assistant:

min_mireds: 153 
max_mireds: 500 
effect_list: 0, 1, 2, 3, 4 
brightness: 181 
color_temp: 274 
hs_color: 0, 0 
rgb_color: 255, 255, 255 
xy_color: 0.323, 0.329 
effect: 0 
friendly_name: Living Room - Main RGBW 
supported_features: 23

This is how that state looks in the UI of Tasmota:

And this is what is stored in a new scene when storing it:

- id: '1604348518886'
  name: New Scene
  entities:
    light.living_room_main_rgbw:
      min_mireds: 153
      max_mireds: 500
      effect_list:
      - '0'
      - '1'
      - '2'
      - '3'
      - '4'
      brightness: 181
      color_temp: 274
      hs_color:
      - 0
      - 0
      rgb_color:
      - 255
      - 255
      - 255
      xy_color:
      - 0.323
      - 0.329
      effect: '0'
      friendly_name: Living Room - Main RGBW
      supported_features: 23
      state: 'on'

When I run the scene, Home Assistant then reports:

min_mireds: 153
max_mireds: 500
effect_list: 0, 1, 2, 3, 4
brightness: 181
color_temp: 274
hs_color: 0, 0
rgb_color: 255, 255, 255
xy_color: 0.323, 0.329
effect: 0
friendly_name: Living Room - Main RGBW
supported_features: 23

All the same, but Tasmota UI looks like this for some reason:

Again: when directly setting colors/white in Home Assistant UI it works fine, just not when setting a scene automatically.I guess one setting is overwriting the other and I don't know exactly how to solve this. Did I miss some kind of setting?

MQTT info after switching to scene (not sure if this is correct one?)

Time: '2020-11-02T21:41:46'
Uptime: '0T17:38:16'
UptimeSec: 63496
Heap: 26
SleepMode: Dynamic
Sleep: 10
LoadAvg: 99
MqttCount: 8
POWER: 'ON'
Dimmer: 71
Color: '181,181,181,0,0'
HSBColor: '1,0,71'
White: 0
CT: 338
Channel:
  - 71
  - 71
  - 71
  - 0
  - 0
Scheme: 0
Fade: 'OFF'
Speed: 1
LedTable: 'ON'

[emontnemery]

Yeah, the problem is exactly that one setting is overwriting another. Just to be clear, can you explain what color the light is when you set it to what you want to store in the scene, and what color the light is when you run the scene?

It would be helpful if you can paste the actual discovery message, you can find it in HA UI: Configuration->Devices->(your RGBWW light)->MQTT Info

[FransTwisk]

What I want:

Time: '2020-11-02T23:06:46'
Uptime: '0T19:03:16'
UptimeSec: 68596
Heap: 26
SleepMode: Dynamic
Sleep: 10
LoadAvg: 99
MqttCount: 8
POWER: 'ON'
Dimmer: 80
Color: '0,0,0,103,101'
HSBColor: '0,0,0'
White: 80
CT: 325
Channel:
  - 0
  - 0
  - 0
  - 40
  - 40
Scheme: 0
Fade: 'OFF'
Speed: 1
LedTable: 'ON'

What I get:

Time: '2020-11-02T23:07:01'
Uptime: '0T19:03:31'
UptimeSec: 68611
Heap: 24
SleepMode: Dynamic
Sleep: 10
LoadAvg: 99
MqttCount: 8
POWER: 'ON'
Dimmer: 80
Color: '204,204,204,0,0'
HSBColor: '0,0,80'
White: 0
CT: 325
Channel:
  - 80
  - 80
  - 80
  - 0
  - 0
Scheme: 0
Fade: 'OFF'
Speed: 1
LedTable: 'ON'

I think these are the MQTT discovery messages you need?

[emontnemery]

I did some more testing, and it seems clear it won't work with Tasmota when trying to store the state of and RGBW or RGBWW light when set to a color. The best option for now is to use SetOption37 128 on the Tasmota, this will split the light to one RGB light + one CT light. Another option is to use the Tasmota integration once #42772 is merged, but it's not yet supported by a stable Tasmota release.

[emontnemery]

@FransTwisk No, those are not what I want. You find the discovery message in the HA UI: HA UI: Configuration->Devices->(your RGBWW light)->MQTT Info Screenshot:

[FransTwisk]

Pardon me, I though you specifically meant the latest messages. So in this example I wanted the light to be full white, with 50% warmth in color temperature. So basically orange-white. When restoring the saved scene, it becomes very cold blueish-white.

MQTT Payload:

{
  "name": "Living Room - Main RGBW",
  "state_topic": "living_room_main_rgbw/tele/STATE",
  "availability_topic": "living_room_main_rgbw/tele/LWT",
  "payload_available": "Online",
  "payload_not_available": "Offline",
  "command_topic": "living_room_main_rgbw/cmnd/POWER",
  "value_template": "{{value_json.POWER}}",
  "payload_off": "OFF",
  "payload_on": "ON",
  "unique_id": "AA2106_LI_1",
  "device": {
    "identifiers": [
      "AA2106"
    ]
  },
  "brightness_command_topic": "living_room_main_rgbw/cmnd/Dimmer",
  "brightness_state_topic": "living_room_main_rgbw/tele/STATE",
  "brightness_scale": 100,
  "on_command_type": "brightness",
  "brightness_value_template": "{{value_json.Dimmer}}",
  "rgb_command_topic": "living_room_main_rgbw/cmnd/Color2",
  "rgb_state_topic": "living_room_main_rgbw/tele/STATE",
  "rgb_value_template": "{{value_json.Color.split(',')[0:3]|join(',')}}",
  "effect_command_topic": "living_room_main_rgbw/cmnd/Scheme",
  "effect_state_topic": "living_room_main_rgbw/tele/STATE",
  "effect_value_template": "{{value_json.Scheme}}",
  "effect_list": [
    "0",
    "1",
    "2",
    "3",
    "4"
  ],
  "color_temp_command_topic": "living_room_main_rgbw/cmnd/CT",
  "color_temp_state_topic": "living_room_main_rgbw/tele/STATE",
  "color_temp_value_template": "{{value_json.CT}}",
  "platform": "mqtt"
}

[emontnemery]

Thanks! In addition to the HA representation of the light's state which you pasted earlier, can you please also paste the result of executing command state in Tasmota console when the light is as you want it and what you get? For example:

23:17:51 CMD: state
23:17:51 MQT: tasmota_B94927/tele/STATE = {"Time":"2020-11-02T23:17:51","Uptime":"0T01:28:04","UptimeSec":5284,"Heap":30,"SleepMode":"Dynamic","Sleep":10,"LoadAvg":99,"MqttCount":1,"POWER":"ON","Dimmer":86,"Color":"0,0,0,24,195","HSBColor":"120,100,0","White":86,"CT":462,"Channel":[0,0,0,9,76],"Scheme":0,"Fade":"OFF","Speed":4,"LedTable":"ON","Wifi":{"AP":1,"SSId":"ErikWLAN","BSSId":"78:8A:20:D1:DB:21","Channel":1,"RSSI":100,"Signal":-50,"LinkCount":1,"Downtime":"0T00:00:03"}}

[ASNNetworks]

Is this thread only about Tasmota now? I mentioned some log somewhere above, but I don’t know if I need to create a separate thread or I’m still in the right one.

Edit: my mistake, I wasn’t reading correctly. I see I need to open at zigbee2mqtt Github, this is core.

[FransTwisk]

Tasmota state before scene (not exactly the same color as the logs above I'm afraid, setting this back manually through the Hass UI):

23:21:57 CMD: state
23:21:57 MQT: living_room_main_rgbw/tele/STATE = {"Time":"2020-11-02T23:21:57","Uptime":"0T19:18:27","UptimeSec":69507,"Heap":26,"SleepMode":"Dynamic","Sleep":10,"LoadAvg":99,"MqttCount":8,"POWER":"ON","Dimmer":80,"Color":"0,0,0,106,98","HSBColor":"0,0,0","White":80,"CT":320,"Channel":[0,0,0,42,38],"Scheme":0,"Fade":"OFF","Speed":1,"LedTable":"ON"}
23:21:57 MQT: living_room_main_rgbw/tele/HASS_STATE = {"Version":"8.5.1(tasmota)","BuildDateTime":"2020-10-02T10:09:47","Module or Template":"Teckin SB50v3","RestartReason":"Software/System restart","Uptime":"0T19:18:27","Hostname":"living_room_main_rgbw-0262","IPAddress":"192.168.0.61","RSSI":"82","Signal (dBm)":"-59","WiFi LinkCount":2,"WiFi Downtime":"0T00:00:29","MqttCount":8,"LoadAvg":99}
23:21:57 MQT: living_room_main_rgbw/stat/RESULT = {"Version":"8.5.1(tasmota)","BuildDateTime":"2020-10-02T10:09:47","Module or Template":"Teckin SB50v3","RestartReason":"Software/System restart","Uptime":"0T19:18:27","Hostname":"living_room_main_rgbw-0262","IPAddress":"192.168.0.61","RSSI":"82","Signal (dBm)":"-59","WiFi LinkCount":2,"WiFi Downtime":"0T00:00:29","MqttCount":8,"LoadAvg":99}

Tasmota state after scene:

23:22:44 CMD: state
23:22:44 MQT: living_room_main_rgbw/tele/STATE = {"Time":"2020-11-02T23:22:44","Uptime":"0T19:19:14","UptimeSec":69554,"Heap":25,"SleepMode":"Dynamic","Sleep":10,"LoadAvg":99,"MqttCount":8,"POWER":"ON","Dimmer":80,"Color":"204,204,204,0,0","HSBColor":"0,0,80","White":0,"CT":320,"Channel":[80,80,80,0,0],"Scheme":0,"Fade":"OFF","Speed":1,"LedTable":"ON"}
23:22:44 MQT: living_room_main_rgbw/tele/HASS_STATE = {"Version":"8.5.1(tasmota)","BuildDateTime":"2020-10-02T10:09:47","Module or Template":"Teckin SB50v3","RestartReason":"Software/System restart","Uptime":"0T19:19:14","Hostname":"living_room_main_rgbw-0262","IPAddress":"192.168.0.61","RSSI":"80","Signal (dBm)":"-60","WiFi LinkCount":2,"WiFi Downtime":"0T00:00:29","MqttCount":8,"LoadAvg":99}
23:22:44 MQT: living_room_main_rgbw/stat/RESULT = {"Version":"8.5.1(tasmota)","BuildDateTime":"2020-10-02T10:09:47","Module or Template":"Teckin SB50v3","RestartReason":"Software/System restart","Uptime":"0T19:19:14","Hostname":"living_room_main_rgbw-0262","IPAddress":"192.168.0.61","RSSI":"80","Signal (dBm)":"-60","WiFi LinkCount":2,"WiFi Downtime":"0T00:00:29","MqttCount":8,"LoadAvg":99}

[emontnemery]

@ASNNetworks Let's hold it a bit with the issue on zigbee2mqtt, I think there will need to be changes also in HomeAssistant core 😓

[emontnemery]

@FransTwisk Thanks a lot for the patience, it's clear why it's not working. Your best option right now is to split the light in two by using SetOption37 128 in Tasmota console. After Tasmota has rebooted, your light should show up as one RGB light and one CT light.

[FransTwisk]

Thanks @emontnemery. I'm ok with waiting for Tasmota 9 and the direct integration with Home Assistant, assuming that will also fix these issues. Not sure if 2 lights is what I want :) Might play around with the prerelease as well.

[emontnemery]

Sure, the direct integration will work (once #42772 is merged, so earliest in 0.117.3)

[ASNNetworks]

@ASNNetworks Let's hold it a bit with the issue on zigbee2mqtt, I think there will need to be changes also in HomeAssistant core 😓

strange how HA treats the same lights under zigbee2mqtt vs deCONZ (the same lights work there fine with restoring scenes). I'll hold off with a new ticket then. Is there anything else I can do right now?

[emontnemery]

@ASNNetworks Right, the problem is basically that Home Assistant expects the lights to be either in white mode or in color mode. However, there's no defined way to specify which mode the light is in using MQTT basic schema (used by Tasmota) or MQTT JSON schema (used by Zigbee2MQTT). It means the state stored in the scenes is ambiguous, and running the scene does not work.

[FransTwisk]

Just tested this again with Tasmota 9.0.0.3 pre-release running on one of my lights, and the just released Home Assistant 0.117.3 and this completely solves the issue. Thanks @emontnemery! I'll go and create some scenes now :)

[emontnemery]

@marknl Let's keep this issue about the MQTT integration. Please open a new issue about the Tasmota integration failing to restore state. 👍

[emontnemery]

For Tasmota in MQTT discovery mode, #42887 should fix this.

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[ASNNetworks]

Nope, this issue is still persistent. Basically z2m retains rgb colors while using kelvin. Creating a scene and then restoring, restores the rgb values prior to using kelvin.

[emontnemery]

The zigbee2mqtt issue is discussed at length in #45145, with an improvement for Home Assistant in #45522. However, zigbee2mqtt also needs to be updated to use the new feature. @Koenkk When do you think zigbee2mqtt will be updated? #45522 made it to Home Assistant 2021.2, so it should be OK to use the new feature by now.

[Koenkk]

@emontnemery I will implement this soon. I guess this issue can be closed since it has been solved from the HA side?

[emontnemery]

@Koenkk Great! Yes, I agree this can be closed because MQTT light now has the needed support to report color mode in state message.

[Koenkk]

@emontnemery I'm currently implementing this. Am I right that this only works when the property is set to null and not when it's not present in the published state?

e.g. when {"brightness":200,"color_temp":500,"linkquality":107,"state":"ON"} is published the light seems to be still in color mode (based that bulb icon is still red)

When {"brightness":200,"color_temp":500,"linkquality":107,"state":"ON","color":null} is published it seems to switch to the color temp mode:

Can this be changed to also work for the case where the property is missing from the payload (instead of setting it to null)

[emontnemery]

@Koenkk yeah, you need to explicitly set the color or color_temperature to null. It would make sense to enforce a color mode, but it could be considered a breaking sense.

[emontnemery]

I created an architecture issue: https://github.com/home-assistant/architecture/discussions/519

[Grandma-Betty]

Has this issue ever been solved? I'm struggling with the same issue on my Philips Hue lights with ZigBee2MQTT: Whenever I take a scene snapshot from a colored light state, when trying to restore the original scene, it is always is being set back to white.