Setting color temperature when bulb is off does not work

djurny commented 1 month ago

The problem

Summary

When setting color temperature of a bulb that is off, it seems that color temperature value is not sent. Bulb used is a (set of) Ikea E26 1100lm bulbs. Zigbee2MQTT is used to talk to the bulbs.

How to reproduce the issue

Turn on the light
Check current color temperature color temp is 286 according to Z2M frontend
Turn OFF the light
Change color temperature to 454 using Z2M frontend
Change color temperature to 286 using light.turn_on with brightness = 0 Z2M log shows: info 2024-07-05 18:49:13MQTT publish: topic 'zigbee2mqtt/Bulb IKEA E26 1100lm No.11', payload '{"brightness":1,"color_mode":"color_temp","color_options":{"execute_if_off":true},"color_temp":454,"last_seen":"2024-07-05T18:49:13-07:00","level_config":{"on_level":"previous"},"linkquality":39,"power_on_behavior":null,"state":"OFF","update":{"installed_version":65554,"latest_version":65554,"state":"idle"},"update_available":null}'
Refresh color temperature in Z2M frontend: still set to 454, light.turn_on did not have any effect.
Turn on the light
Observe color temperature not changing when turning on the bulb (still at 454 as set at previous step) It appears that color_temp (nor kelvin) is not sent to Zigbee2MQTT when brightness = 0 by light.turn_on.

What version of Home Assistant Core has the issue?

core-2024.7.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Z2M execute if off is enabled in color options. Function seems to do what it says when using the Z2M frontend to change color temperature while bulb is off.

djurny commented 1 month ago

Workaround based on this comment: Add an mqtt.publish message to enable execute_if_off and mqtt.publish a message to set the color_temp wanted before the light is turned on.

//snip
        - repeat:
            for_each: '{{ state_attr(entity_light, "entity_id") }}'
            sequence:
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set/color_options'
                payload: |
                  {
                    "execute_if_off": "true"
                  }
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set'
                payload: |
                  {
                    "color_temp": "{{turn_on_quirk_color_temp_mired_current}}",
                    "transition": "0"
                  }
//snip

This workaround will only work when using Zigbee lights + zigbee2mqtt. In my case, the Philips Hue and Ikea bulbs luckily support the execute_if_off feature for setting the color temperature.

Related issues:

Kuchiru commented 1 month ago

Workaround based on this comment: Add an mqtt.publish message to enable execute_if_off and mqtt.publish a message to set the color_temp wanted before the light is turned on.
//snip
        - repeat:
            for_each: '{{ state_attr(entity_light, "entity_id") }}'
            sequence:
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set/color_options'
                payload: |
                  {
                    "execute_if_off": "true"
                  }
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set'
                payload: |
                  {
                    "color_temp": "{{turn_on_quirk_color_temp_mired_current}}",
                    "transition": "0"
                  }
//snip
This workaround will only work when using Zigbee lights + zigbee2mqtt. In my case, the Philips Hue and Ikea bulbs luckily support the execute_if_off feature for setting the color temperature.

Related issues:

HUE suppress color transition from off state Koenkk/zigbee2mqtt#13156 (reply in thread)

adaptive_lighting.apply does not set color temperature when "turn_on_lights" is set to "false" basnijholt/adaptive-lighting#1022 (comment)

I've been looking for something like this! Does this code work for ambiance lights and RGB bulbs (i'm not using the RGB option in AL)?

I have several ambiance bulbs, old and new.

And just wondering, how/where should i use/apply this snippet?

Thanks!

djurny commented 1 month ago

Hi, I would assume that this will also work for RGB lights. Just check what you see z2m write to your bulbs in the z2m frontend logging at debug level.

As for the snippet, to enable this, you would need to do several things:

Install decluttering-card.
Install adaptive lighting.
Group your lights, even if it's one light per group.
Configure adaptive lighting per light group, which means to have one adaptive lighting entity per light group that have defined.
Use the trimmed down snippets of the dashboard and script below - modify them to suit your dashboard/configuration.
Optionally group the Zigbee lights in zigbee2mqtt to allow the lights to react and behave as 'one' light source. Otherwise AL will control them 'one-by-one'.

See below for a (trimmed down) snippet of my dashboard:

decluttering_templates:
  auto_lighting_header_dimmable:
    card:
      type: custom:mushroom-light-card
      entity: light.[[area_id]]
      tap_action:
        action: call-service
        service: script.light_action_in_area
        data:
          action: toggle
          area: '[[area_id]]'
          turn_on_quirk: 'on'

Here I have made groups of all 'areas' that have lights in them, so area_id here can be 'living_room'.

- platform:   group
  name:       Living room
  entities:
    - light.bulb_ikea_e26_1100lm_no_9
    - light.bulb_ikea_e26_1100lm_no_10

See below for a trimmed down version of the script:

light_action_in_area:
  alias:        "Light action in area"
  description:  "Turn on lighting in an area"
  variables:
    entity_light:                 light.{{area}}
    entity_adaptive_lighting:     switch.adaptive_lighting_{{area}}
    entity_adaptive_brightness:   switch.adaptive_lighting_adapt_brightness_{{area}}
    entity_adaptive_color:        switch.adaptive_lighting_adapt_color_{{area}}
    turn_on_quirk:                '{{ turn_on_quirk|tolower if turn_on_quirk is defined else "off" }}'
    turn_on_quirk_color_temp_mired_current: '{{ state_attr(entity_adaptive_lighting, "color_temp_mired") }}'
    bool_light_on:                '{{ is_state(entity_light, "on") }}'
    bool_adaptive_color_enabled:  '{{ is_state(entity_adaptive_color, "on") }}'
  sequence:
  - choose:
    - conditions:
      - condition: or
        conditions:
        - condition: template
          value_template: |-
            {{ action == "on" or (action == "toggle" and is_state(entity_light, "off")) }}
      sequence:
      - service: adaptive_lighting.set_manual_control
        data:
          entity_id: '{{entity_adaptive_lighting}}'
          manual_control: false
      - service: switch.turn_on
        target:
          entity_id: '{{entity_adaptive_brightness}}'
      - if:
        - condition: and
          conditions:
          - condition: template
            value_template: '{{not bool_light_on}}'
          - condition: template
            value_template: '{{turn_on_quirk|lower == "on"}}'
          - condition: template
            value_template: '{{bool_adaptive_color_enabled}}'
        then:
        - repeat:
            for_each: '{{state_attr(entity_light, "entity_id")}}'
            sequence:
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set/color_options'
                payload: |
                   {
                     "execute_if_off": "true"
                   }
            - service: mqtt.publish
              data_template:
                topic: 'zigbee2mqtt/{{state_attr(repeat.item, "friendly_name")}}/set'
                ## here the "color_temp" option should be replaced with something related to setting
                ## RGB values for the light
                payload: |
                  {
                    "color_temp": "{{turn_on_quirk_color_temp_mired_current}}",
                    "transition": "0"
                  }
      - service: adaptive_lighting.apply
        data:
          entity_id:        '{{entity_adaptive_lighting}}'
          #[AL v1.21] transition:       '{{states[entity_adaptive_lighting].attributes.configuration.initial_transition}}'
          transition:       2
          adapt_color:      true
          adapt_brightness: true
          turn_on_lights:   true

    - conditions:
        - condition: or
          conditions:
            - condition: template
              value_template: |-
                {{ action == "off" or (action == "toggle" and is_state(entity_light, "on")) }}
      sequence:
      - service: light.turn_off
        data: {}
        target:
          entity_id: "{{entity_light}}"

The script still has some leftovers that I do not care to remove (superfluous 'or' condition, 'and' conditions that can be put into one template condition, etc.)

Also, the adaptive_light.apply should be replaceable with a regular light.turn_on if intercept and multi_light_intercept is enabled in AL. But using adaptive_light.apply works fine for me. (If it does not appear to be broken, do not attempt to fix it.)

Feel free to try this out, but it might have some syntax issues due to the trimming.

YMMV as my configuration has morphed into an Ansible templated beast that I sometimes am afraid of touching.

Groetjes,

home-assistant[bot] commented 4 weeks ago

Hey there @emontnemery, @jbouwh, @bdraco, mind taking a look at this issue as it has been labeled with an integration (mqtt) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `mqtt` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign mqtt` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

mqtt documentation mqtt source _{^{(message by IssueLinks)}}

jbouwh commented 4 weeks ago

This is not a place for Zigbee2Mqtt support. To see if there is an issue with mqtt I need the mqtt config for the light and the state update. Home Assistant will assign the color temp that Z2M reports back. In Home Assistant color temp can only be set when the light is turned on. The actual state of the color temp the is to be reported by the light.

djurny commented 4 weeks ago

This is not a place for Zigbee2Mqtt support. To see if there is an issue with mqtt I need the mqtt config for the light and the state update. Home Assistant will assign the color temp that Z2M reports back. In Home Assistant color temp can only be set when the light is turned on. The actual state of the color temp the is to be reported by the light.

Hi Jan, Thanks for your response.

Seems you are already stating the issue:

In Home Assistant color temp can only be set when the light is turned on.

My goal is to set the color temperature of a light that is currently off. My (Ikea and Hue) zigbee bulbs support setting the color_temp when the light is off. This will prevent a color_temp transition of the bulb when it does turn on. (Only brightness will transition from '0' to the brightness set in the light.turn_on service call.)

In order to do this with as little extra effort as possible, I was expecting that a light.turn_on with color_temp set and brightness = 0 will still result in a call to the MQTT component which will reach z2m to set the color_temp without turning on the light.

However, when I do a light.turn_on with brightness = 0, zigbee2mqtt appears to receive nothing. When I issue a light.turn_on with brightness > 0, zigbee2mqtt does receive a command to send something to the bulb.

If this is indeed intended behavior as you implied, then my question is answered.

Groetjes,

jbouwh commented 4 weeks ago

When the light is turned on without options, the light should report back the correct color temp. It does not seem to make sense to set it when te light is off. HA does not support it.

Kuchiru commented 4 weeks ago

So it is intended to always transition from the last state that the lamp was on? for example i have my sleep mode set to 1% brightness for all lamps, when i turn a lamp on who’s last state was 100% brightness the light will turn on full blast transitioning to 1%. HA might not support but this kind of QoL would be great.

jbouwh commented 4 weeks ago

If ha turns a lamp without any additional parameters, then it will I'll just send a state change. If you pick a brightness or color temp from the ui, then ha will send those along. It is up to z2m to send an update to mqtt to reflect to the correct state.

djurny commented 4 weeks ago

Hi Jan, Not sure why you mention a state report or current state of the light in your replies, as that is not really what I'm interested in knowing in my automations/script.

Perhaps the usecase was not clearly laid out, so let me try to expand a bit on that:

We are using the Adaptive Lighting custom integration that changes color temperature and brightness of lights during the day depending on your location and sunrise/set times. When lights are turned off, the integration still calculates the color temperature/brightness according to it's configuration settings. The lights that are off however are not fed with the new settings. So if a light is turned off during noon, where color temperature is 'cold' and brightness is 'high', these settings are not updated when the light is off.

When these lights are turned on again during evening or nighttime - where Adaptive Lighting would have set low brightness and warm color temperatures, the light will still be set to the state of noon and transition from the 'cold' color temperature to the 'warm' color temperature. Some lights will even transition from the high brightness from noon time (last time the light was on) to the newly calculated brightness. Giving the color temperature explicitly to light.turn_on together with a brightness is also not always supported by all lights (Ikea is a good/bad example of that).

I'm lucky that the Ikea and Hue lights I got, only show this transition for the color temperatures, but it seems others (like @Kuchiru) see this effect also with the brightness of the lights when they are turned on at a different of day.

If I recall I also tried other ways, like light.turn_on with just color_temp and lowest brightness with transition = 0 and then have Adaptive Lighting 'take the wheel', but the result looks clonky and not fluid at all. The results also are not the same across bulb brands and/or types.

So this is where the setting the color temperature for lights that are off comes from. For LiFX lights, the integration offers the lifx.set_state service to do this.

Hope this makes it a bit more clear, as I am not that interested in knowing the current color temperature of the light before turning it on, I just want to prevent any transition the light might start when turning it on, by explicitly setting the color temperature to the one Adaptive Lighting has calculated before turning it on.

Hope that makes my (our) situation more clear?

All in all, I do think it would make sense to have a light.turn_on that supports not sending brightness to the light, but just color temperature, let the light(bulb) decide for itself if it needs or wants to turn on in case it receives such event.

Anyway, I also opened an issue at Adaptive Lighting, perhaps something can be done there as well, or I'll find some sort of "suggestion box" to put this one into.

Thanks for your time!

Groetjes,

jbouwh commented 4 weeks ago

I mention the state reports as this determines what HA shows as state for the light, not the values sent to the light. So, related to the title of this issue, and the setting happens outside of HA, it is the state report that determines the actual state of the light in most cases. The light on service actually supports sending an on state without the extra attributes AFAIN, or just a single color temp attribute. You can test this from the Service section with the developer tools.

home-assistant / core