Adjust Hue scene(s)

[claytonjn]

Allow user to specify a Hue scene to be programatically adjusted, allowing devices like the Hue tap/Hue dimmer to turn on lights directly to the proper color temperature/brightness.

[claytonjn]

I think it's best just to include a write-up on how to set this up manually with an automation. I don't want to make the configuration even more complicated or make this component seem like it is only compatible with Hue lights.

[Bluefries]

I think it's best just to include a write-up on how to set this up manually with an automation. I don't want to make the configuration even more complicated or make this component seem like it is only compatible with Hue lights.

Hello @claytonjn ,

First, thank you for all the good work you're doing developing this component, I am testing it for some time at home on multiple rooms equiped with hue lights and I am very happy with it!

With my very little experience in python I managed to add a function and a call to your sensor.py in order to update Hue scenes same time as the sensor.

Here is what I was able to do until now

• get all scenes that contain "Circadian"
• get each light type so I would adapt the api call (xy/ct/brightness)
• Iterate through all lights in each scene to update it using your values

I would be more than happy to share it, just let me know if you are interested.

Keep up the good work!

[codycodes]

@Bluefries this would be useful for me! I'm going to setup the Hue remote through Home Assistant, but I was hoping to also integrate with other services that set hue scenes outside of HA. I might end up just doing an HA automation to update the scenes, but it might be nicer if I could use your code to do so.

[Bluefries]

@Bluefries this would be useful for me! I'm going to setup the Hue remote through Home Assistant, but I was hoping to also integrate with other services that set hue scenes outside of HA. I might end up just doing an HA automation to update the scenes, but it might be nicer if I could use your code to do so.

Hey @codycodes, I would be happy to share, It's not really pretty, but It will work :)

I brought these small changes to sensor.py file :

• Function update_scene_lights has to be added at the top of the file, as well as the imports
• The function update_sensor exists and has to be updated at the bottom of the file

import re
import requests
import json
import math
from datetime import timedelta
from homeassistant.util.color import (
    color_RGB_to_xy, color_temperature_kelvin_to_mired,
    color_temperature_to_rgb, color_xy_to_hs

hue_gateway = "<your_hue_gateway_IP_here>"
key = "<your_hue_key_here>"

def update_scene_lights(scene, brightness, x_val, y_val, mired):
    url = "http://" + hue_gateway + "/api/" + key + "/scenes/" + scene + "/"
    r = requests.get(url).json()
    r = r['lights']
    _LOGGER.debug("Updating scene id:" + str(scene))
    for val in r:
        url = "http://" + hue_gateway + "/api/" + key + "/lights/" + str(val)
        t = requests.get(url).json()
        type = t['type']
        url = "http://" + hue_gateway + "/api/" + key + "/scenes/" + scene + "/lightstates/" + str(val)
        body = json.dumps({'on': True, 'bri': brightness, 'ct': mired})
        r = requests.put(url, data=body)
        _LOGGER.debug("light id: " + str(val) + " body " + str(body) + " status code: " + str(r.status_code))
        if int(r.status_code) != int(200):
            _LOGGER.error("light id: " + str(val) + " body" + str(body) + " status code: " + str(r.status_code))

def update_sensor(self):
    if self._cl.data is not None:
        self._state = self._cl.data['percent']
        self._hs_color = self._cl.data['hs_color']
        self._attributes = self._cl.data
        min_brightness = 10
        max_brightness = 90
        brightness = int(((max_brightness - min_brightness) * ((100+self._cl.data['percent']) / 100)) + (min_brightness / 100) * 254)
        ct = color_temperature_kelvin_to_mired(self._cl.data['colortemp'])
        rgb = color_temperature_to_rgb(self._cl.data['colortemp'])
        _LOGGER.debug("RGB values: " + str(rgb))
        xy = color_RGB_to_xy(rgb[0],rgb[1],rgb[2])

        url = "http://" + hue_gateway + "/api/" + key + "/scenes/"
        r = requests.get(url).json()

        scenes = []
        for val in r:
            name = r[val]['name']
            if re.match(r"Circadian", name):
                scenes.append(val)

        for val in scenes:
            update_scene_lights(val, brightness, xy[0], xy[1], ct)

If you want to see the logs, make sure you have these lines in your configuration.yaml file :

logger:
  default: error
  logs:
    custom_components.circadian_lighting.sensor: debug

I am updating the scenes every 10 minutes so in configuration.yaml file I put an interval to 600 :

circadian_lighting:
  interval: 600

Also, make sure you have a scene per room called "Circadian" so it will update them.

Edit: Thank you Samuel for noticing the missing ct for the other two lights types and brightness going up to 250 in the api (instead of 100).

[RobertDWhite]

@Bluefries Thanks for sharing. I am attempting to implement now. I just want to confirm that the Hue Key is the "authorized user" that you obtain from the Hue API. Can you confirm? Thanks!

[Bluefries]

Hey @robertomano24

The Hue key is the API key allowing you to authenticate on hue gateway and control hue lights from HA

[RobertDWhite]

Thanks @Bluefries ! I think I will need to work through some kinks. The "Circadian" scenes are not responding to light change currently. Do you have any suggestions why they would not update even though the log does not indicate any errors? Is the Circadian switch in the configuration.yaml interfering somehow?

[Bluefries]

Thanks @Bluefries ! I think I will need to work through some kinks. The "Circadian" scenes are not responding to light change currently. Do you have any suggestions why they would not update even though the log does not indicate any errors? Is the Circadian switch in the configuration.yaml interfering somehow?

Hey, could you double check if the key is correct? Go to \config.storage\core.config_entries file and search for Philips hue. Then, look for username in that block and that would be your philips hue api key. At the same time, check host value and make sure you have the same IP configured in sensor.py

[RobertDWhite]

I went ahead and changed the API key to the key that HA was using (previously, I had just made a new one via the Hue debug interface). Also, the IP is definitely the same, and each room does indeed have a scene named "Circadian" When choosing that scene, it remains at the default value assigned when I made the scene. It does not update, even after 10 minutes.

[Bluefries]

I went ahead and changed the API key to the key that HA was using (previously, I had just made a new one via the Hue debug interface). Also, the IP is definitely the same, and each room does indeed have a scene named "Circadian" When choosing that scene, it remains at the default value assigned when I made the scene. It does not update, even after 10 minutes.

I updated my initial post according to Samuel's feedback, lights should be less orange and brightness better handled now.

[swallace17]

@Bluefries Somehow I accidentally deleted my comment, so I'm adding it back to the bottom of this response. Glad the input was helpful, but I'm afraid the color suggestion I had has not quite done the trick. The colors are still quite a bit more orange than the colors produced by the circadian light switch. I'm going to continue trying to narrow down the issue, but just wanted to get that down on the record. Cheers!

---

Thanks for the writeup! I was able to get my hue scene to update itself with the sensor as you described based on the code you posted. I did run into a couple issues though, one of which I've been able to resolve.

Firstly, the brightness values of my lights were being updated by the update_scene_lights function you wrote, but the brightness values were not what they should have been. I believe the issue is caused passing the brightness value as an integer from 0-100 instead of on a 0-254 scale as the API expects? Not sure I totally understand it, but I modified this line in your code:

brightness = int(((max_brightness - min_brightness) * ((100+self._cl.data['percent']) / 100)) + min_brightness)

to be this line:

brightness = int(((max_brightness - min_brightness) * ((100+self._cl.data['percent']) / 100)) + (min_brightness / 100) * 254)

and everything seems to be behaving as it should.

Secondly- The color values for hue color + ambiance lights are not being adjusted properly. I've been unable to get it functioning properly. I think the issue stems from not sending a color temperature value over the Hue API at all when the light happens to be a Color + Ambiance bulb (extended color). In these two lines:

if type == 'Extended color light': body = json.dumps({'on': True, 'bri': brightness, 'xy': [x_val, y_val]})

you pass a value for brightness, color (xy), but no color temperature. Shouldnt there be 4 values passed to these lights? Brightness, x, y, and ct? I'm confident I'm mistaken here somewhere, but I've worked through about as much as I can and hoping you or someone else might be able to jump in here and help me out. Have you tested your modifications with rooms including color + ambiance bulbs without any issues? It's totally possible I've just screwed something up in my implimentation, but, as far as I can tell, I've just copy and pasted your modifications, and added my own to your brightness calculation. So who knows. Thanks!

[RobertDWhite]

My goal was to match the switch values to the sensor values. In the switch, I had all of my lights set to CT, so the XY values in the code were messing me up and giving me very orange lights. Modifying the portion of @Bluefries code to remove the XY values from the "Extended color lights" worked for me. Now, when I trigger a Hue switch to turn on the Circadian scene, the switch takes over as the day goes on, but no transition occurs within the 5 seconds after turning the scene on. Success!

if type == 'Extended color light':
            body = json.dumps({'on': True, 'bri': brightness, 'ct': mired})

[Bluefries]

My goal was to match the switch values to the sensor values. In the switch, I had all of my lights set to CT, so the XY values in the code were messing me up and giving me very orange lights. Modifying the portion of @Bluefries code to remove the XY values from the "Extended color lights" worked for me. Now, when I trigger a Hue switch to turn on the Circadian scene, the switch takes over as the day goes on, but no transition occurs within the 5 seconds after turning the scene on. Success!

if type == 'Extended color light':
          body = json.dumps({'on': True, 'bri': brightness, 'ct': mired})

I think you found what was wrong about that orange colour, I tested by replacing 'xy' with 'ct' value and I find it a lot better now. I updated the code on this post. Thank you both for your feedbacks/contribution!

[probus]

I tried this and it sort of works, but there are a few issues.

1. The lights are not set to correct brightness in the scenes. Instead, every light is set the same.
2. This doesn't take night mode into account.
3. There should be rate limiting. It's quite easy to overload the hue bridge with too many commands leading to all sorts of problems.

Also, you don't need to set up different scenes per room, one is enough. Unless you want to of course.

[swallace17]

@probus

1 & 2: Both of these issues stem from the fact that, currently, the update_sensor function is duplicating a lot of the basic logic from circadian lights, without accounting for the many different advanced configurations that circadian lights supports (like settings different minimum brightness values for different lights by configuring multiple circadian light switches, or setting a sleep entity, both of which you mentioned). For example, the correct color values for lights in a scene are currently calculated inside the update_sensor function, rather than pulled directly from the rest of the circadian lights where they have already been calculated. Fixing this is complicated. The quick and dirty approach would be to simply modify the logic in the two functions @Bluefries wrote in order to account for these different configurations, but that is an untenable solution imo. As soon as you have accounted for everything, you will have rewritten circadian lights from scratch in order to accommodate it's entire feature-set with hue scene writing functionality working along-side it. A better solution would be to somehow integrate the logic for writing to hue-scenes in the update_scene_lights function into the larger circadian lights code base. @claytonjn has already expressed hesitation towards this approach though, which he expressed here:

I think it's best just to include a write-up on how to set this up manually with an automation. I don't want to make the configuration even more complicated or make this component seem like it is only compatible with Hue lights.

Personally, I think it would be ideal to add this functionality directly into circadian lights, and add support for updating hue scenes via additional configuration in configuration.yaml. I'm planning on forking circadian lights to try to accomplish this, but I've honestly got quite a bit to learn about HA development and do not expect to get very far.

3. Rate limiting is already accounted for. @Bluefries recommended setting the interval to 600 in this comment: https://github.com/claytonjn/hass-circadian_lighting/issues/3#issuecomment-534765184 . This should prevent overloading the hue bridge.

[claytonjn]

FWIW, with the way I have the wiki set up I would be okay with adding support natively BUT I would want it to be one configuration option that would take a regex for scene name(s) to update. If that's set CL would know to update the scenes and it would use the same connection as the HA Hue integration, so no API key would have to be supplied.

Figuring out how to connect without the API key is the main holdup - I'm 99% sure it's possible, but I haven't had the time to figure it out. Heck, I have Hue scene syncing working perfectly in my setup just using simple automations but I haven't had the time to even write that up. :/

[swallace17]

@claytonjn any chance you could post the source for the automations you're using currently? I've been working on it myself, but curious about how you're approaching it. Write up would be appreciated at some point of course, but I'd bet some of us could get it going without the documentation, and might even be able to help contribute to the documentation process. Thanks!

[claytonjn]

Sure, but my automation isn't going to help much, that part is really simple:

automation:
  - id: cl_hue_scenes
    alias: "CL: Hue Scenes"
    trigger:
      platform: state
      entity_id: sensor.circadian_values
    action:
      - service: rest_command.circadian_lighting_clayton_s_vanity_hue_scene
      - service: rest_command.circadian_lighting_cory_s_vanity_hue_scene
      - service: rest_command.circadian_lighting_dinette_hue_scene
      - service: rest_command.circadian_lighting_dining_room_hue_scene
      - service: rest_command.circadian_lighting_family_room_bias_lighting_hue_scene
      - service: rest_command.circadian_lighting_foyer_hue_scene
      - service: rest_command.circadian_lighting_garage_entry_hue_scene
      - service: rest_command.circadian_lighting_hue_go_hue_scene
      - service: rest_command.circadian_lighting_kitchen_hue_scene
      - service: rest_command.circadian_lighting_kitchen_motion_lights_hue_scene
      - service: rest_command.circadian_lighting_living_room_hue_scene
      - service: rest_command.circadian_lighting_master_bathroom_hue_scene
      - service: rest_command.circadian_lighting_master_bedroom_hue_scene
      - service: rest_command.circadian_lighting_second_floor_hallway_hue_scene

The rest commands look like this:

rest_command:
  circadian_lighting_clayton_s_vanity_hue_scene:
    content_type:  'application/json'
    method: put
    payload: >
      {% set brightness = state_attr('switch.circadian_lighting_clayton_s_vanity_circadian_lighting', 'brightness') %}
      {% set colortemp = state_attr('sensor.circadian_values', 'colortemp') %}
      {% if brightness is not none and colortemp is not none %}
        {% set bri = ((((brightness|float) * 254) / 100)|int) %}
        {% set ct = ((1000000 / (colortemp|float))|int) %}
        {
          "lightstates": {
            "1": {
              "on": true,
              "bri": {{bri}},
              "ct": {{ct}}
            },
            "2": {
              "on": true,
              "bri": {{bri}},
              "ct": {{ct}}
            },
            "3": {
              "on": true,
              "bri": {{bri}},
              "ct": {{ct}}
            }
          }
        }
      {% endif %}
    url: !secret rest_command_circadian_lighting_clayton_s_vanity_hue_scene_url

There's nothing too tricky about it, but it's a bit complicated to write up how to find the scene id and light ids for users who have never used the debug page of the Hue bridge.

[RobertDWhite]

Is there an easy way to query all scenes with a particular name so you don't have to find each scene ID individually? I currently have a scene called "Circadian" in each room, that would be excellent if I could simply setup a single automation that affects each scene? If that even makes sense. It would be akin to the previous stuff we chatted about with BlueFries above, but as an automation rather than modification of the sensor or switch.

[claytonjn]

Yes. You would have to query all scenes and do some logic on the JSON to get the IDs with a name that match and the IDs of the lights for those scenes. Then you would have to query each of the lights to get their name to match them up with the lights that are configured in CL.

Personally, that seems like a lot of overhead to put on the hue bridge every time a scene gets adjusted, rather than just figuring it out manually when you set everything up. I have 14 scenes and it only tool maybe 10 minutes to do that all manually, and none of that is really going to change that often.

[RobertDWhite]

Great! Thanks for your time and response. I think I agree, and I think I will just go find all the IDs. Just to clarify, the rest_URL you're using is the URL to the HUE API for that scene, correct?

[claytonjn]

Yeah, so: http://{{ip}}/api/{{api_key}}/scenes/{{scene_id}}

[RobertDWhite]

Nice. Thank you! Looking forward to this implementation as it will likely increase the speed and decrease the Hue overhead like you said as opposed to my current setup. Still love it regardless.

[RobertDWhite]

Got the above working thanks to @claytonjn For others interested in a full .yaml example with a few tips on how I got there, check out my HA here: https://github.com/robertomano24/home-assistant/commit/74824b78a2e541fb4d4369e415563840e9d5b43c

[RobertDWhite]

FYI, just realized I must have done something incorrectly. The scenes are not adjusting (coincidentally, I thought they were because of the time and whatnot I was testing). Therefore, use the example above with that in mind until I fix it.

[RobertDWhite]

@claytonjn I'm at a loss after a few days of troubleshooting. Getting this in the log:

INFO (MainThread) [homeassistant.components.automation] Executing CL: Hue Scenes
 INFO (MainThread) [homeassistant.components.automation] CL: Hue Scenes: Running script
 INFO (MainThread) [homeassistant.components.automation] CL: Hue Scenes: Executing step call service
 ERROR (MainThread) [homeassistant.components.automation] CL: Hue Scenes: Error executing script. Service not found for call_service at pos 1: Unable to find service rest_command/circadian_lighting_basement_hue_scene

In configuration.yaml, I have this:

rest_command:
  # Basement
  circadian_lighting_basement_hue_scene:
    content_type: 'application/json'
    method: put
    payload: >
      {% set brightness = state_attr('switch.circadian_lighting_circadian_basement', 'brightness') %}
      {% set colortemp = state_attr('sensor.circadian_values', 'colortemp') %}
      {% if brightness is not none and colortemp is not none %}
        {% set bri = ((((brightness|float) * 254) / 100)|int) %}
        {% set ct = ((1000000 / (colortemp|float))|int) %}
        {
          "lightstates": {
            "1": {
              "on": true,
              "bri": {{bri}},
              "ct": {{ct}}
            },

            "9": {
              "on": true,
              "bri": {{bri}},
              "ct": {{ct}}
            }
          }
        }
      {% endif %}
    url: !secret rest_command_circadian_lighting_basement_url

In my automations folder, I have:

- id: cl_hue_scenes
  alias: "CL: Hue Scenes"
  trigger:
    platform: state
    entity_id: sensor.circadian_values
  action:
    - service: rest_command.circadian_lighting_basement_hue_scene

Any idea why that error would be thrown? @swallace17 literally copy+pasted my config, etc, then edited for his info. His works, mine does not. Both of us are on the latest pull from HACS. The URLs are good, and the API key is good. Thanks for your time!

[claytonjn]

Per the error: Unable to find service rest_command/circadian_lighting_basement_hue_scene, for some reason the rest command is not being created. That rules out the URLs, API key, etc. Do you see the rest_command.circadian_lighting_basement_hue_scene service listed at /developer-tools/service?

[RobertDWhite]

Yes I do! The service shows up fine: rest_command.circadian_lighting_basement_hue_scene

[claytonjn]

In that case I have no idea what the problem could be...

[RobertDWhite]

Yeah, that is what I was thinking. Haha.

[RobertDWhite]

Not to waste more of your time, but it randomly started working and NOT throwing that error. Now, all my rooms are giving this error:

aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 10.0.0.201:80 ssl:None [Connect call failed ('10.0.0.201', 80)]
2020-03-19 06:59:59 INFO (MainThread) [homeassistant.components.automation] CL: Hue Scenes: Executing step call service
2020-03-19 06:59:59 ERROR (MainThread) [homeassistant.components.rest_command] Client error http://10.0.0.201/api/***************************/scenes/rJz3cc--6UCasfl.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/aiohttp/connector.py", line 936, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 958, in create_connection
    raise exceptions[0]
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 945, in create_connection
    await self.sock_connect(sock, address)
  File "/usr/local/lib/python3.7/asyncio/selector_events.py", line 473, in sock_connect
    return await fut
  File "/usr/local/lib/python3.7/asyncio/selector_events.py", line 503, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('10.0.0.201', 80)

This is closer, anyway!

[swallace17]

@claytonjn I have everything working at this point, but running into some weird issues where the scene is being updated, but the dimmer switches do not seem to be pulling the updated scene regularly. I have been using the iConnectHue app to both create the scenes which the REST commands update, and also to assign the scenes to my dimmer switch buttons. I think this may be the source of my problems somehow.

I went ahead and created the scenes from scratch by direct interface with the hue debug api, but I'm curious as to how you are going about assigning scenes to the buttons on your switches?

[claytonjn]

I assign scenes to switches using the standard Hue app. I use the Android app "all 4 hue" if I need to make advanced changes to the behavior.

[RobertDWhite]

@claytonjn in light of your response to @swallace17, how are you making the scenes show up in the Hue app? I successfully created the scenes via the API and even assigned them to the switch (finally) with the API too. However, the scenes don't show up in the Hue app for me. Do they just automatically show up for you? I am using the API key generated by HA rather than a user I created, so maybe it has to do with that? Otherwise, maybe there is a trick I am missing.

[StSimmons]

@robertomano24 - I created the scenes in the 'Hue Essentials' app and assigned them to the switches there. I then used the API to pull the IDs and set up HASS with the above automations.

[gcorgnet]

Hi, does the automation given by @claytonjn above still work with CL switches that have brightness disabled for them? It looks like after I disabled brightness on my switches, my Hue scenes stopped updating. How could we make it work in that case? Thanks

[RobertDWhite]

@Bluefries have you been able to replicate the functionality you've described previously with these new update? Mine havent worked since .9

[Getslow6]

For everyone's info, there is now a similar functionality built within the Hue app; https://www.reddit.com/r/Hue/comments/zq82eq/philips_hue_natural_light_is_now_officially/