Suggestion: Dim to warm mode

[crazyfx1]

I'm using the adaptive lighting component and it works fine, thank you very much.

In parallel I'm testing the new Apple Homekit Adaptive Lighting feature.

In general both features work quite similar. The difference is that Homekit keeps the color generally cooler for 100% brightness (i.e. in the night its about 324 mired in HA). If you then reduce the brightness the color gets warmer and warmer. At around 65% brightness the color is the warmest. The values might differ with other lamps and bridges.

I find this approach very interesting, since you can have rather neutral light in the evening if you need light to work for example. If you want warmer colors you can just dim the light.

Is this mode something you would consider implementing on this component?

[igiannakas]

Ive made a change to the component to continue adapting the colour temperature after sunset downwards, making it dim and warm up further rather than staying a static colour temp. The change dims it down to the sleep temp at the lowest point. I love that effect as it gently warms the light further getting me ready for sleep, rather than stay at a static warm temperature throughout the night.

change us made at the switch.py file

def calc_color_temp_kelvin(self, percent: float, is_sleep: bool) -> float:
    """Calculate the color temperature in Kelvin."""
    if is_sleep:
        return self.sleep_color_temp
    if percent > 0:
        delta = self.max_color_temp - self.min_color_temp
        return (delta * percent) + self.min_color_temp
    if percent < 0:
        delta = abs(self.min_color_temp - self.sleep_color_temp)
        return (delta * abs(1+percent)) + self.sleep_color_temp
    return self.min_color_temp

[crazyfx1]

That's not what I suggested. I suggested to couple the color_temperature to the current brightness of the light.

[RubenKelevra]

Hey @crazyfx1,

thanks for the feature request! :)

I was wondering if sending an override for the sunset for a day would also work for you. This would avoid any manual input - but could, for example, be tied to calendar entries:

https://github.com/basnijholt/adaptive-lighting/issues/57

[crazyfx1]

This has nothing to do with the sunset. I believe I have described my feature request quite clear: I suggest to have the color temperature also depending on the current brightness of the lamp. If you lower the brightness the color should get warmer and warmer.

[hcassady]

I'm interested in this as a possible feature addition too, although I don't think the message of what @crazyfx1 was trying to say came through clear so let me try.

Suppose we have a switch that only adapts color temperature. Currently, the integration sets color temperature as a function of current time. Suppose I have a light at 10%, and AL has set its temperature to 2200K and everything looks great. If I then set the light to 80%, the light is now way too warm for the given light output, so I now manually have to set it to 3300K. It would be fantastic if the integration could automatically make this color change. This also models how incandescent bulbs behave as their color temperature depends on the amount of dimming.

This is how Adaptive Lighting is implemented in Apple's HomeKit - color is warmer at lower brightnesses and cooler at higher brightnesses - with the range changing as a function of the time of day. Something like this:

[jdknoll]

I'm not sure how to vote for this feature, but in my oppinion, it's the biggest difference between Apple's HomeKit's implementation and this implementation. It's also very similar to how a standard dimming incandescent bulb works. A dimming incandescent bulb will transition to warmer color temps when dimming as the light is physically getting cooloer (which is why it's measured in the kelvin temperature of an equivalent incandescent bulb). Another example is the Philips warmglow effect on their non-smart dimming bulbs. In some of my personal automations, I've used the following: {{ (((trigger.to_state.attributes.brightness | float) / 255) * (3500 - 2200)) + 2200 }}

In this case, 2200 is the minimum kelvin, and it's being used as an offset while the percent brightness is being used as a gain multiplied by the range of brightness values, so it can go from 2200 to 3500 kelvin depending on the brightness. I'd assume that an actual implementation may be more nuanced than this.

Thank you so much for your integration!

[lougreenwood]

@RubenKelevra Are all of the tags still relevant? Seems that the more recent replies provide clarifications etc?

Also, +1 for this feature

[th3w1zard1]

I believe this is completed in #87 , you'd just switch adaptive-lighting's sleep switch and it'd automatically dim and set the color temp.

If you're asking for a method for users to change the brightness and adaptive-lighting to simply set a color temp associated with the manually changed brightness, that goes pretty far outside the scope of what adaptive-lighting is made for, as any manual change fires a manually controlled event for said light. You should use an automation to implement this yourself.

Adaptive-lighting's goal is to adapt your lights automatically based on the sun position while allowing manual control when desired. sunrise and sunset variables do exist in the config to allow tweaking.

[crazyfx1]

It's baffling to me how hard this feature request is to understand. The "adaptive lighting" feature implemented by Apple HomeKit has this feature so I'm sure it's in the scope of "adaptive lighting". I suggest people to read how Apple adaptive lighting works regarding the brightness of the bulb.

Furthermore "dim to warm" is a common term used for this kind of lighting. Here is an additional explaination: https://www.ledyilighting.com/dim-to-warm-what-it-is-and-how-does-it-work/

In short form: The color temperature is colder at maximum brightness, warmer at minimum brightness.

In long form: Current state of adaptive lighting: The color temprature is decided by the time of day.

Example (CT is the color temperature) Time: 08:00 CT at 100% brightness: 4000 K CT at 50% brightness: 4000 K

Time: 12:00 CT at 100% brightness: 6500 K CT at 50% brightness: 6500 K

Suggested dim to warm mode: The color temperature is decided by the time of day and the brightness value. Example (CT is the color temperature) Time: 08:00 CT at 100% brightness: 4000 K CT at 50% brightness: 3000 K

Time: 12:00 CT at 100% brightness: 6500 K CT at 50% brightness: 4500 K

(The actual values have to be determined yet, this is just an example).

So in addition to the curve for the time of day, there is a second curve which gradually reduced the color temperature according to the brightness value. The second curve should be toggleable by the switch "dim to warm".

This is a very common feature since there are normal LED light bulbs available which mimic this behavior even without a controlling software. Examples are Philips WarmGlow Bulbs. https://www.usa.lighting.philips.com/consumer/choose-a-bulb/warm-glow-dimmable-led-lighting

[th3w1zard1]

This entire comment can be ignored, I assumed he wanted an entire override for the brightness switch and dim_to_warm to function based off of a manually controlled remote. Derp moment.

I believe I understand the feature request fully. Even read your articles. While I see it's inherent value I strongly feel it's too different from how the core integration works to be implemented in the project.

One major problem I can think of: how will the integration detect a manually controlled event? The take_over_control and detect_non_ha_changes features directly conflict with this feature. Currently the integration looks for any rapid fire light off/on events, or a brightness/colortemp/rgb_color change. You expecting users to execute the KONAMI code with their remotes to trigger a manually controlled event? With this feature, the only way to trigger a manually controlled event is to have users get off the couch, open HASS on their computers, and execute adaptive_lighting.set_manual_control to turn off adaptive-lighting's integration.

If you can answer that question and propose a method of how this feature can coincide with the other features that already exist, I'd be happy to code it for you right now. Would take under an hour to do. (I am not a dev on this project but I can add this on my own fork and submit a PR for the actual devs to decide on)

[th3w1zard1]

For your color temp to brightness relation, how do you feel about the following equation?

color_temp_kelvin = ((min_ct-max_ct)/(min_brightness-max_brightness)*2)(brightness-max_brightness)**2+max_ct

y = a(x-h)^2+k where h,k is the vertex (255,6500) or (max_brightness,max_ct) a = (min_ct-max_ct)/(min_brightness-max_brightness)^2 check: y = (1000-6500)/((1-h)^2)*(x-255)^2+6500 if x=2 then y=1043.221836 when min_brightness=1,max_brightness=255,max_ct=6500,min_ct=1000

for brightness values between 0 and 255 it looks like the photo on a parabolic graph in the example photo, on a brightness scale from 0 to 255, a brightness of 1 produces a ct of 1000 kelvin, 255 produces a ct of 6500 kelvin

[th3w1zard1]

@crazyfx1 did it anyway. grab the build from https://github.com/th3w1zard1/adaptive-lighting/tree/dim-to-warm just set dim_to_warm: true in your config. take_over_control and detect_non_ha_changes will not function with this option set, see my comment above. it works correctly with all other features now. If anyone has suggestions for the curve, I'm all ears.

[th3w1zard1]

PR build exists, test it yourself at #452

[th3w1zard1]

I'm still not totally satisfied with the color temp equation... basically the way this works is it calculates color temp once based on sun position, then using that value it calculates again based on the brightness. My main conflict is I'm unsure whether to calculate the color temp based on the brightness first or based on the sun's position first.

I don't have a apple homekit to test with and dim to warm while a popular feature is largely undocumented from what I could find on Google. I even had to come up with the above equation on my own. Some articles claim that setting brightness to 100 would set the color temp higher than what adaptive-lighting would normally set based on the sun position, others, such as @crazyfx1 's comment claim the opposite. Perhaps another config option is needed?

I need some outside opinions to continue this project further.

[basnijholt]

I cannot find proper plots of people that are using HomeKit, just https://www.reddit.com/r/Hue/comments/k73e7f/colour_temperature_using_homekit_adaptive/.

I think I should be able to derive the equation based on the points there, however, in the thread folks aren't happy with the results.

@th3w1zard1, how did you get to that equation? Trial and error? Or based on something else?

Also, @sizors, remember where you got that plot from?

[th3w1zard1]

@basnijholt It is a very basic parabolic equation I came up with myself. I had the same issue searching online for a decent equation from other implementations.

[crazyfx1]

Thanks for your implementation. I've tried it but unfortunately it throws this error as soon as I'm changing the brightness of a lamp via HA.

`2023-03-25 15:52:40.315 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/config/custom_components/adaptive_lighting/switch.py", line 1054, in _light_event await self._update_attrs_and_maybe_adapt_lights( File "/config/custom_components/adaptive_lighting/switch.py", line 950, in _update_attrs_and_maybe_adapt_lights await self._adapt_lights(lights, transition, force, context) File "/config/custom_components/adaptive_lighting/switch.py", line 988, in _adapt_lights await self._adapt_light(light, transition, force=force, context=context) File "/config/custom_components/adaptive_lighting/switch.py", line 856, in _adapt_light max_brightness = max((max_brightness * 2.55), brightness) UnboundLocalError: local variable 'brightness' referenced before assignment

2023-03-25 15:52:43.159 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/config/custom_components/adaptive_lighting/switch.py", line 775, in _async_update_at_interval await self._update_attrs_and_maybe_adapt_lights( File "/config/custom_components/adaptive_lighting/switch.py", line 950, in _update_attrs_and_maybe_adapt_lights await self._adapt_lights(lights, transition, force, context) File "/config/custom_components/adaptive_lighting/switch.py", line 988, in _adapt_lights await self._adapt_light(light, transition, force=force, context=context) File "/config/custom_components/adaptive_lighting/switch.py", line 856, in _adapt_light max_brightness = max((max_brightness * 2.55), brightness) UnboundLocalError: local variable 'brightness' referenced before assignment`

[th3w1zard1]

@crazyfx1 It's fixed in the last commit I just pushed. I double-checked on my own system and added some failsafes as well. You can either redownload the PR or uncomment lines 839 through 841 (the lines that say 'uncomment' above them) in switch.py (remove the # in front of the line)

A restart is required after updating any custom component files.

[th3w1zard1]

@crazyfx1 Were you able to see dim to warm in your config flow? I couldn't get the config option to display and I'm wondering if it's only on my end. See the three config options at the bottom of the screenshot that have no text.

[crazyfx1]

I've tried the current version of the branch but everything seems very delayed in that version.

1. I turn on the light via HA. It's at 100% brightness and around 5900 K color temperature.
2. I set the light to 50% via HA. It takes several seconds for the color temperature to adapt.
3. After the color temperature was adapted, I'm not able to change the brightness anymore for some seconds. The slider in HA changes but the lamp in the real world does nothing.
4. I need to turn off and turn on the light again for it to be control again.

[th3w1zard1]

@crazyfx1

1. I turn on the light via HA. It's at 100% brightness and around 5900 K color temperature.

Sounds normal if that's around your max color temp, the only time it'd be at the max_color_temp setting is noon.

2. I set the light to 50% via HA. It takes several seconds for the color temperature to adapt.

If you're getting delays it's not due to dim_to_warm it's due to your interval setting. Lights are only adapted on every interval

3. After the color temperature was adapted, I'm not able to change the brightness anymore for some seconds. The slider in HA changes but the lamp in the real world does nothing.

This is very bizarre as non of the new code should cause this, all the new code does is call update_entity to force the state to update to the current state in memory, then uses the current brightness to calc a new color temp for the already existing adapt call. ~Can you post debug logs of this happening, please? And the light's model number or store page so I can search some known issues with hass? Your light might not support a forced update with update_entity() for some reason. If update_entity() is causing the problem I'm very concerned as a lot of my intended code in upcoming changes relies on it.~ EDIT: None of this is relevant. You can try to comment line 853 by putting a # in front of it and see if that fixes it.

4. I need to turn off and turn on the light again for it to be control again.

See my answer for 3

[crazyfx1]

If you're getting delays it's not due to dim_to_warm it's due to your interval setting. Lights are only adapted on every interval

I don't have an interval set, so the default value is 90 seconds. That would mean I have to wait up to 90 seconds for the change? I don't want to be ungrateful but in this case the implementation is not usable. The color temperature should change immediately, just like it does when turning on the light.

[th3w1zard1]

@crazyfx1 If 90 seconds is too long simply lower it in your config... read the readme.

I've asked for your feedback on how it should work several times and you haven't responded to half my comments here. Some actual data points from an actual homekit dim to warm light over a 24 hour period or longer would also be helpful.

[th3w1zard1]

@crazyfx1 You're missing the point on how the integration works. Adaptive Lighting adapts your lights based on the sun position, and it does this by calling light.turn_on with those calculations every interval.

With dim_to_warm, it'll ensure all color temperatures set are calculated again before it sends the new calculation to light.turn_on.

If you're trying to change your brightness outside of adaptive-lighting and you have an actual use case for that, you'll have to wait for a future update to the code or again just lower your interval. You can usually lower it as low as 3-5 seconds without noticeable performance impact, feel free to try even lower.

[hcassady]

I'm super excited to see this getting some attention! I haven't had time to dig into this and see how it's being implemented yet, but I did some digging a few months ago into what Apple is doing for it's implementation. I paired a Nanoleaf Essentials A19, set different brightness's and recorded the color that HomeKit changed it to.

The brightness-color relation is simple, just a linear formula on mired between 12.5 and 100, and it's flat below 12.5:

The time relationship seems more complicated, but I didn't do a good job at getting a bunch of points:

I did this back in December, but I can try and do another round to fill in a full day. Obviously it would be best to have it all on a single day since then it could be related to the sun position of that day, but TBH it just needs to be approximate.

[hcassady]

One possible thought would be to basically use the logic already built into AL. Something like:

Current: Adaptive lighting adjusts the color temp as a function of time

Revised: Adaptive lighting adjusts two color temps as a function of time. One is the 0% color temp, the other is the 100% color temp. The actual value for a given light is the interpolated value between these two points.

[th3w1zard1]

@sizors Hey I just saw your PR. Nice work! I'll take a look at it as soon as I finish this project.

Could you verify if apple homekit cools the lights further than what the sun would normally set it to in adaptive-lighting? E.g. noon brightness is usually around 6500K, let's say it's 2:30PM and brightness is something like 70%. Then you manually set brightness on a light to 100%. Would apple homekit set the color temp colder?

[hcassady]

Yes, it would. Basically, HomeKit's implementation is similar to two parallel instances of this one. Normally, it will control both brightness and color, but if you change one, that becomes manually controlled but the other is still controlled. So, let's say you go from (12 PM, 100%, 6500 K) -> (3 PM, 70%, 5000 K) if you manually set the light to 100% you will get something like (3 PM, 100%, 6100 K).

This is actually how I've got this working too. I set two instances of Adaptive Lighting on the same lights, one for controlling brightness and one for controlling color temp. That way both brightness and color are auto, but if I manually change the brightness, the first one gets set to manual control, but the second one keeps adapting the color. I'd be nice to separate manual_control into "color_manual_control" and "brightness_manual_control" but running two instances isn't too bad.

Yeah take a look, I tried a simpler form of equation but I think in this case simpler is better. Basically I defined a scaling factor, let let Adaptive Lighting determine $T{100}$, scale the resulting mired $M{100}$ by the factor to get $M_0$. Then interpolate between those two points to get the actual value.

It seems to look good. You do have to set the color_temp values in the config higher since they are the color temp @ 100%. Also, with this form of the equation, it gives a single scaling factor that can be tweaked by the user, and possibly varied throughout the day (which HomeKit does) but I think it works pretty good with the static factor too

[crazyfx1]

I've asked for your feedback on how it should work several times and you haven't responded to half my comments here. Some actual data points from an actual homekit dim to warm light over a 24 hour period or longer would also be helpful.

I can't provide feedback to actual values since I'm not using Apple Adaptive Lighting.

@crazyfx1 You're missing the point on how the integration works. Adaptive Lighting adapts your lights based on the sun position, and it does this by calling light.turn_on with those calculations every interval.

No. The integration adapts the light as soon as the light is turned on. And after that it updates every interval.

Your aggressive behavior and constant telling that I'm wrong is quite exhausting. I understand that this integration is behaving like you described. All I'm saying is that it's not usable if it's not updated immediately. Apple Adaptive Lighting is also updating immediately after changing the brightness.

[basnijholt]

Just to clear up some confusion. This integration immediately adapts the lights after they are turned on, then every interval. It can detect any change that is made within Home Assistant (any light.turn_on event) and it will mark the lights as manually controlled. For lights that are operated outside of Home Assistant, there are some methods to detect whether the changes are significant or not. If they are, they will also be marked as manually controlled.

[basnijholt]

Also a little note because this issue is heating up 😅

Chatting online can be a bit tricky sometimes. We can't hear each other's tones or see facial expressions, so things can easily get lost in translation. It's important to assume that people mean well and try not to take things too personally.

[crazyfx1]

It can detect any change that is made within Home Assistant (any light.turn_on event) and it will mark the lights as manually controlled.

Yes, I understand that this is a problem and @th3w1zard1 also identified this problem earlier in this thread. The obvious solution would be, that the light is not marked as manually controlled when changing brightness and "dim to warm" is enabled. Furthermore every brightness change should trigger the light adaption.

If that's not possible then maybe this feature can't be implemented in this integration.

[hcassady]

Okay actually this doesn't work. The color-control instance ends up canceling the is_manually_controlled state of the brightness-control instance.

This is actually how I've got this working too. I set two instances of Adaptive Lighting on the same lights, one for controlling brightness and one for controlling color temp.

So really there should be two manual control flags, brightness_manually_controlled and color_manually_controlled. That way you could change the brightness, have it go into manual control mode, and have the color keep updating. However, it still would allow for changing the color manually since that could also enter a manual control state as well. This is basically how it works in HomeKit.

I agree with @crazyfx1 that the color changes should happen right after a brightness change. I don't think this would be too hard, basically call the same function that instantly adapts the light after a turn_on event after a brightness change event if color_manually_controlled is set. But I think getting the manual control stuff working is more important since setting the interval to 5 sec works good enough.

Honestly I think this is a killer feature and I'm super stoked it's already working as good as it is. Thank you @th3w1zard1 for getting this started, there's still a few details to iron out but I don't think it's too far off.

[th3w1zard1]

I've asked for your feedback on how it should work several times and you haven't responded to half my comments here. Some actual data points from an actual homekit dim to warm light over a 24 hour period or longer would also be helpful.

I can't provide feedback to actual values since I'm not using Apple Adaptive Lighting.

@crazyfx1 You're missing the point on how the integration works. Adaptive Lighting adapts your lights based on the sun position, and it does this by calling light.turn_on with those calculations every interval.

No. The integration adapts the light as soon as the light is turned on. And after that it updates every interval.

Sorry if I wasn't clear. When you turn your light on adaptive lighting does indeed adapt your lights instantly. With my PR, anytime adaptive lighting adapts a light, it'll do the whole dim to warm thing. There are several times this may happen outside of an interval (service calls, switches turning on, hass restarting, etc)

When the light is off, adaptive lighting will simply use the brightness value it was already going to set with the sun in its dim to warm calculations.

I understand that this integration is behaving like you described. All I'm saying is that it's not usable if it's not updated immediately. Apple Adaptive Lighting is also updating immediately after changing the brightness.

You've commented that it wasn't useable to you in its current format. I hear you and want to work with you. I responded back asking 'Please answer my previous questions on how you'd like it to function.' There are several pending concerns on how the equation should work, I'd love to hear your input on the graphs we posted.

It can detect any change that is made within Home Assistant (any light.turn_on event) and it will mark the lights as manually controlled.

Yes, I understand that this is a problem and @th3w1zard1 also identified this problem earlier in this thread. The obvious solution would be, that the light is not marked as manually controlled when changing brightness and "dim to warm" is enabled. Furthermore every brightness change should trigger the light adaption.

I hear you and considered removing the brightness check, but that also raises the problem of use cases where users may want the brightness check to remain. A possible solution would be to add another config option where manual brightness checks are disabled while dim to warm is enabled, which in my opinion should default to off. This wouldn't be as simple as checking the state of the adapt brightness switch as that means adaptive lighting wouldn't be adapting the brightness at all.

[th3w1zard1]

It's worth mentioning #486 improves performance significantly while the interval setting is extremely low. I'm able to run an interval of one second without any noticeable impact. See https://github.com/basnijholt/adaptive-lighting/pull/486#issuecomment-1486129797 for CPU usage comparisons

[th3w1zard1]

@sizors Could you post a graph of your new equation on the pull request? Perhaps something on https://www.desmos.com/calculator ?

[th3w1zard1]

I've added the config option dim_to_warm_brightness_check defaulting to False in that PR. I've also taken @sizors 's advice and it now sets the ct based on the difference between both CT calcs.

E.g. a brightness of 6500 and max ct of 6500 would calculate the dim to warm ct to be 6500, a sun position calc at 3pm may be 4500 the difference would net adaptive-lighting to use 5500K in the light.turn_on call.

[LindyBalboa]

This is actually how I've got this working too. I set two instances of Adaptive Lighting on the same lights, one for controlling brightness and one for controlling color temp.

@sizors Would you mind expanding upon how you did that? I don't see any way to disable brightness or color control either directly or indirectly?

Edit: I see the relevant entities now. Do they maintain their position even after HA restarts?

[RubenKelevra]

Edit: I see the relevant entities now. Do they maintain their position even after HA restarts?

Yes

[basnijholt]

[Copy pasting this message in a few recent open issues]

I just wanted to take a moment to express my heartfelt thanks to everyone that is active in this repo. Your contributions, from answering questions to addressing issues, have been invaluable. It's amazing to see how supportive and helpful our community is!

Adaptive Lighting is all about enhancing your living spaces with smart, sunlight-responsive lighting. We've had quite a few discussions and open issues recently, and I see this as a positive sign of our community's engagement and growth. If you come across anything in the documentation that's unclear or if you have suggestions for improvement, please don't hesitate to share!. Your feedback is crucial for making Adaptive Lighting better for everyone.

On a personal note, I've recently welcomed twin boys into my family, which has been an incredible and life-changing experience. As you can imagine, my time is now more limited, and while I'm doing my best to keep up with the project, there may be delays in my responses. I appreciate your understanding and patience during this time.

Rest assured, I'm fully committed to addressing any bugs, especially those related to new Home Assistant updates, as swiftly as possible. I understand that many issues may stem from hardware limitations or misunderstandings about things like Zigbee groups. Your continued support and collaboration in helping each other out not only strengthen our community but also enhance the Adaptive Lighting experience for all.

Thank you once again for your understanding, patience, and support. Let's keep our houses well lit and adaptive for maximal enjoyment of life! 🌞🏠🌙