Philips Hue XY Colour not matching input values

[NaffanDroo]

Home Assistant release with the issue:

0.67.0

Last working Home Assistant release (if known): 0.65.6

Operating environment (Hass.io/Docker/Windows/etc.):

Hass.io

Component/platform:

https://www.home-assistant.io/components/hue/

Description of problem: As in https://github.com/home-assistant/home-assistant/issues/13686 which has been partially improved by the merge requests https://github.com/home-assistant/home-assistant/pull/13665 and https://github.com/home-assistant/home-assistant/pull/13828 there are still flaws in the translation from XY input through to actual values in the Attribute values reported in the UI attributes section and in the Hue API Debug client.

As a test, I used the service part of the UI to set the light on using light.turn_on with 0,0 as the XY value to make it easy to see the drift that occurs within the HASS implementation:

{
  "entity_id": "light.living_room_lightstrip",
   "xy_color": [0,0]
}

The value gets skewed to this:

{
  "min_mireds": 153,
  "max_mireds": 500,
  "brightness": 254,
  "hs_color": [
    255.672,
    99.608
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    67,
    0,
    255
  ],
  "xy_color": [
    0.16,
    0.051
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 63
}

When you check the Hue API debug tool, you see a different value again:

    "state": {
        "on": true,
        "bri": 254,
        "hue": 46543,
        "sat": 254,
        "effect": "none",
        "xy": [
            0.1536,
            0.0646
        ],
        "ct": 153,
        "alert": "none",
        "colormode": "xy",
        "mode": "homeautomation",
        "reachable": true
    },

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant): An example scene with this would be:

  light.living_room_lightstrip:
    state: on
    brightness: 254
    xy_color: [0,0]

Traceback (if applicable):

Additional information: Conversely setting from the Hue API debug client (http://bridgip/debug/clip.html) to 0,0 you get the following:

Setting this with a PUT to: /api/myuserid/lights/4/state:

{
   "xy": [0.0,0.0]
}

Getting the state back within the Hue API debug tool:

    "state": {
        "on": true,
        "bri": 254,
        "hue": 47104,
        "sat": 254,
        "effect": "none",
        "xy": [
            0,
            0
        ],
        "ct": 153,
        "alert": "none",
        "colormode": "xy",
        "mode": "homeautomation",
        "reachable": true
    },

Then if you check what Hass has detected:

{
  "min_mireds": 153,
  "max_mireds": 500,
  "brightness": 254,
  "hs_color": [
    255.672,
    99.608
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    67,
    0,
    255
  ],
  "xy_color": [
    0.16,
    0.051
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 63
}

This shows that using the Hue API - if you give it a 0.0,0.0 XY value, it actually sets it, I also tested 0.50,0.50 and that got set in the same way. It would appear that HASS is changing the value before it gets to the light itself.

[amelchio]

None of those values are actual colors. If you set an xy_color: [0.4, 0.4] with Home Assistant, I expect that the debug tool will read out (nearly) the same value?

[NaffanDroo]

Good point! https://en.wikipedia.org/wiki/CIE_1931_color_space

I've retested my scene now and I get the exact values back in the Hue API:

        "xy": [
            0.4575,
            0.4099
        ],

HASS Attributes:

  "xy_color": [
    0.4575,
    0.4099
  ],

From my scene:

  light.living_room:
    state: on
    brightness: 254
    xy_color: [0.4575, 0.4099]

[amelchio]

Thanks. So it seems that we finally fixed the setting of xy colors.

However, I think this issue is still valid if you would do the full test with [0.4, 0.4]. Home Assistant should read out values closer to what it sets and I think I know why it doesn't: we read HS and convert it to xy with the wrong gamut.

[NaffanDroo]

It definitely looks a lot better.

(Re-opening based on your comment)

When I test from the HASS UI it is very close: UI input:

{
  "entity_id": "light.living_room_lightstrip",
  "xy_color": [0.4,0.4]
}

HUE Debug page:

        "xy": [
            0.3999,
            0.3999
        ],

Any other values you want me to test?

[amelchio]

I'm missing the HA state, i.e. the part with "friendly_name".

Sorry that I cannot test this myself, I do not have Hue bulbs.

[NaffanDroo]

No problem...

{
  "min_mireds": 154,
  "max_mireds": 500,
  "brightness": 254,
  "color_temp": 366,
  "xy_color": [
    0.4,
    0.4
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    255,
    234,
    164
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 127
}

[amelchio]

Hmm, that looks like output from 0.65 🤔

[NaffanDroo]

:facepalm: I forgot I put it back to the old working version last night. Will have to update and retest tomorrow

[NaffanDroo]

Right ok, sorry for the confusion with 0.65... this is 0.67.0

UI input:

{
  "entity_id": "light.living_room_lightstrip",
  "xy_color": [0.4,0.4]
}

HASS attributes in UI:

{
  "min_mireds": 153,
  "max_mireds": 500,
  "brightness": 254,
  "hs_color": [
    60.871,
    22.745
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    254,
    255,
    197
  ],
  "xy_color": [
    0.361,
    0.39
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 63
}

Hue API debug client (will give you it all except some identifier stuff blanked:

{
    "state": {
        "on": true,
        "bri": 254,
        "hue": 11081,
        "sat": 58,
        "effect": "none",
        "xy": [
            0.3999,
            0.3999
        ],
        "ct": 275,
        "alert": "none",
        "colormode": "xy",
        "mode": "homeautomation",
        "reachable": true
    },
    "swupdate": {
        "state": "noupdates",
        "lastinstall": "2017-11-28T02:44:34"
    },
    "type": "Extended color light",
    "name": "Living room Lightstrip",
    "modelid": "LST002",
    "manufacturername": "Philips",
    "productname": "Hue lightstrip plus",
    "capabilities": {
        "certified": true,
        "control": {
            "mindimlevel": 40,
            "maxlumen": 1600,
            "colorgamuttype": "C",
            "colorgamut": [
                [
                    0.6915,
                    0.3083
                ],
                [
                    0.17,
                    0.7
                ],
                [
                    0.1532,
                    0.0475
                ]
            ],
            "ct": {
                "min": 153,
                "max": 500
            }
        },
        "streaming": {
            "renderer": true,
            "proxy": true
        }
    },
    "config": {
        "archetype": "huelightstrip",
        "function": "mixed",
        "direction": "omnidirectional"
    },
    "uniqueid": "BLANKED",
    "swversion": "5.105.0.21169"
}

So that was with 0.4,0.4...

Now for my scene (full disclosure: note that this is a Hue room group, but all lights within it get the same):

  light.living_room:
    state: on
    brightness: 254
    xy_color: [0.4575, 0.4099]

UI Attributes:

{
  "min_mireds": 153,
  "max_mireds": 500,
  "brightness": 254,
  "hs_color": [
    46.292,
    54.902
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    255,
    223,
    114
  ],
  "xy_color": [
    0.445,
    0.434
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 63
}

Hue API Debug client

{
    "state": {
        "on": true,
        "bri": 254,
        "hue": 8427,
        "sat": 140,
        "effect": "none",
        "xy": [
            0.4572,
            0.4101
        ],
        "ct": 366,
        "alert": "none",
        "colormode": "xy",
        "mode": "homeautomation",
        "reachable": true
    },
    "swupdate": {
        "state": "noupdates",
        "lastinstall": "2017-11-28T02:44:34"
    },
    "type": "Extended color light",
    "name": "Living room Lightstrip",
    "modelid": "LST002",
    "manufacturername": "Philips",
    "productname": "Hue lightstrip plus",
    "capabilities": {
        "certified": true,
        "control": {
            "mindimlevel": 40,
            "maxlumen": 1600,
            "colorgamuttype": "C",
            "colorgamut": [
                [
                    0.6915,
                    0.3083
                ],
                [
                    0.17,
                    0.7
                ],
                [
                    0.1532,
                    0.0475
                ]
            ],
            "ct": {
                "min": 153,
                "max": 500
            }
        },
        "streaming": {
            "renderer": true,
            "proxy": true
        }
    },
    "config": {
        "archetype": "huelightstrip",
        "function": "mixed",
        "direction": "omnidirectional"
    },
    "uniqueid": "BLANKED",
    "swversion": "5.105.0.21169"
}

[amelchio]

Thanks, that fits my assumption.

I think this file as a custom component should fix it (if I did not make a typo).

[NaffanDroo]

Want me to try and replace the hue python file in the docker container tomorrow?

[amelchio]

Feel free. You can just put it in config/custom_components/light/hue.py.

[Nutti85]

Do I need to do anything else than placing the file in the folder? Do I need to change my config for example?

[NaffanDroo]

@Nutti85 https://www.home-assistant.io/developers/component_loading/ should just override the module

[Nutti85]

OK, I tried that and it didn't solve the probem :(

[NaffanDroo]

@amelchio Unfortunately for me, putting that file in place and restarting HASS causes all of my hue lights to disappear; the file is in place:

core-ssh:~# ls -l /config/custom_components/light/
total 20
drwxr-xr-x    2 root     root          4096 Apr 20 20:36 __pycache__
-rw-r--r--    1 root     root         12648 Apr 20 20:34 hue.py

[amelchio]

Oh. Sorry, I don't know how to fix that :-(. Just delete the file again to get back.

[amelchio]

Just to be clear, you need to be running 0.67 for this file to work.

[NaffanDroo]

Yeah I was on 0.67.0

[Nutti85]

Any updates in this? It's driving me mad 😮

[NaffanDroo]

Guessing it’s not in the 0.68.0b0 beta as the ui colour wheel thinks my yellow lights are green (not done a full test as it’s obvious from the wheel that it hasn’t changed) and there is nothing on the release notes https://github.com/home-assistant/home-assistant/releases/tag/0.68.0b0

[amelchio]

@armills Do you want to join the discussion?

I think I know the issue here: we now set xy but read HS from bulbs, so different xy<->HS converters are used ... but I am unable to test any fix because I do not have Hue.

A proposed (and apparently broken) fix is here: https://github.com/amelchio/home-assistant/commit/918d7191cfdf6a7b1ae49b63e5f7e431c13daf51

[emlove]

Yeah, I think that approach is sensible. If we're using the XY API to set the color, it would make sense that we use the XY API to read the color back, even if we use our converters to switch it to HS internally.

[amelchio]

Cool. Can you test/fix my branch?

[omriasta]

Thanks @Nutti85 I didn't notice this thread. I tested the proposed hue.py as a custom component but no luck. Oddly enough it appeared fine on one of the light groups but not in another... @NaffanDroo not sure how you are running home assistant but maybe you need to change the owner/permissions on the file if the instance is not running as root....

[emlove]

@amelchio Gave it a go with my bulbs and everything seems to be working as expected!

[omriasta]

OK, so the patched hue.py has helped with one of the group lights but not another. You can see states prior to the patch in #13589 Attached is what the UI displays:

And below are the state attribs: min_mireds: 153 max_mireds: 500 brightness: 254 hs_color: 82.163,55.294 effect_list: colorloop,random rgb_color: 202,255,113 xy_color: 0.347,0.511 is_hue_group: true friendly_name: Kitchen supported_features: 63

min_mireds: 153 max_mireds: 500 brightness: 254 hs_color: 38.667,52.941 effect_list: colorloop,random rgb_color: 255,207,120 xy_color: 0.457,0.41 is_hue_group: true friendly_name: Master supported_features: 63

min_mireds: 153 max_mireds: 500 brightness: 254 hs_color: 38.222,52.941 effect_list: colorloop,random rgb_color: 255,205,120 xy_color: 0.459,0.408 is_hue_group: true friendly_name: Living room supported_features: 63

And the bulbs themselves are all set to the default "Bright" scene which comes with Hue. They all are on, with no color, in other words the UI displays Living room and master correctly but not "Kitchen". Master has 2nd gen hue color bulbs(LCT014), LR has 3rd gen hue color bulbs (LCT007) and the Kitchen has 3rd gen color Spot lights (LCT007)

[amelchio]

@omriasta: Thanks for testing, I will move on with the update as a partial solution.

If you could get the three states using the Hue API Debug client that @NaffanDroo mentions above, that would be very helpful.

Please create a new issue because your situation is slightly different and will need a different fix (you are setting colors with an external app, not Home Assistant like in this issue).

[omriasta]

The issue is present even when using the home assistant color wheel to choose yellow the UI still displays green. The only reason I used the hue app scene is because it's easier to set all 3 light groups to the exact same setting of color and brightness in order to compare the attributes. I am pretty sure if I use home assistant to create a scene and set all 3 groups, the kitchen will still display as green. I will test this later today.

[amelchio]

The color wheel is not using xy_color so it is still a different issue.

[NaffanDroo]

Hi guys, thanks for persisting with this; not had much time to test stuff recently, but thought I'd try and confirm that its Ok...

Testing again with 0.68.1: UI Input:

{
  "entity_id": "light.living_room_lightstrip",
  "xy_color": [0.4,0.4]
}

HASS UI Attributes:

{
  "min_mireds": 153,
  "max_mireds": 500,
  "brightness": 254,
  "hs_color": [
    46.154,
    35.686
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    255,
    234,
    164
  ],
  "xy_color": [
    0.4,
    0.4
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 63
}

Hue API:

{
    "state": {
        "on": true,
        "bri": 254,
        "hue": 8402,
        "sat": 140,
        "effect": "none",
        "xy": [
            0.4,
            0.4
        ],
        "ct": 366,
        "alert": "none",
        "colormode": "xy",
        "mode": "homeautomation",
        "reachable": true
    },
    "swupdate": {
        "state": "noupdates",
        "lastinstall": "2017-11-28T02:44:34"
    },
    "type": "Extended color light",
    "name": "Living room Lightstrip",
    "modelid": "LST002",
    "manufacturername": "Philips",
    "productname": "Hue lightstrip plus",
    "capabilities": {
        "certified": true,
        "control": {
            "mindimlevel": 40,
            "maxlumen": 1600,
            "colorgamuttype": "C",
            "colorgamut": [
                [
                    0.6915,
                    0.3083
                ],
                [
                    0.17,
                    0.7
                ],
                [
                    0.1532,
                    0.0475
                ]
            ],
            "ct": {
                "min": 153,
                "max": 500
            }
        },
        "streaming": {
            "renderer": true,
            "proxy": true
        }
    },
    "config": {
        "archetype": "huelightstrip",
        "function": "mixed",
        "direction": "omnidirectional"
    },
    "uniqueid": "BLANKED",
    "swversion": "5.105.0.21169"
}

So that looks good I think.

Trying the scene again:

name: Evening time in the Living room
entities:
  light.living_room:
    state: on
    brightness: 254
    xy_color: [0.4575, 0.4099]

HASS UI Attributes:

{
  "min_mireds": 153,
  "max_mireds": 500,
  "brightness": 254,
  "hs_color": [
    38.667,
    52.941
  ],
  "effect_list": [
    "colorloop",
    "random"
  ],
  "rgb_color": [
    255,
    207,
    120
  ],
  "xy_color": [
    0.457,
    0.41
  ],
  "friendly_name": "Living room Lightstrip",
  "supported_features": 63
}

Hue API debug client:

{
    "state": {
        "on": true,
        "bri": 254,
        "hue": 11081,
        "sat": 58,
        "effect": "none",
        "xy": [
            0.457,
            0.41
        ],
        "ct": 275,
        "alert": "none",
        "colormode": "xy",
        "mode": "homeautomation",
        "reachable": true
    },
    "swupdate": {
        "state": "noupdates",
        "lastinstall": "2017-11-28T02:44:34"
    },
    "type": "Extended color light",
    "name": "Living room Lightstrip",
    "modelid": "LST002",
    "manufacturername": "Philips",
    "productname": "Hue lightstrip plus",
    "capabilities": {
        "certified": true,
        "control": {
            "mindimlevel": 40,
            "maxlumen": 1600,
            "colorgamuttype": "C",
            "colorgamut": [
                [
                    0.6915,
                    0.3083
                ],
                [
                    0.17,
                    0.7
                ],
                [
                    0.1532,
                    0.0475
                ]
            ],
            "ct": {
                "min": 153,
                "max": 500
            }
        },
        "streaming": {
            "renderer": true,
            "proxy": true
        }
    },
    "config": {
        "archetype": "huelightstrip",
        "function": "mixed",
        "direction": "omnidirectional"
    },
    "uniqueid": "BLANKED",
    "swversion": "5.105.0.21169"
}

So its very close - are you expecting it to round it to 3DP? (Guessing the 0.41 is 0.410 but not bothering to show the extra 0 in the client)

[amelchio]

Thanks for the confirmation. Those small differences are from rounding errors because the value actually goes xy -> HS -> xy -> HS -> xy now.

[NaffanDroo]

Haha ok. Being a fan of keeping it simple... does that not seem a bit convoluted?