Workaround for light power-on default state

[ebaauw]

Folks, I would like your thoughts on using homebridge-hue to overcome the issue of the Hue light power-on defaults. Note: by "power on" or "power off" I mean connecting/disconnecting a light to/from the mains power, e.g. by using a traditional wall switch (so 20th century ;-).

Issue When powered on, most lights, notably the Philips Hue bulbs and spots, turn on using some unholy power-on defaults for brightness and colour, instead of using the previous light state (like when turning a light on through the Hue bridge). This is a nuisance when powering the lights on using a wall switch, but becomes plainly obnoxious when mains power is restored after an outage. Of course, Murphy will make sure this happens in the middle of the night or when you're away for a couple of days. Even though this issue is by far the largest topic on the Hue developer's forum, it doesn't look like Philips will be changing this behaviour.

For each Zigbee device (lights but also switches and motion sensors), the Hue bridge keeps track whether that device is reachable. When the Zigbee communication to that device is interrupted, e.g. when a light is powered-off, the Hue bridge sets the reachable state attribute to false. When the communication is restored (after the light is powered-on), the Hue bridge sets reachable to true. Note that it typically takes some (several seconds up to a minute?) time for the bridge to update reachable after a light is powered off or on.

Possible Solution - scenario 1 Most of the workarounds suggested in the Hue developer's forum involve a programme monitoring the light state for the unholy power-on default state, overwriting it with some preferred state. As homebridge-hue already monitors the light states, this functionality could be added to it.

When a light state changes to its power-on defaults, homebridge-hue could (re-)apply the previous light state from HomeKit to the Hue bridge, rather than propagating the Hue bridge change to HomeKit. When combined with issue #40, one could even change the desired power-on state while the light is powered off. This would work for unholy power-on defaults, that will never be set from any Hue app. Alternatively, homebridge-hue would only do it when reachable was false, but then it might miss a brief power outage, where the bridge would not change reachable.

The main limitation of this approach is the time lag between powering on the light and the Hue bridge re-establishing communication with the light and updating the light state.

Possible Solution - scenario 2 I actually have a "heartbeat" timer schedule and a number of rules on my Hue bridge, that boldly recall a "heartbeat" scene with the desired light states every 5 seconds. Not only does this overwrite any unholy power-on details within 5 seconds, it also undoes any pranks my kids are playing with the light colours. For any legit change, I suspend this heartbeat (1), do the change (2), update the heartbeat scene (3), and resume the heartbeat (4), all from Hue bridge rules. When we want to play with the light colours, I simply disable the heartbeat timer schedule (from HomeKit as homebridge-hue can expose schedules).

The main limitation of this scheme is that all legit changes have to go through Hue bridge rules (so they can suspend the heartbeat). This is not as bad as it sounds in my case, where these changes are triggered by Hue motions sensors, Hue dimmer switches, or through virtual switches (CLIPGenericFlag sensors). Each of these sensors fires a rule on the bridge for (2) anyways. I use a CLIPGenericFlag sensor to suspend the heartbeat, so I only need to add a single action for (1) to each of these rules setting this flag; (3) and (4) are triggered automatically a fixed time after this flag has been set. I had some challenges with the timing of updating the scene, especially for changes with longer transition times (typically when dimming the lights using the Hue dimmer switch), but it's working quite to my satisfaction.

homebridge-hue could implement a similar scheme: applying a heartbeat Hue bridge scene on every heartbeat. When any change would be made from HomeKit, homebridge-hue would suspend the heartbeat (1), propagate the change to the Hue bridge (2), update the Hue bridge heartbeat scene (3), and re-enable the heartbeat (4). Effectively, this would treat any change from HomeKit as legit, and overwrite any other change, including those from Hue bridge rules for motion sensors and dimmer switches. So these changes would need to be enacted through HomeKit rules instead. Alternatively, homebridge-hue would use a CLIPGenericFlag sensor to suspend the heartbeat, so bridge rules can make legit changes just as with the heartbeat implemented on the Hue bridge.

Help wanted I like to hear your thoughts on this. Would either scenario work in your setup? Do you interact with your lights exclusively from HomeKit? Using Hue switches and motion sensors? From Hue apps?

[pponce]

I like scenario 2 that avoids the potential long delay. My setup and usage: I have one tap switch and 7 or so dimmer philips dimmer switches. Personally i use a combination of apps. Homekit apple home app, Philips hue app, iconnecthue and iHue. Wife likes iHue. To set colors/temp though i use philips hue app or the other apps that work with philips hue api's directly (iconnecthue,ihue). I also use the ability to change scenes of the tap and dimmer switches.

[NorthernMan54]

After being woke up last night to a room with lights blazing, I vote for scenario 1. I'm only using HomeKit for my lights as well.

[ebaauw]

I assume that was due to a power outage? I'm afraid scenario 1 will be of limited use here.

Unless you have a no-break installation for the server running homebridge (and for the Hue bridge), it would take some time (minutes?) after power has been restored before homebridge is running again. Also homebridge-hue will have lost it's memory, including the previous light state. Even with dynamic platform accessories (issue #4), the accessory context would only be persisted when homebridge exits cleanly. I don't know if persisting the context on every change is feasible or supported by the homebridge api.

I'm not sure what light state the Hue bridge reports just after it has started, but I doubt it will show reachable as false. Depending on the number of lights connected to your bridge, it might take some minutes before the bridge will have polled the lights and report the power-on state.

Considering the above, I think the best way to cope with a power outage is simply to force all lights off on startup of the Hue bridge. On startup, the bridge initialises all CLIPGenericStatus sensors to 0 and sets the Daylight sensor when it connects to the meethue portal. This can be used to create a startup rule on the Hue bridge:

• Create a CLIPGenericStatus sensor Boot Time, say /sensors/2, to store whether the startup rule has been fired. I use a status rather than a flag, to prevent it being exposed as switch to HomeKit as it shouldn't be updated manually. If you expose the status to HomeKit, its Last Updated characteristic will show the time the bridge booted.
• Create the startup rule to set the status and turn off the lights when the Daylight sensor changes and the status hadn't been set.

  {
  "name": "Boot Time",
  "conditions": [
  {
    "address": "/sensors/1/state/lastupdated",
    "operator": "dx"
  },
  {
    "address": "/sensors/2/state/status",
    "operator": "eq",
    "value": "0"
  }
  ],
  "actions": [
  {
    "address": "/sensors/2/state",
    "method": "PUT",
    "body": { "status": 1 }
  },
  {
    "address": "/groups/0/action",
    "method": "PUT",
    "body": { "on": false }
  }
  ]
  }

[NorthernMan54]

Tks Erik for this, let me give it a try in my setup

On Jul 22, 2017, at 1:07 AM, Erik Baauw notifications@github.com wrote:

I assume that was due to a power outage? I'm afraid scenario 1 will be of limited use here.

Unless you have a no-break installation for the server running homebridge (and for the Hue bridge), it would take some time (minutes?) after power has been restored before homebridge is running again. Also homebridge-hue will have lost it's memory, including the previous light state. Even with dynamic platform accessories (issue #4 https://github.com/ebaauw/homebridge-hue/issues/4), the accessory context would only be persisted when homebridge exits cleanly. I don't know if persisting the context on every change is feasible or supported by the homebridge api.

I'm not sure what light state the Hue bridge reports just after it has started, but I doubt it will show reachable as false. Depending on the number of lights connected to your bridge, it might take some minutes before the bridge will have polled the lights and report the power-on state.

Considering the above, I think the best way to cope with a power outage is simply to force all lights off on startup of the Hue bridge. On startup, the bridge initialises all CLIPGenericStatus sensors to 0 and sets the Daylight sensor when it connects to the meethue portal. This can be used to create a startup rule on the Hue bridge:

Create a CLIPGenericStatus sensor Boot Time, say /sensors/2, to store whether the startup rule has been fired. I use a status rather than a flag, to prevent it being exposed as switch to HomeKit as it shouldn't be updated manually. If you expose the status to HomeKit, its Last Updated characteristic will show the time the bridge booted. Create the startup rule to set the status and turn off the lights when the Daylight sensor changes and the status hadn't been set. { "name": "Boot Time", "conditions": [ { "address": "/sensors/1/state/lastupdated", "operator": "dx" }, { "address": "/sensors/2/state/status", "operator": "eq", "value": "0" } ], "actions": [ { "address": "/sensors/2/state", "method": "PUT", "body": { "status": 1 } }, { "address": "/groups/0/action", "method": "PUT", "body": { "on": false } } ] } — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ebaauw/homebridge-hue/issues/41#issuecomment-317155887, or mute the thread https://github.com/notifications/unsubscribe-auth/AS5CmHWg1PLbIW36hKHpHSc7PKZghshKks5sQYN9gaJpZM4LdNvK.

[deanlyoung]

Scenario 2 seems like it is a) faster, and b) more flexible for different use cases. The only drawback, and it seems like a big one, is that it requires a lot more work to setup. At the very least, it would need some thorough step-by-step walkthroughs and documentation (btw, happy to help with those!).

On a personal note, I never realized until last night how big of an issue this could be. My lights came on after a split-second power outage (Scenario 1 wouldn’t have worked in this case), and Siri on my Apple Watch was still offline due to the router coming back online, so I had to fumble around for my phone and manually turn the lights off. (As a side note, I’m looking into a UPS device to keep my router/hue hub/etc. powered on in this scenario) Meanwhile, all of the lights still would have been power cycled, so Scenario 2 seems like it would be very helpful!

[ebaauw]

Scenario 2 seems like it is a) faster, and b) more flexible for different use cases. The only drawback, and it seems like a big one, is that it requires a lot more work to setup.

Indeed. Scenario 1 has a lousy user experience, and 2 is very, very cumbersome and has some technical challenges: The solution of storing a scene with every change might wear the lights, as they store scenes in non-volatile memory, which has a limited number of write cycles. Furthermore, I still haven't been able to get rid completely of the occasional "ghost" effect, when light states haven't been updated properly (due to longer transition times?) when storing the heartbeat scene, causing the lights to flash or blink when subsequently recalling the heartbeat scene. I've actually kept the heartbeat timer on my Hue bridge disabled lately...

Maybe the best alternative of reacting to a power failure is to force the lights off as soon as possible after power has been restored. I think the Hue bridge Boot Time rule described above will take about a minute (for the Hue bridge to startup, connect to the Internet and set the Daylight sensor based on your location). That might be good enough when being away from home, but way too long when sleeping. And it won't work with deCONZ... Scenario 2 would be quicker though, as the heartbeat kicks in before the Hue bridge connects to the Internet. After a short power failure you would like the lights to be restored to the previous state, but after a longer failure, you would probably want them turned off.

At the very least, it would need some thorough step-by-step walkthroughs and documentation (btw, happy to help with those!).

I almost finished sanitising my scripts of (re-)creating my Hue bridge rules. I'll post them to my ph.sh repository when done.

Siri on my Apple Watch was still offline due to the router coming back online, so I had to fumble around for my phone and manually turn the lights off.

One of the many advantages of deCONZ over the Hue bridge is, that is let's you to pair the Hue dimmer switch (or any other switch like the Ikea Trådfri remote) simultaneously with the deCONZ gateway and with a group. That means that the switch still works when the gateway is down, albeit at limited functionality (like when the dimmer is used standalone, without a bridge). Instead of reaching for your phone, you could reach for the switch.

I’m looking into a UPS device to keep my router/hue hub/etc. powered on in this scenario)

That sounds like a great idea, but keep in mind that the lights won't be powered anyways. The main benefit of course would be that homebridge continues running and could react when power comes back on. Still, homebridge would need a signal that power is back on, other than looking at light power-on or reachable state. In scenario 2, homebridge-hue or the Hue bridge would simply continue to recall the heartbeat scene every 5 seconds, so that could still be a shorter reaction time.

When running homebridge on a Raspberry Pi (the same running deCONZ anyways), the cheapest UPS-like solution (~ $20) would be a power bank that supports pass-through charging, in combination with some software that monitors the ethernet port to detect power failure/restoration, as described here. The easiest integration would be to restart homebridge when power is restored, and have homebridge-hue force-power-off the lights on startup (configurable though config.json, of course).

Bloody Amazon won't ship the mentioned power banks to the Netherlands, though. I did successfully run my Raspberry Pi with RaspBee board powered by a 2A Urban Revolt power bank (moving it next to my lights for touch linking), but that power bank doesn't support pass-through charging.

[lespatots]

I have a completely new solution, which might be combined with either above or with something else. I run my lighting on UPS. Led bulbs are so low in power that UPS is an easy option. However, this does leave some issues. Whilst it is easy to move a complete lighting circuit to UPS, this is not easy with table lamps etc which are just connected to a convenient outlet. It does not resolve the eventual battery depletion of the UPS but perhaps additional batteries would. There are some accessories such as ceiling fans which are best defaulted off?

[ebaauw]

Interesting approach. However, here in the Netherlands, lighting and power outlets typically share circuits. The outlets are wired from the central lighting point on the ceiling.

I finally got a pass-thru powerbank, which seems to be able to power my Raspberry running deCONZ and homebridge for at least 20 hours. Checking for the ethernet link status (with no UPS on the network switch) seems reliable enough to detect a power outage and subsequent restore.

I haven’t tried any automation yet, but I guess I would be keeping a CLIP flag or status with the current power situation (normal, failure, just restored), and start broadcasting off to group 0 as soon as power has been restored for a minute or so. I need to check if deCONZ actually issues a broadcast for group 0 - some lights and plugs don’t support groups. Alternatively, I could create a resourcelink (cf. the blacklist) with resources to turn off on power restore.

deCONZ does have a Daylight sensor now, and it can be used for Boot Time, just as the Daylight sensor on the Hue bridge. The rules above work, but I find I’m restarting deCONZ way more often than experiencing power outages. I’m actually using a flag now, instead of a status. By setting swversion to 0, and manufacturername to homebridge-hue, the flag will be exposed read-only to HomeKit.

[lespatots]

One advantage of involving a UPS might be that it is very good at detecting a power fail and there is a NUT plugin for Homebridge that can provide status data to Homekit. However, you need a good use case to buy a UPS and unRaid/Plex is mine!

On a side note I'm very interested in your experience of deCONZ as I decided that it was probably the best solution to integrate some Fibaro modules (shutter controls) as the latest Fibaro kit is Bluetooth and the range would possibly be an issue. The older kit is reasonably priced but requires a bridge from Z-Wave and Homebridge!

[ebaauw]

On a side note I'm very interested in your experience of deCONZ as I decided that it was probably the best solution to integrate some Fibaro modules (shutter controls) as the latest Fibaro kit is Bluetooth and the range would possibly be an issue. The older kit is reasonably priced but requires a bridge from Z-Wave and Homebridge!

I have no experience with Z-Wave. deCONZ speaks ZigBee, not Z-Wave. There's various ZigBee-based shutter controls, but we haven't integrated any of them in deCONZ yet.

In my experience, Bluetooth (even BTLE) isn't the right protocol for home automation, it's simply not responsive enough. I have some Elgato devices (Eve Energy, Eve Door) that natively support HomeKit over BTLE. I haven't run into any range issues (they're close enough to my AppleTV), but it takes at least two seconds before the door sensor reports that the door has been opened. The ZigBee-based Xiaomi door sensor works immediately (and is five times less expensive). Similarly, when instructed to turn on, the Eve Energy takes 2 to 7 seconds to react. With the inclusion of fakegato-history in homebridge-hue, ZigBee based plugs are almost as functionally rich, and react immediately.

[ebaauw]

I haven’t tried any automation yet, but I guess I would be keeping a CLIP flag or status with the current power situation (normal, failure, just restored), and start broadcasting off to group 0 as soon as power has been restored for a minute or so

Using the Raspberry Pi UPS solution, I created an initial automation to power off the lights automatically after power has been restored.

• Make sure you've installed homebridge-hue (or homebridge-hue-utils) and that you've created a username for ph;
• After installing upsd, add the following line to /usr/lib/upsd/power_back. The script runs as root, hence the su - pi:

  su - pi -c "ph put /sensors/100/state '{\"flag\": true}'"

• Create a series of rules, using the script from ph.sh:

  $ . ph_rules.sh
  $ ph_rules_power 100

• Make sure, your Raspberry's WiFi is enabled, and run journalctl -afn 1000 -u upsd to monitor the upsd daemon and unplug and reconnect the ethernet cable:

  $ journalctl -afn 1000 -u upsd
  May 09 12:13:11 pi2 upsd[417]: There is a power outage right now.
  May 09 12:13:11 pi2 upsd[417]: The estimated current battery charge level is 100 %.
  May 09 12:13:11 pi2 upsd[417]: There are 18:00:00 hours remaining before the battery will be empty.
  May 09 12:13:11 pi2 upsd[417]: If the battery should actually run empty, appropiate action will be taken.
  May 09 12:13:17 pi2 upsd[417]: The power outage is over now. It could be successfully bridged over by using the battery.
  May 09 12:13:17 pi2 upsd[417]: The outage lasted for 00:00:06 hours.
  May 09 12:13:17 pi2 upsd[417]: The estimated battery charge level decreased from 100 to 99 % during the outage.
  May 09 12:13:17 pi2 upsd[417]: The system could have run for 17:59:53 more hours before the battery would have been empty.
  May 09 12:13:17 pi2 su[30340]: Successful su for pi by root
  May 09 12:13:17 pi2 su[30340]: + ??? root:pi
  May 09 12:13:17 pi2 su[30340]: pam_unix(su:session): session opened for user pi by (uid=0)

• Check that the rules have fired:

  $ ph get /rules
  "241": {
  "actions": [
    {
      "address": "/groups/0/action",
      "body": {
        "on": false
      },
      "method": "PUT"
    }
  ],
  "conditions": [
    {
      "address": "/sensors/100/state/flag",
      "operator": "eq",
      "value": "true"
    },
    {
      "address": "/sensors/100/state/lastupdated",
      "operator": "dx"
    }
  ],
  "created": "2018-05-09T10:09:47",
  "lasttriggered": "2018-05-09T12:13:19",
  "name": "Power 1/5",
  "periodic": 0,
  "status": "enabled",
  "timestriggered": 2
  },
  "242": {
  "actions": [
    {
      "address": "/groups/0/action",
      "body": {
        "on": false
      },
      "method": "PUT"
    }
  ],
  "conditions": [
    {
      "address": "/sensors/100/state/flag",
      "operator": "eq",
      "value": "true"
    },
    {
      "address": "/sensors/100/state/lastupdated",
      "operator": "ddx",
      "value": "PT00:00:02"
    }
  ],
  "created": "2018-05-09T10:09:47",
  "lasttriggered": "2018-05-09T12:13:21",
  "name": "Power 2/5",
  "periodic": 0,
  "status": "enabled",
  "timestriggered": 3
  },
  "243": {
  "actions": [
    {
      "address": "/groups/0/action",
      "body": {
        "on": false
      },
      "method": "PUT"
    }
  ],
  "conditions": [
    {
      "address": "/sensors/100/state/flag",
      "operator": "eq",
      "value": "true"
    },
    {
      "address": "/sensors/100/state/lastupdated",
      "operator": "ddx",
      "value": "PT00:00:04"
    }
  ],
  "created": "2018-05-09T10:09:47",
  "lasttriggered": "2018-05-09T12:13:23",
  "name": "Power 3/5",
  "periodic": 0,
  "status": "enabled",
  "timestriggered": 3
  },
  "244": {
  "actions": [
    {
      "address": "/groups/0/action",
      "body": {
        "on": false
      },
      "method": "PUT"
    }
  ],
  "conditions": [
    {
      "address": "/sensors/100/state/flag",
      "operator": "eq",
      "value": "true"
    },
    {
      "address": "/sensors/100/state/lastupdated",
      "operator": "ddx",
      "value": "PT00:00:06"
    }
  ],
  "created": "2018-05-09T10:09:48",
  "lasttriggered": "2018-05-09T12:13:25",
  "name": "Power 4/5",
  "periodic": 0,
  "status": "enabled",
  "timestriggered": 3
  },
  "245": {
  "actions": [
    {
      "address": "/groups/0/action",
      "body": {
        "on": false
      },
      "method": "PUT"
    },
    {
      "address": "/sensors/100/state",
      "body": {
        "flag": false
      },
      "method": "PUT"
    }
  ],
  "conditions": [
    {
      "address": "/sensors/100/state/flag",
      "operator": "eq",
      "value": "true"
    },
    {
      "address": "/sensors/100/state/lastupdated",
      "operator": "ddx",
      "value": "PT00:00:08"
    }
  ],
  "created": "2018-05-09T10:09:48",
  "lasttriggered": "2018-05-09T12:13:27",
  "name": "Power 5/5",
  "periodic": 0,
  "status": "enabled",
  "timestriggered": 3
  }

• When testing with the network cable, you can actually observe from HomeKit that the flag turns on for some 10 seconds and then turns off again. Of course, on a real power outage, your WiFi will fail as well.

[lespatots]

My Bad, too many things in my life. As you point out Z-Wave is for Fibaro and Zigbee is of course Hue. On the Zigbee front I search for reasonably priced wall switches and an interface (waiting for the new Philips partner products and an advance in the number of supported switches) whilst Z-Wave does seem to cover a lot of other stuff including the old Fibaro range. Totally agree about BLE, just tried a Koogeek door/window sensor and immediatley realised I need a Homehub to interface it!

Had two power outages the other night and providing the network infrastructure is on UPS a Siri 'all light off' works well. I have three Hue bridges operating which is a management problem. How would that scenario impact your solutions?

[flavio20002]

Hi, what about that? https://www.reddit.com/r/Hue/comments/9btzb6/hue_announces_poweron_behavior_hue_lights_will/

[ebaauw]

That's good news! Power-on to the previous settings, apparently through a firmware upgrade of the bulbs. I'll need to move all my Hue lights from deCONZ to the Hue bridge for the upgrade and then back again...

Nothing yet on the Hue developers forum, though. From the video (https://youtu.be/e6fkmQHZWpE) it looks like there will be a setting to choose between the current default power on behaviour or power on to the previous settings. We'll have to change deCONZ to provide this setting through their API as well, and maybe change homebridge-hue to expose this setting to HomeKit.

[flavio20002]

Philips say: Hey David, the new ‘power on’ option will be rolled out towards the end of the year. Since we will enable this feature on all the official Hue lamps that we have ever released, the roll out procedure takes some time. We will keep you posted when the feature is available.

[ebaauw]

The new firmware for the Hue bulbs is available here in the Netherlands. See https://github.com/dresden-elektronik/deconz-rest-plugin/issues/1014.

As far as I can tell, at ZigBee level you can specify the power-on settings for state.on, state.bri, state.ct, and state.xy, either to a predefined value, or to the previous value (from before the light was powered off). Although I haven't seen any documentation, the power-on settings for on and bri seem standard: my IKEA Trådfri bulbs and ubisys D1 dimmer have implemented them as well. The current (previous) firmware for the Hue lights already supports the power-on setting for ct.

I'm not sure if I need to expose these through homebridge-hue - you probably want to set them only once (of course, after playing with the settings initially). I think it might be sufficient to document how to do this using ph, once deCONZ supports this. This already works for the Hue bridge.