Question: Set colors/brightness without turning on light?

[imbaimbaimba]

Hi!

So just a quick question. Is it possible to set values for a bulb while it is off(soft turned off)?

My problem with my current system is to change the color and brightness that a bulb wants to turn on with next, it needs to turn it on. I cannot seem to get this working properly in my Homey setup.

To me, it seems necessary to have that functionality for scenes to work properly, while also using switches/motion sensors. So that for example at a given time of day, the bulbs are set to a warm tone and low brightness, and doesn't first turn on at full brightness white and then dim down and change color.

[ebaauw]

Is it possible to set values for a bulb while it is off(soft turned off)?

No, as per the Zigbee spec, the bulb ignores commands to change brightness or colour while it’s off.

[imbaimbaimba]

Hmm ok. What does the Hue app(using bridge) do differently? When I used that, I had automations for lights to turn of with different colors/tones/brightness throughout the day. When they triggered, the bulb would turn on beginning with the exact right parameters without any flickering around. Everything else I tried, it seems impossible to make automations work smoothly, because I have to: When .... Then: -Turn on given Hue bulbs -Set color/tone -Set brightness

This means that if I trigger a light that is meant to be a discrete orangeish nightlight, and the last state it had was all blue, it goes flashing and flickering for a sec before being what was intended.

I got rid of my Hue bridge to ensure I could have my bulbs on the same network as all my Xiaomi etc sensors so the coverage was good. I tried Zigbee2mqtt, Deconz and Homey, but they all seem to act the same way.

Therefore I feel like it might be the only solution to revert to the Hue bridge to get the smooth operation.

Am I missing something?

[ebaauw]

What does the Hue app(using bridge) do differently?

The Hue app probably uses scenes. You would have to check the rules it created on the Hue bridge to be sure.

I tried Zigbee2mqtt, Deconz and Homey, but they all seem to act the same way.

The issue is with ZigBee: it takes two separate commands to turn on the light (at a certain brightness), and to set its colour. No gateway can change that. Recalling a scene only takes a single command for the light to restore its previously stored state, incl. on, brightness, and colour.

[imbaimbaimba]

Makes sense. I did in fact use scenes when I had Hue bridge. So I would define the different scenes for different times of day and have automation trigger them.

So since a zigbee bulb can only accept one command at a time, does that mean that they have some memory since it could work like it did on the Hue bridge? I assume that is the only way that a single command could make the light turn on at a specific setting, if it referred to a set of settings that is saved on the bulb? If that is the case, couldn’t other automation systems also use that?

[ebaauw]

So since a zigbee bulb can only accept one command at a time, does that mean that they have some memory since it could work like it did on the Hue bridge?

Each ZigBee device has persistent memory (retained when the device is powered off) and non-persistent memory (lost when the device is powered off). This is needed for the device to participate in a ZigBee network, e.g. the ZigBee network key is stored in persistent memory, the neighbour tables are stored in non-persistent memory.

I assume that is the only way that a single command could make the light turn on at a specific setting, if it referred to a set of settings that is saved on the bulb?

Lights generally have a groups table, and a scenes table in persistent memory. These are handled by the Groups and the Scenes server clusters (the blue ones in the deCONZ GUI). The groups table lists the (four-byte) group ids that the light has "subscribed" to, i.e. to which groupcast messages it will respond. The scenes table contains the state to restore when the scene is recalled. Scenes are identified by a group id and a (two-byte) scene id. The sizes of these tables (and thus the number of groups or scenes supported) differ per light type. This should be a consideration when buying lights, but none of the manufacturers seem to report these.

The REST API (as well as the Hue API) is confusing, in that it shows groups and scenes containing lights and light states, where it's actually the other way round: the light contains group and scene memberships.

If that is the case, couldn’t other automation systems also use that?

Any system that interfaces with the REST API can, in principle, create and recall scenes. Particularly, this can be done through rules on the deCONZ gateway or Hue bridge. Home automation systems typically create their own groups and scenes, and don't understand nor support ZigBee groups and scenes. homebridge-hue can expose ZigBee groups to HomeKit (as Lightbulb or Outlet), but doesn't expose ZigBee scenes. I typically use a CLIPGenericFlag as virtual switch (exposed as Switch to HomeKit) to fire a rule that recalls the ZigBee scene.

Generally, it's a bad idea to use the groups and scenes of another home automation system in conjunction with ZigBee lights. While I've taken every measure to make HomeKit scenes work with homebridge-hue, using ZigBee scenes is far more efficient and provides a better user experience (all lights change at the same time), see https://github.com/ebaauw/homebridge-hue/issues/15.

[imbaimbaimba]

Thank you very much for taking your time to answer and explain all this.

I think I understood most of it, but I am a bit uncertain about how I can make use of what I now know. Also I didn’t quite find much about the CLIPGenericFlag.

So currently I am using an Athom Homey, where all my Hue bulbs etc are connected directly to Homey by zigbee. I asked the developer for the hue zigbee app, and they will not currently implement features to help with what we talked about here. Also I generally feel like the reliability of zigbee on Homey is not great, and having a multitude of other problems as I need stuff connected to it by all sorts of workarounds.

Therefore I am considering going back to HomeKit which was always trusty before I first went with Home Assistant(a bit too advanced for my taste) and then to Homey. So I have much stuff that is already HomeKit compatible, but would like to obviously get the rest of my stuff in there or replace it(will look at the various homebridge plugins).

So for Hue, as I see it, I have three options. 1. Using a Hue bridge and the built in HomeKit integration

2. Using a Hue bridge and your homebridge integration
3. Using a Deconz and your homebridge plugin (I have a ConBee stick)

Now I can’t really see what the difference and pros/cons are on option 1/2, can you briefly explain that, or link me to a resource, as I couldn’t find one?

Option 3, I assume, is the only options that would allow me to also use my Xiaomi switches/sensors etc that are battery driven on the same zigbee mesh. With my current setup, I need that, due to the distances and my only mains powered devices being the Hue lighting. I heard a few times that mixing these things on the same mesh can be problematic. Do you know anything about that?

Either way, to be able to save a few sets of parameters(scenes right?) for brightness/color/tone to my lights persistent memory, that can then be called for the bulbs to fire up straight into these scenes, what setup would you recommend me? And how should I handle the light scenes?

In general, I have had a bit of a hard time finding opinions/insight as to why some people use homebridge instead of say Home Assistant, Homey etc.? If you feel like telling a bit about that, I would be glad to hear that too :)

Sorry for taking so much of your time. Your insight is highly appreciated.

[ebaauw]

In general, I have had a bit of a hard time finding opinions/insight as to why some people use homebridge instead of say Home Assistant, Homey etc.? If you feel like telling a bit about that, I would be glad to hear that too :)

I run most of my home automation on deCONZ (using deCONZ rules). I use scripts to create and maintain these rules, see https://github.com/ebaauw/ph.sh. The syntax is horrible, but the functionally is rich enough for my purposes, using CLIPGenericFlag and CLIPGenericStatus sensors as variables. I'm currently on 102 ZigBee devices, half routers (mostly Hue lights, some innr, 1 GLEDOPTO, 2 Heiman, 2 iCasa, 1 OSRAM), half end devices (20 Hue, 20 Xioami, 8 Eurotronic, 2 IKEA).

For manual interaction/overrides, I use HomeKit and/or Siri, because I like the user experience. I use HomeKit automations for integrating my non-ZigBee devices (Sonos speakers, smart electricity meter, OpenTherm Gateway, and virtual weather station based on OpenWeatherMap), and to detect who's at home.

Between deCONZ and HomeKit, I have no need for another home automation system, so I never tried any. Less is more.

I run deCONZ, two homebridge instances (one for homebridge-hue, one for my other plugins), and the OpenTherm Monitor natively on a single Raspberry Pi 3B+, using a passthrough USB power bank as UPS. It runs ridiculously stable, at a ridiculously low the price tag, more stable even than my mac mini server. I keep the system as simple as possible; I don't mess with Docker. Less is more.
I decommissioned my Hue bridge. I only keep it for testing and for upgrading the firmware of my Hue devices.

So for Hue, as I see it, I have three options.

These options are not exclusive - you could combine them.

1. Using a Hue bridge and the built in HomeKit integration

Has better responsiveness than 2 (the Hue bridge pushes changes to HomeKit, but not to API clients, like homebridge-hue), and is potentially more stable (has fewer components) than 2 or 3. Also, far more suited for non-technical people. Provides unique functionality (Hue entertainment).

2. Using a Hue bridge and your homebridge integration

Provides more functionality than 1 (non-Hue lights, Eve history). You could expose the Hue lights by the Hue bridge, and the sensors and non-Hue lights by homebridge-hue.

3. Using a Deconz and your homebridge plugin (I have a ConBee stick)

Provides better functionality (non-Hue devices) and scalability (more lights, sensors, rules) than 1 and 2. Provides better responsiveness (push notifications) than 2.

Since you already have a ConBee, I'd go for 3, unless you want/need Hue Entertainment. In that case, pair the lights used for that to the Hue bridge, exposing them natively to HomeKit, and the rest to deCONZ.

Option 3, I assume, is the only options that would allow me to also use my Xiaomi switches/sensors etc that are battery driven on the same zigbee mesh. With my current setup, I need that, due to the distances and my only mains powered devices being the Hue lighting.

Yes, you need (mains powered) ZigBee routers (lights) to bridge the distance between the gateway and the (battery powered) end devices, and to connect more end-devices than the gateway can handle directly. I think the limit is 16, due to memory limitations in the ConBee. The Hue lights are actually the most stable routers in combination with deCONZ.

I heard a few times that mixing these things on the same mesh can be problematic. Do you know anything about that?

Mixing different routers (lights) can be problematic, especially IKEA lights. Since you only have Hue lights, that shouldn't lead to any problems. My Xioami end devices are really reliable. I've only seen issues when the battery is low. They stick to their parent router like glue, so make sure you pair them at their target location and never power off any of your lights.

And how should I handle the light scenes?

Best in deCONZ (or Hue bridge) rules. To activate them from HomeKit automations, use a rule triggered by a CLIPGenericFlag or CLIPGenericStatus exposed to HomeKit, and only set the flag or status from HomeKit.

[imbaimbaimba]

Thank you so much for all the valuable feedback. I will go with your recommendation.

I assume the whole CLIPGenericFlag and CLIPGenericStatus being exposed to HomeKit is something that will make more sense as soon after I get HomeBridge up and running? I couldn't really make much of it for now, but now I at least know what to look for.

Have you messed with any Z-wave also during your smart home ventures? I got a couple of Z-wave devices I need running as well - especially a Qubino 0-10v dimmer that is controlling our house ventilation system. Everything else than that seems to be available as wifi/zigbee versions, but just not that.

I got an Aeotec Z-stick that I used for Home Assistant to have my z-wave devices on there, but unfortunately, exactly that was one of the reasons my Home Assistant adventure ended, because the open z-wave platform on there was extremely unreliable. I would actually have to go and manually delete config files daily for my HA setup to be alive. It seems this stick is not supported by Z-way which is the homebridge integration I can find for z-wave. :(

[ebaauw]

No, I don’t have any experience with Z-wave. It might not be a good idea to have both a ZigBee and a Z-wave stick connected to the same server, due to radio interference. At least, use USB extension cables to keep some distance between the sticks, and make sure the Z-wave software doesn’t try and access the ConBee’s serial port.

I don’t know if they would meet the specs of your ventilation system, but there’s plenty of 12-24V DC ZigBee dimmers for LED strips.

[imbaimbaimba]

Got it all running now.

I tested Homebridge in parallel with Home Assistant(as I still have a server running that, which creates a dashboard from MQTT from my Homey).

I added a single bulb to deconz first, added a couple of very different scenes and tested it out from deconz, works great and with the intended behavior as per what we talked about.

The scenes are automatically added to Home Assistant, and automatically forwarded to HomeKit in the form of switches. These work exactly like in deconz which is great.

I haven't been able to get the scenes from deconz to show through homebridge though, which is where you refered to:

Best in deCONZ (or Hue bridge) rules. To activate them from HomeKit automations, use a rule triggered by a CLIPGenericFlag or CLIPGenericStatus exposed to HomeKit, and only set the flag or status from HomeKit.

Can you explain or link me to somewhere, where I can learn what that is all about? I couldn't figure it out unfortunately.

For now, it actually seems like Home Assistant is easier to achieve the same result with than Homebridge. However, I am still unsure which is more responsive, stable and has the best functionality.Maybe you are aware of some pros/cons?

[ebaauw]

See #158.