Switches only affect Non-Zigbee HomeKit Devices

Happy New Year!

I'm experiencing an issue that I find difficult to understand, let alone explain. Please bear with me:

I have a sizeable HomeKit installation with combination of platforms such as Homebridge deCONZ (Zigbee), IKEA Tradfri hub (Zigbee), Homebridge Tahoma, Netatmo devices, node-RED (node-red-contrib-homekit-bridged) and more. An Apple TV acts as the HomeKit hub. Most lights and switches are joined to Homebridge deCONZ, with a minority being pure IKEA Tradfri. deCONZ is hosted on a Raspberry Pi 3b, which also hosts HomeBridge and HomeBridge deCONZ (127.0.0.1).

Scenes and automations are configured in HomeKit. There are no scenes or other logic configured in deCONZ. Switches and lights are logically placed in deCONZ Groups ("rooms") as per their physical layout, and this layout is largely mirrored in HomeKit Rooms.

Problem: For some HomeBridge deCONZ Zigbee input devices (switches and motion sensors) that are configured through HomeKit to perform actions on multiple devices across both Homebridge deCONZ and non-deCONZ platforms, only the non-deCONZ devices will physically be affected by the actions.

Example 1a: The IKEA TRADFRI On/Off switch "Living Room Upstairs Switch" placed at the top of the stairs is joined to deCONZ. Button 2 Single Press is assigned to trigger a scene (named "Going Down") that turns off all lights on that floor. When the button is pushed, the scene is triggered in HomeKit and all devices in HomeKit are shown to have entered the desired state, but the action actually ends up affecting only the devices that are not part of the deCONZ network. The physical result is that the IKEA Tradfri hub lights and sockets are turned off, the Apple TV is turned off, but the deCONZ lights and sockets remain ON.

Example 1b: Activating the scene on my watch or on my phone, or by voice ("Siri, Going Down") results in a 100% desired state, and all lights and devices are physically switched off.

Example 2a: The IKEA TRADFRI On/Off switch "Second Kitchen Light Switch" placed physically in the Second Kitchen is joined to deCONZ. Button 2 Single Press is assigned to directly trigger an action on 3 accessories, turning off Second Kitchen Bench Lights (IKEA), Second Kitchen Drawer Lights (IKEA) and Second Kitchen Mezzanine (deCONZ). When the button is pushed, only the Second Kitchen Bench Lights (IKEA) and Second Kitchen Drawer Lights (IKEA) are physically switched off.

Example 2b: Issuing "Siri, turn off lights in Second Kitchen" results in all of Second Kitchen Bench Lights (IKEA), Second Kitchen Drawer Lights (IKEA) and Second Kitchen Mezzanine (deCONZ) being physically turned off.

I have tried to experiment with parallelRequests, waitTimePut and waitTimeUpdate to no success. I'm at my wits end on where to go from here. Any ideas?

The entirety of config.json is:

{
    "bridge": {
        "name": "[…]",
        "username": "[…]",
        "port": […],
        "pin": "[…]"
    }

    "platforms": [
        {
                "platform": "deCONZ",
                "hosts": ["127.0.0.1"],
                "_bridge": {
                        "username": "[…]",
                        "port": […]
                }
        }
    ]
}

Environment:

Homebridge v1.7.0 (HAP v0.11.1)
homebridge-deconz 1.0.9
Deconz: 2.25.0 (latest)
Adapter: ConBee II, firmware: 26780700 (latest)
node v20.10.0

You need to determine whether your issues are caused by HomeKit, by Homebridge deCONZ, or by deCONZ. As your Zigbee devices work when recalling a HomeKit scene directly, or through Siri, I doubt the issue is with Homebridge deCONZ. I would need to see a full debug log file to be certain.

I would look for opportunities to simplify your setup (making it more robust). I recommend to use deCONZ rules to control Zigbee devices from Zigbee controllers, and only use HomeKit automations where non-Zigbee devices are involved. Also, I would pair all IKEA devices to deCONZ and ditch the Trådfri hub.

You should see a notification by Homebridge decONZ in the Homebridge log of the button event that Homebridge deCONZ sends to HomeKit, and of the characteristic changes it receives from HomeKit when the HomeKit scene is recalled. Make sure LogLevel is 1 or higher to see these info messages. If they don’t appear, the issue is in HomeKit, see 1) below.

With LogLevel 2, Homebridge deCONZ logs the API calls it makes to deCONZ. You should see log messages for the PUT requests to the lights resources corresponding to the accessories in the HomeKit scene. There should be a corresponding OK response by deCONZ that it has accepted the request. If you see error responses, the issue is with deCONZ. If you see OK responses, but the devices don't change state physically, the issue is with deCONZ, see 2) below.

Ad 1. HomeKit automations run on your home hub (I.e. your AppleTV) whereas recalling a HomeKit scene, or invoking Siri. runs on your iPhone, iPad, or whatever. Unfortunately there is no way to check whether the HomeKit automations and scenes where synced correctly to your home hub; best disable them, wait for a couple of minutes, and re-enable them to trigger a sync.

Ad 2. There might be errors in the deCONZ log, with the right logging options. See https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Zigbee-Error-Codes-in-the-Log.

Thank you for taking the time, @ebaauw . Turns out the rabbit hole was rather deep this evening.

It seems like the problem is (at least tangentially) related to the desire of deCONZ to interpret the button press as a command to all devices, at least in the Phoscon UI. What I observed was the following: Shutting down Homebridge deCONZ, I had a look at the Phoscon UI, and noticed that pressing On/Off on the IKEA TRADFRI On/Off switch resulted in ALL lights being turned On/Off in the UI, but not physically!

This is an example of what homebridge-deconz logs simultaneously when I push a button on the remote, with no actions configured for the buttons (abbreviated for readability):

[homebridge-deconz] Outside Back Door: On: set to false (from true)
[homebridge-deconz] Hallway Stair Lights: On: set to false (from true)
[homebridge-deconz] Living Room High Lamp: On: set to false (from true)
[homebridge-deconz] Office Ceiling Lights: On: set to false (from true)
[homebridge-deconz] Second Kitchen Mezzanine: On: set to false (from true)
[homebridge-deconz] Kitchen Dining Table: On: set to false (from true)
[homebridge-deconz] Kitchen Bench Ceiling Lights: On: set to false (from true)
[homebridge-deconz] Living Room Upstairs Switch Off: Programmable Switch Event: Single Press

Note that the switch at this time had no actions configured to any button in HomeKit, had been freshly re-paired, and was never configured to control some of these lights, such as the Outside Back Door.

This seems like the same, or at least similar to, the issue described in https://github.com/dresden-elektronik/deconz-rest-plugin/issues/846 and https://github.com/dresden-elektronik/deconz-rest-plugin/issues/853 . Perhaps foolishly, I tried using the deCONZ GUI to unbind the remote from 0xFFF0, but the remote reported that the binding was unknown. I also tried adding the button to a specific group in Phoscon, but this didn't change the behaviour.

This lead me to https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/What-you-can-do-when-a-switch-turns-(almost)-all-devices-on-off by @SwoopX . Here, the guidance is to edit zll.db by issuing update groups set state="deleted" where gid="0xFFF0"; and then re-pairing the switch. This works and solves the problem of the switch turning On/Off all lights in the deCONZ UI. I thought I had reached salvation! However… starting up Homebridge deCONZ, the platform was unhappy that /groups/0 returned a 404 and refused to load. Reverting the change to zll.db made Homebridge deCONZ load again. I thought that as long as the switch was re-paired while 0xFFF0 was disabled, I was home free – but alas, now the switch controls all lights again.

To top it off, Homebridge deCONZ marked all of my accessories as stale and re-discovered them in the process, so now all of my devices are back in the default room, all automations and scenes are lost, its 23:59 on a work night and I hope I will have warm water tomorrow as my electric water boiler is on a deCONZ Zigbee relay…

I will restore basic functionality to get the house through the night, and then I will follow up tomorrow.

Any help from hereon out is greatly appreciated.

/groups/0 is supposed to include all lights. Typically, it is mapped to group ID 0xFFF0. It should be managed by deCONZ automatically, and should not be removed.

Some controllers will send a broadcast when no lights nor groups are bound to its client clusters. So when the joining process is incomplete, or when the device support is missing the bindings, these controllers work on all lights. To change this, you can manually bind the client clusters to a (real) group, or to the coordinator device. Depending on the controller, you might need to bind all relevant client (grey) clusters: On/Off, Level Control, Color Control, Scenes.

Some (IKEA) controllers don’t (no longer with recent firmware) honour group bindings. For these, each light needs to be bound individually. Make sure to include a binding to the coordinator as well, so deCONZ can eavesdrop the commands to control the lights and generate the appropriate button events.

From your log: deCONZ updates the state of the lights, which, by design, Homebridge deCONZ forwards to HomeKit. Only after that, deCONZ reports the button event, which, again Homebridge deCONZ forwards to HomeKit as a Single Press. By design, on recalling a scene, HomeKit only updates the characteristics that need to be changed, so the light states that were just incorrectly updated aren’t included.

Mightily appreciated. I understand that this behaviour on the part of Homebridge deCONZ, and it makes sense as HomeKit should not try to change the state of devices that are already in the desired state.

I brought back /groups/0, and then I spent a while troubleshooting as per your suggestions. Binding the controller lights in deCONZ did nothing to change the behaviour, and I tried a multitude of lights, (real) groups and/or the coordinator.

I racked my brain as to how this might be solves, until I thought about checking for new software for the switches. The switches in question were all running SW Build ID 2.2.008 (Vendor 0x117c, Image 0x11c5, Version 0x22008631). I hadn't thought about this for ages, so I updated them to 24.4.6 (Version 0x24040006) – and Lo and Behold, the switches now work exactly as I want them to, without any specific bindings to anything at all! The scenes are fast and responsive, and nothing lags behind any longer.

So, quite a journey full of learning and discoveries, and a few wrong turns along the way, with a rather simple and obvious solution that escaped me from the beginning. Thanks for your help @ebaauw !

ebaauw / homebridge-deconz

Switches only affect Non-Zigbee HomeKit Devices #203