[Problem]: ZGP / greenPower messages are not relayed by HUE routers

[florianbrede-ayet]

What happened?

Zigbee Green Power (ZGP) messages are only broadcasted by the coordinator, so they only work in close proximity to it.

What did you expect to happen?

I have several HUE routers in range of the ZGP switches (EnOcean PTM 215Z). I expect those device to also pick up and relay ZGP messages.

How to reproduce it (minimal and precise)

1.) Create a Zigbee network with z2m and a conbee II coordinator on channel 11 2.) Configure several ZGP switches for channel 11 3.) Add HUE routers to the network 4.) Position the ZGP switches far from the coordinator but in close range to HUE routers

Zigbee2MQTT version

1.22.2-dev commit: e6b0414 (older versions had the same problem)

Adapter firmware version

0x26700700

Adapter

ConBee2

Debug log

I have no debug logs for this issue, but sniffed the zigbee traffic with a second conbee and wireshark.

In this example, the switch was in range of the coordinator. If it isn't, none of the 125 byte broadcast messages will appear:

The settings for one of the HUE routers look like this:

"0x001788010b56d3e7": {
            "date_code": "20211119",
            "definition": {
                "description": "Hue smart plug",
                "exposes": [
                    {
                        "features": [
                            {
                                "access": 7,
                                "description": "On/off state of the switch",
                                "name": "state",
                                "property": "state",
                                "type": "binary",
                                "value_off": "OFF",
                                "value_on": "ON",
                                "value_toggle": "TOGGLE"
                            }
                        ],
                        "type": "switch"
                    },
                    {
                        "access": 1,
                        "description": "Link quality (signal strength)",
                        "name": "linkquality",
                        "property": "linkquality",
                        "type": "numeric",
                        "unit": "lqi",
                        "value_max": 255,
                        "value_min": 0
                    }
                ],
                "model": "929003050601",
                "options": [],
                "supports_ota": true,
                "vendor": "Philips"
            },
            "endpoints": {
                "11": {
                    "bindings": [
                        {
                            "cluster": "genOnOff",
                            "target": {
                                "endpoint": 1,
                                "ieee_address": "0x00212effff07bad5",
                                "type": "endpoint"
                            }
                        }
                    ],
                    "clusters": {
                        "input": [
                            "genBasic",
                            "genIdentify",
                            "genGroups",
                            "genScenes",
                            "genOnOff",
                            "genLevelCtrl",
                            "touchlink",
                            "manuSpecificSamsungAccelerometer"
                        ],
                        "output": [
                            "genOta"
                        ]
                    },
                    "configured_reportings": [
                        {
                            "attribute": "onOff",
                            "cluster": "genOnOff",
                            "maximum_report_interval": 3600,
                            "minimum_report_interval": 0,
                            "reportable_change": 0
                        }
                    ],
                    "scenes": []
                },
                "242": {
                    "bindings": [],
                    "clusters": {
                        "input": [],
                        "output": [
                            "greenPower"
                        ]
                    },
                    "configured_reportings": [],
                    "scenes": []
                }
            },
            "friendly_name": "0x001788010b56d3e7",
            "ieee_address": "0x001788010b56d3e7",
            "interview_completed": true,
            "interviewing": false,
            "manufacturer": "Signify Netherlands B.V.",
            "model_id": "LOM007",
            "network_address": 38420,
            "power_source": "Mains (single phase)",
            "software_build_id": "1.93.6",
            "supported": true,
            "type": "Router"
        }

[Koenkk]

I believe that only the router that you've paired the device via will relay these messages. If you move it out of range you have to repair the green power device.

[florianbrede-ayet]

Thanks for your reply. That's something I was thinking as well, so I tried to Recommission the devices several times. The zigbee2mqtt site (https://www.zigbee2mqtt.io/devices/PTM_215Z.html) states for the PTM 215Z that it's possible to reset the switch, however that's only true for the PTM 216Z. The datasheet for PTM 215Z does not mention any reset functionality and commissioning is performed on channel selection.

What I did multiple times is: 1.) Force remove the device from zigbee2mqtt 2.) Select a different channel on the switch 3.) Switch back to channel 11 on the switch, which sends a Commission frame. 4.) Commissioning only completes (and it only shows up in z2m) if the coordinator is directly reachable. Afterwards the switch works the same as before - only in range of the coordinator.

Commission Frame:

Zigbee2Mqtt Log:

Debug Received Zigbee message from '0x000000000174c288', type 'commandCommisioningNotification', cluster 'greenPower', data '{"commandFrame":{"deviceID":2,"extendedOptions":242,"keyMic":1393009487,"options":197,"outgoingCounter":1095,"securityKey":{"data":[165,96,106,116,173,44,26,53,144,76,173,220,165,94,84,54],"type":"Buffer"}},"commandID":224,"frameCounter":1095,"options":62149,"payloadSize":46,"srcID":24429192}' from endpoint 242 with groupID 2948

[Koenkk]

Could you provide me a sniff of:

• permitting joining
• send a commission frame when having the switch close to a hue device

[florianbrede-ayet]

I've created 2 sniffs. I have a lot of devices in the network (and don't know which packets might be relevant), but the "Commissioning" frames are easily identifiable. My network key should still be the default one (1, 3, 5, 7, 9, 11, 13, 15, 0, 2, 4, 6, 8, 10, 12, 13). There are always a number of devices in range, but the closest hue device that could route ZGP is a hue power socket next to the switch on my desk.

Commissioning switch (0x000000000174c288) with hue power socket (0x001788010b56d3e7) in range, coordinator out of range. Didn't succeed as expected.

1. Permit join
2. Commissioning sequence by settings channel (emits: "Commissioning")
3. Confirm channel with A1 + B0 (emits: "Short press 2 of 2")

commissioning_hue_range.pcapng.txt

Commissioning switch (0x000000000174c288) with hue power socket (0x001788010b56d3e7) and coordinator in range. This worked, the device appeared in zigbee2mqtt and the switch works in range of the coordinator.

1. Permit join
2. Commissioning sequence by settings channel (emits: "Commissioning")
3. Confirm channel with A1 + B0 (emits: "Short press 2 of 2")

commissioning_hue_and_coordinator_range.pcapng.txt

[Koenkk]

@florianbrede-ayet forgot to ask this, but could you include the herdsman debug log of this first sniff? (commissioning_hue_range.pcapng.txt)

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

[florianbrede-ayet]

Hi, I repeated the procedure (with coordinator out of range) and this time also captured the herdsmanlog.

With my limited knowledge of the ZigBee protocol it still looks like there isn't any traffic in herdsman around 16:33:17 UTC. I also don't see any traffic from 0x001788010b56d3e7 in the whole herdsmanlog (it's just idling, so expected) but also not from the at least 10 other hue bulbs which likely were in range of the switch as well (an example is eingang_flur_deckenleuchte / 0x001788010b9b1f41 (0x2C4B), which was also in range).

Procedure was again after force-removing the switch: 1.) Permit join 2.) Commissioning sequence by setting channel (emits: "Commissioning") 3.) Confirm channel with A1 + B0 (emits: "Short press 2 of 2")

wireshark.pcapng.txt herdsmanlog.txt

[Koenkk]

I've compared your sniff with me pairing a Hue Tap via a Hue bulb, the problem is that your Hue router doesn't send the commissioning notification. (left mine, right yours)

My sniff: hue_tap (1).pcapng.txt

I honestly have no clue why it doesn't, maybe it has something to do with the Conbee II (but I highly doubt since it already goes wrong on the bulb side).

@pklokke do you maybe have an idea?

[florianbrede-ayet]

@Koenkk thanks for your efforts. After going through your cap, I retried multiple times with different routers & switches but really none of the hue routers send commissioning notifications.

I was hoping that my z2m configuration or something with herdsman might be wrong which configures the hue devices to discard ZGP messages. The ZCL Green Power: GP Proxy Commissioning Mode messages (and the general communication) across the network look identical to yours, so that's my last straw - before I have 20 unused Senic Friends Of Hue switches to sell :smiling_face_with_tear:.

[pklokke]

Hi Guys,

I have been using hue lamps here (along with others) as well as SR-ZGP2801K4-FOH-E units fitting in some local (Danish) switch sockets.

As @Koenkk knows I'm currently working on some unicast code to improve performance, but the recent Hue Lamps/Routers do act as perfectly normal Green Power Proxies.

I just had a try on my own setup, and it worked as expected, as you can see here:

One interesting thing I noted about the forwarded frames from Hue GPP vs. others is that the sequence number starts at 1, I wonder if that could be an issue.

Network key in case its needed is: DE4EAA6842EA171176F72E5365E996AB pklokke network.pcapng.txt .

[florianbrede-ayet]

@pklokke thanks for the sniff, that was quite interesting because your PTM has an identical configuration / firmware as mine.

What I found as main difference between my and both your and @Koenkk s messages is that my routers reply with a different profile in their GP Proxy Commissioning Mode message (left yours, right mine):

My cluster and endpoints match however the profile is Home Automation instead of Green Power. Could that explain my issue?

PS: My coordinator message is also sending the Home Automation profile instead of Green Power - and this applies to all routers in the mesh

PPS: Yeah my mesh is doing garbage there

[Koenkk]

@florianbrede-ayet well found, I think this indeed explains your problem. Could you check if it is fixed in the latest-dev?

Changes will be available in the dev branch in a few hours from now. (https://www.zigbee2mqtt.io/advanced/more/switch-to-dev-branch.html)

[florianbrede-ayet]

@Koenkk I'm running edge anyway and with the latest herdsman commit it works as expected, the profileId was the culprit.

I had already checked the deconz implementation and found that the profile was hardcoded, so thanks for the quick fix.

[Koenkk]

@chrishae FYI I've applied the following fix: https://github.com/Koenkk/zigbee-herdsman/commit/a4fad01e82e76f6e646d402912abd2861b26ed89

[SlawekMM]

I have one question. I try understand if I can send from z2m message in greenPower format? I know I need have a proxy when I want reading messages from greepower Switch Enocean...But If i can make virtual greepower switch and sending greepower messages in mqtt? I need it to break some limitations in some zigbee system...

[Dande768]

I seem to run into the exact same issue running the HomeAssistant plugin. The only difference I see compared to @florianbrede-ayet is that I am running the Conbee 2 firmware version suggested on the zigbee2mqtt homepage instead of the newest on. Could that be the issue?

Zigbee2MQTT Version 1.30.3 commit: unknown

Coordinator-Typ ConBee2/RaspBee2

Coordinator-Version 0x26580700

Coordinator IEEE Adresse 0x00212effff082faf

Frontend Version 0.6.127