Feature Request: Pair Philips products using serial number

[itavero]

One of the things I liked about the Philips Hue bridge/app, is that you can just enter a list of serial numbers of the lights it should pair with. Given that there is no easy way to reset the Philips lights at this point in time, I actually used this mechanism to reset some of the lights at home (pair them to the Philips bridge by serial number and then remove them from the bridge, which resets them and then they join my zigbee2mqtt network).

I think it would be interesting to see if we can figure out how this pairing procedure works (what the magic behind it is). It would make pairing Philips Hue lights so much easier if we could just send an MQTT message with a list of serial numbers, while zigbee2mqtt is allowing new devices onto the network.

Unfortunately, I'm very new to zigbee2mqtt and I still have to find my way around the code base. However (once I find the right tools in my moving boxes) I can probably help in analyzing the pairing procedure.

[dreimer1986]

Not a fix to your request, but: https://www.zigbee2mqtt.io/information/touchlink.html

[itavero]

I'm aware of the Touchlink feature, @dreimer1986. However, I think if this particular feature can be implemented it would be way more practical (instead of having a spare E27 socket near my coordinator 😉).

[itavero]

I have setup my "sniffing" environment and started investigating on the communication (I've attached the PCAP files of the first logging attempts, while I was listening on channel 11, which is a ZLL channel).

Three times I told the app to search for one or two lights by serial number, using the following (fake) serials:

1. ABCD01
2. DEAD01 and 0FF1CE
3. ABCD01 (again)

A saw similar messages on channel 11 each time containing the following in the data field:

1. e6 ac ad 13 21 e6 b1 f5 01 01 cd ab 00
2. e6 ac ad 13 21 e6 b1 f5 02 01 ad de 00 ce f1 0f 00
3. e6 ac ad 13 21 e6 b1 f5 01 01 cd ab 00

A few important things stand out:

• The first and last data is identical (meaning that it's probably easier to reproduce the messages).
• The first 8 bytes are identical.
• 9th byte appears to be the number of serial numbers that are in the mesage.
• 10th byte and further seems to be the 24-bit serial numbers (but with a swapped byte order) and followed by a "padding" byte (00) to make it a 32-bit number.

After this I actually bound a bulb to the Philips Hue bridge via this method. I saw a similar message (that also fits the things I wrote down above), but surprisingly I saw it from two different sources. One of them was the same as in the previous capture, but before that i saw the exact same data from a different source address. I reckon this different source address might be the bulb (maybe a confirmation on a message it received on another channel?)

After binding, I could tell in the Philips Hue app that the bridge is actually using channel 25. Also, it started updating the light bulb, so I guess I'll have to continue "hacking" another time.

Next time I will sniff on channel 25, first of all to try and confirm that the source address was indeed the lightbulb. Secondly, I hope to see the entire pairing on this particular channel, so I can start making sense on how the first 8 bytes are used (which I reckon are some kind of key).

For the PCAP logs, see pcap_pair-by-serial_20200504.zip

Disclaimer: it's been over 5 years since I last glanced at ZigBee traffic and looked over any ZigBee specifications.. bear with me 😉

[itavero]

I quickly did some more logging today. Today I setup the sniffer on the same channel as the Hue bridge, removed the light in the app, started the sniffer and added the light again using its serial number.

What I noticed today is that the first 8 byte of the special message mentioned previously, is actually the Extended PAN ID that can also be found in the beacons send by the Hue bridge. As mentioned, I really need to refresh my ZigBee terminology, but here's what I got from the capture today (which I've also attached):

• Hue bridge sends out a cluster-specific Zigbee Cluster Library Frame (command 0; sequence 0), containing its Extended PAN ID, followed by the number of serial numbers and all serial numbers as "little-endian" 32 bit numbers.
• After that I see a beacon request (from the light bulb that recognized its serial, maybe?)
• Beacon request send by Hue bridge (amongst others), with previously sent Extended PAN ID.
• Association Request send by light bulb to the Hue bridge (using the source address of the beacon).
• Followed by exchange of some data (?) and an Association Response from the Hue bridge to the light bulb.
• Followed by more exchange of data (transport key and some other stuff, which I reckon is pretty standard?)

I hope to dive in to the messages soon and also start replaying some of the messages of the Philips Hue bridge (with the actual bridge off) to see how the light bulb actually responds.

PCAP log: 2020050821_pcap_pairing.zip

[Koenkk]

Only the first point is specific to this, the other points are normal zigbee pairing behaviour.

It would be interesting to see how the bridge deals with different channels. E.g. when the bulb is on channel 11, and the bridge is on channel 25, it has to transmit a message on channel 11.

[itavero]

The first capture I did, I was listening to channel 11 and also saw the message even though it paired on channel 25. It looks as if the bridge sends the message on every ZLL channel (although I haven't confirmed that).

I think I'll do another test where I pair the bulb, turn it off and reset the Hue bridge. After the reset, I'll set the Hue bridge to a different channel and pair the bulb again (assuming that this way the bulb will initially stay on the other channel).

[itavero]

By the way, is there an "easy" way to try and retransmit these messages?

[Koenkk]

There is not an easy way to retransmit these messages. Implementation wise I think this will be similar to touchlink (https://github.com/Koenkk/zigbee-herdsman/blob/master/src/controller/touchlink.ts). It for example also does channel switching.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[junosuarez]

Unstale. Had a need for this today to try to add some second-hand hue devices. Instead I gave up and ordered a Hue Dimmer, which I plan on using only to force the devices back into pairing mode to work with my z2m coordinator. In a perfect world, I could have added them by serial number from within z2m, like this Amazon Alexa flow https://www.youtube.com/watch?v=UmaPWKP79Bw

[st-sfdc]

Tried to add a hue lightstrip that I forgot to remove from the bridge first. No way to put it into pairing mode by power cycling, searching by serial would come in nice right now.

[fractalcounty]

Tried to add a hue lightstrip that I forgot to remove from the bridge first. No way to put it into pairing mode by power cycling, searching by serial would come in nice right now.

I have three hue Lightstrips sitting in a pile because they're impossible to factory reset. Neither power cycling, nor touch link, nor dimmer switch works. I feel your pain here!

[Blackspell01]

Same here. I didn't even know that such a problem could occur. Tf