Is there something more I need to do to get command listeners working?

[reinux]

I have a bunch of switches -- two Ikea switches and a motion sensor, and a Philips dimmer.

I get various responses via AddCommandListener: NetworkAddressRequest, ReportAttributesCommand, NodeDescriptorResponse, PowerDescriptorResponse etc. coming form various devices. ReportAttributesCommand in particular seems somewhat interesting, as it reports as Cluster 6 and Cluster 8 -- so something to do with on/off and levels. But they don't seem to come when I press any buttons.

I have AddSupportedClientCluster and AddSupportedServerCluster set for 6 and 8 (not sure which I need to be doing).

Things happen in the playground when I press buttons, though, so I think the output is just being filtered for some reason. I'm pretty sure though that in my database in my own code, the nodes should be getting recognized.

Is there something more that I'm missing?

[02:53:56 DBG] Slip Recv:0E-17-00-07-00-AA-00-2A-FF [02:53:56 DBG] Slip Send:17-17-00-07-00-00-00-CB-FF [02:53:56 DBG] Slip Recv:17-17-00-21-00-1A-00-22-01-09-00-00-02-3E-8D-01-04-01-06-00-03-00-01-19-00-00-AF-FF-59-5F-07-00-DF-29-FB [02:53:56 DBG] RX APS: ZigBeeApsFrame [sourceAddress=36158/1, destinationAddress=9/0, profile=260, cluster=6, addressMode=0, radius=0, apsCounter=0, payload=1 25 0] [02:53:56 DBG] Incoming message from unknown node 8D3E: Notifying announce listeners [02:53:56 INF] 36158: NWK Discovery node discovery already in progress [02:53:56 DBG] RX ZCL: ZclHeader [frameType=CLUSTER_SPECIFIC_COMMAND, manufacturerSpecific=False, direction=CLIENT_TO_SERVER, disableDefaultResponse=False, manufacturerCode=0, sequenceNumber=25, commandId=0] [02:53:56 DBG] Unknown node 36158 [02:53:56 DBG] Incoming message did not translate to command.

[Mr-Markus]

It seems that node is not added to local Nodes list in your process, yet.

As you can see: [02:53:56 INF] 36158: NWK Discovery node discovery already in progress It is not finished yet.

After your programm is started it reads nodes from your database, runs discovery to get endpoints etc and add it to local nodes list. And just of your node is there it can send and receive messages.

Is your program running for a while when you get this message, or does it happen right after you started it?

[reinux]

Hmm getting my things to connect to the playground wasn't going too well, so I set up Serilog for my own program (which also doesn't work, of course), and I get this:

[03:11:18 DBG] Slip Recv:0E-6B-00-07-00-AA-00-D6-FE [03:11:18 DBG] Slip Send:17-6B-00-07-00-00-00-77-FF [03:11:18 DBG] Slip Recv:17-6B-00-21-00-1A-00-22-01-09-00-00-02-3E-8D-01-04-01-06-00-03-00-01-2E-01-00-AF-FF-35-60-07-00-DE-E3-FA [03:11:18 DBG] RX APS: ZigBeeApsFrame [sourceAddress=36158/1, destinationAddress=9/0, profile=260, cluster=6, addressMode=0, radius=0, apsCounter=0, payload=1 46 1] [03:11:18 DBG] Unknown local endpoint for APS frame ZigBeeApsFrame [sourceAddress=36158/1, destinationAddress=9/0, profile=260, cluster=6, addressMode=0, radius=0, apsCounter=0, payload=1 46 1] [03:11:18 DBG] Incoming message did not translate to command.

[DavidKarlas]

I think what you are missing is Zigbee Bind, https://github.com/Mr-Markus/ZigbeeNet/blob/master/libraries/ZigBeeNet/ZDO/Command/BindRequest.cs You basically need to tell switch or any other sensor device, that when some action happens like button, where to send this information... There is also concept of BindingTable https://github.com/Mr-Markus/ZigbeeNet/blob/master/libraries/ZigBeeNet/ZDO/Command/ManagementBindResponse.cs#L43 which lists to which devices some commands will be sent... The way Ikea Gateway and most other gateways work is, that they make Bind request between specific Switch and Light or multiple lights(so they fill Switch binding table with one or more devices) so even when Gateway is offline things keep working... What you need to do is... send Zigbee BindRequest to Switch and tell it to send commands to your Coordinator so it will show up in your app.

[DavidKarlas]

Notice that Endpoint has GetInputClusterIds and GetOutputClusterIds so for cluster On/Off, Switch has ClusterOnOff as output, Light has ClusterOnOff as input, so you need to bind this 2 things so Switch will control light over that cluster... Thats why there are all this clusters for Levels, Colors, OnOff... So device can support only subset and other device also supports only subset, but then Coordinator makes correct Bindings to connect this Inputs and Outputs... Atleast thats my understanding how this should work...

[reinux]

That makes sense, though it still doesn't seem to be working.

I'm able to update and get the binding tables for my lights, but for my switches, either Ikea or Philips, UpdateBindingTable returns FAILURE and there are no items in the table.

BindingRequest likewise doesn't seem to do anything. Looks like it's supposed to be on Cluster 0x0021, which is Green Power, which some of my lights have but not my switches. Maybe I have the parameters wrong...

  varbr = new BindRequest();
  br.BindCluster = uint16 0x06;
  br.DstAddrMode = byte 0x03;
  br.DstAddress = network.LocalIeeeAddress;
  br.SrcAddress = node.IeeeAddress;
  br.SrcEndpoint = byte endpoint;

[DavidKarlas]

Im not sure how is sleeping/waking up on this end devices supposed to work... Maybe try to send this command right after rejoining network or maybe pressing one of buttons... :/

[reinux]

Progress!

I'm now able to bind and unbind the controller, but it's still not sending me OnOff commands (or anything under ZCL):

BindingTable [srcAddr=00178801086E2D79/2, dstAddr=00212EFFFF05CF3C/1, clusterId=1] BindingTable [srcAddr=00178801086E2D79/2, dstAddr=00212EFFFF05CF3C/1, clusterId=64512] BindingTable [srcAddr=00178801086E2D79/1, dstAddr=11, clusterId=6] BindingTable [srcAddr=00178801086E2D79/1, dstAddr=11, clusterId=8] BindingTable [srcAddr=00178801086E2D79/1, dstAddr=53052, clusterId=6] BindingTable [srcAddr=00178801086E2D79/1, dstAddr=53052, clusterId=5] BindingTable [srcAddr=00178801086E2D79/1, dstAddr=00212EFFFF05CF3C/1, clusterId=8]<- bound to controller BindingTable [srcAddr=00178801086E2D79/1, dstAddr=00212EFFFF05CF3C/1, clusterId=6]

Logs show this when I press a button (what's Cluster 64512?):

[13:33:30 DBG] Slip Recv:0E-A1-00-07-00-AA-00-A0-FE [13:33:30 DBG] Slip Send:17-A1-00-07-00-00-00-41-FF [13:33:30 DBG] Slip Recv:17-A1-00-2B-00-24-00-22-02-00-00-01-02-0C-B7-02-04-01-00-FC-0D-00-1D-0B-10-FC-00-04-00-00-30-02-21-01-00-00-AF-FF-8E-A4-07-00-D8-B4-F7 [13:33:30 DBG] RX APS: ZigBeeApsFrame [sourceAddress=46860/2, destinationAddress=0/1, profile=260, cluster=64512, addressMode=0, radius=0, apsCounter=0, payload=29 11 16 252 0 4 0 0 48 2 33 1 0] [13:33:30 DBG] RX ZCL: ZclHeader [frameType=CLUSTER_SPECIFIC_COMMAND, manufacturerSpecific=True, direction=SERVER_TO_CLIENT, disableDefaultResponse=True, manufacturerCode=4107, sequenceNumber=252, commandId=0] [13:33:30 DBG] Unknown input cluster 64512 [13:33:30 DBG] Incoming message did not translate to command.

[spudwebb]

looks like 64512 is a manufacturer specific cluster, so it's not directly supported by the library.

what are the input and output clusters of your device?

[reinux]

 Input clusters
  Endpoint 1: 0 Basic
 Output clusters
  Endpoint 1: 0 Basic
  Endpoint 1: 3 Identify
  Endpoint 1: 4 Groups
  Endpoint 1: 5 Scenes
  Endpoint 1: 6 On/Off
  Endpoint 1: 8 Level Control
 Input clusters
  Endpoint 2: 0 Basic
  Endpoint 2: 1 Power Configuration
  Endpoint 2: 3 Identify
  Endpoint 2: 15 Binary Input (Basic)
 Output clusters
  Endpoint 2: 25 Ota Upgrade

[DavidKarlas]

BindingTable [srcAddr=00178801086E2D79/1, dstAddr=00212EFFFF05CF3C/1, clusterId=8]<- bound to controller

Maybe problem is endpoint on dstAddr=00212EFFFF05CF3C/1 it maybe should be dstAddr=00212EFFFF05CF3C/0... Can you maybe also post log for BindRequest being sent?

[reinux]

When I set DstEndpoint to 0, even though it says SUCCESS, it doesn't seem to actually update the table when I do an UpdateBindingTable after -- only when the endpoint is 1.

Not too sure where the relevant logs end, but here's when I set DstEndpoint to 0:

[13:36:15 DBG] TX CMD: BindRequest [0/0 -> 46860/1, cluster=33, transId=6, SrcAddress=00178801086E2D79, SrcEndpoint=1, BindCluster=6, DstAddrMode=3, DstAddress=00212EFFFF05CF3C, DstEndpoint=0] [13:36:15 DBG] Slip Recv:12-10-00-09-00-02-00-2E-05-A0-FF [13:36:15 DBG] 588E81FFFE5EDF7C: Node SVC Discovery: scheduled ["NWK_ADDRESS", "NODE_DESCRIPTOR", "POWER_DESCRIPTOR"] [13:36:15 DBG] TX APS: ZigBeeApsFrame [sourceAddress=0/0, destinationAddress=46860/0, profile=0, cluster=33, addressMode=Device, radius=31, apsCounter=6, payload=6 121 45 110 8 1 136 23 0 1 6 0 3 60 207 5 255 255 46 33 0 0] [13:36:15 DBG] Slip Recv:0E-11-00-07-00-AE-00-2C-FF [13:36:15 DBG] 588E81FFFE5EDF7C: Node SVC Discovery: running [13:36:15 DBG] TX CMD: ManagementBindRequest [0/0 -> 46860/0, cluster=51, transId=7, StartIndex=0] [13:36:15 DBG] Slip Send:17-11-00-07-00-00-00-D1-FF [13:36:15 DBG] Slip Recv:17-11-00-20-00-19-00-2E-02-00-00-00-02-00-00-00-00-00-36-80-02-00-04-00-00-AF-F2-0C-2A-09-00-04-CD-FC [13:36:15 DBG] TX APS: ZigBeeApsFrame [sourceAddress=0/0, destinationAddress=46860/0, profile=0, cluster=51, addressMode=Device, radius=31, apsCounter=7, payload=7 0] [13:36:15 DBG] TX CMD: NetworkAddressRequest [0/0 -> 65535/0, cluster=0, transId=8, IeeeAddr=588E81FFFE5EDF7C, RequestType=0, StartIndex=0] [13:36:15 DBG] RX APS: ZigBeeApsFrame [sourceAddress=0/0, destinationAddress=0/0, profile=0, cluster=32822, addressMode=0, radius=0, apsCounter=0, payload=4 0] [13:36:15 DBG] TX APS: ZigBeeApsFrame [sourceAddress=0/0, destinationAddress=65535/0, profile=0, cluster=0, addressMode=Device, radius=31, apsCounter=8, payload=8 124 223 94 254 255 129 142 88 0 0] [13:36:15 DBG] RX CMD: ManagementPermitJoiningResponse [0/0 -> 0/0, cluster=32822, transId=4, Status=SUCCESS]

And here's when I set it to 1:

[13:37:48 DBG] TX CMD: BindRequest [0/0 -> 46860/1, cluster=33, transId=6, SrcAddress=00178801086E2D79, SrcEndpoint=1, BindCluster=6, DstAddrMode=3, DstAddress=00212EFFFF05CF3C, DstEndpoint=1] [13:37:48 DBG] Slip Recv:0E-11-00-07-00-AE-00-2C-FF [13:37:48 DBG] Slip Send:17-11-00-07-00-00-00-D1-FF [13:37:48 DBG] TX APS: ZigBeeApsFrame [sourceAddress=0/0, destinationAddress=46860/0, profile=0, cluster=33, addressMode=Device, radius=31, apsCounter=6, payload=6 121 45 110 8 1 136 23 0 1 6 0 3 60 207 5 255 255 46 33 0 1] [13:37:48 DBG] Slip Recv:17-11-00-20-00-19-00-2E-02-00-00-00-02-00-00-00-00-00-36-80-02-00-04-00-00-AF-A5-4F-2A-09-00-01-DA-FC [13:37:48 DBG] B0CE181400033D89: Node SVC Discovery: running [13:37:48 DBG] RX APS: ZigBeeApsFrame [sourceAddress=0/0, destinationAddress=0/0, profile=0, cluster=32822, addressMode=0, radius=0, apsCounter=0, payload=4 0] [13:37:48 DBG] TX CMD: ManagementBindRequest [0/0 -> 46860/0, cluster=51, transId=7, StartIndex=0] [13:37:48 DBG] TX CMD: NetworkAddressRequest [0/0 -> 65535/0, cluster=0, transId=8, IeeeAddr=B0CE181400033D89, RequestType=0, StartIndex=0] [13:37:48 DBG] RX CMD: ManagementPermitJoiningResponse [0/0 -> 0/0, cluster=32822, transId=4, Status=SUCCESS]

[reinux]

Is Endpoint 0 a "catch-all" endpoint or something?

Also, deCONZ shows a couple endpoints and clusters, but I'm not sure if these are being opened by the config app at runtime or if they're built-in:

I'm looking at the switch's binding list from earlier, and it doesn't seem like deCONZ had bound it to the controller at all, which makes me wonder if maybe it's just intercepting traffic as it comes.

I'm not sure if that would work if the command happens not to go through the controller, though, would it? Or does everything on the mesh get passed along to every device or at least to the controller?

[Mr-Markus]

It depends on manufacturer implementation. You can have multiple Endpoints with different clusters. Usually Endpoint 0 should be the "default", but as mentioned it can be different if the manufacturer does something special.

Because of this we do a node discovery to get all endpoints and it's clusters. As I understand you would Bind two devices to each other?

The ZDO BindRequest does have an SrcEnpoint Property https://github.com/Mr-Markus/ZigbeeNet/blob/45b2020450573af70280f58b4e61f052ffd6194d/libraries/ZigBeeNet/ZDO/Command/BindRequest.cs#L46

This will be used in the ZclCluster class https://github.com/Mr-Markus/ZigbeeNet/blob/d1f0201eab1ac9d46cdbdd8a9dfbf94edafc7cb0/libraries/ZigBeeNet/ZCL/ZclCluster.cs#L625-L637

And here the endpoint will be taken from the endpoint you created the cluster with. This will be done in node discovery task. If there is a cluster on two endpoints you need to choose the endpoint by your own and use it's cluster object

[reinux]

And here the endpoint will be taken from the endpoint you created the cluster with.

Maybe this is what I'm missing... I've been using AddSupportedServerCluster. Is there something else I need to do?

[Mr-Markus]

Is your discovery task running sucessfull? Then reuse the node's endpoint and clusters to bind it to another node.

A code example will be helpful

[reinux]

I followed the playground code and added discoveryExtension, but I'm not too sure if it's doing what it needs to.

Sorry this is all F#. I edited it so that it's hopefully more straightforward.

open System

open ZigBeeNet.App.Discovery
open ZigBeeNet
open ZigBeeNet.ZCL.Clusters.OnOff
open ZigBeeNet.ZCL.Clusters.LevelControl
open Serilog
open ZigBeeNet.ZDO.Command
open ZigBeeNet.ZCL.Clusters.Basic
open ZigBeeNet.ZCL

let port =
  ZigBeeNet.Tranport.SerialPort.ZigBeeSerialPort "COM3"

let dongle =
  ZigbeeNet.Hardware.ConBee.ZigbeeDongleConBee port

let store =
  ZigBeeNet.DataStore.Json.JsonNetworkDataStore("devices")

let network = ZigBeeNetworkManager dongle
network.SetNetworkDataStore(store)

let discoveryExtension = ZigBeeDiscoveryExtension()
discoveryExtension.SetUpdatePeriod 60
network.AddExtension discoveryExtension

network.AddNetworkNodeListener
  { new IZigBeeNetworkNodeListener with
      member x.NodeAdded node = Log.Information("NodeAdded: {0}", node)

      member x.NodeRemoved node =
        Log.Information("NodeRemoved: {0}", node)

      member x.NodeUpdated node =
        Log.Information("NodeUpdated: {0}", node) }

Log.Logger <-
  LoggerConfiguration()
    //.MinimumLevel.Debug()
    .MinimumLevel.Information()
    .WriteTo.Console()
    .CreateLogger()

printfn $"Initialize: {network.Initialize()}"

printfn "Set channel status: %A"
<| network.SetZigBeeChannel(ZigBeeChannel.CHANNEL_11)

network.AddSupportedClientCluster(uint16 0x06)
|> ignore

network.AddSupportedServerCluster(uint16 0x06)
|> fun result -> Log.Information("AddSupportedServerCluster", result)

network.AddSupportedClientCluster(uint16 0x08)
|> ignore

network.AddSupportedClientCluster(uint16 0x0021)
|> ignore

network.AddSupportedServerCluster(uint16 0x08)
|> ignore

Log.Information("Startup: {0}", network.Startup(false))

Log.Information("PermitJoin: {0}", network.PermitJoin(60uy))

Log.Information("Network state: {0}", network.NetworkState)

Log.Information("Local address: {0}", (network.LocalIeeeAddress, network.LocalNwkAddress))

// This creates a new IZigBeeCommandListener implementation.
network.AddCommandListener
  { new IZigBeeCommandListener with
      member x.CommandReceived cmd =
        let ieee =
          network
            .GetNode(cmd.SourceAddress.Address)
            .IeeeAddress

        Log.Information("Command received: {0}",
          {| clusterId = cmd.ClusterId
             sourceAddress = cmd.SourceAddress
             sourceIEEE = ieee
             ``type`` = (cmd.GetType().FullName)
          |}
        ) }

// I just hard-coded a bunch of nodes and endpoints for testing.
// philipsSwitch is the switch that I'm testing with.
let philipsSwitch =
  network.GetNode(IeeeAddress("00178801086E2D79")), 1

let node, endpoint = philipsSwitch

let br = BindRequest()

br.BindCluster <- uint16 0x06
br.DstAddrMode <- byte 0x03
br.DstAddress <- network.LocalIeeeAddress
br.SrcAddress <- node.IeeeAddress
br.SrcEndpoint <- byte endpoint
br.DstEndpoint <- byte 0

node.SendTransaction(br, br)
|> Async.AwaitTask
|> Async.RunSynchronously
|> fun result -> Log.Information("BindRequest: {0}", result)

let ubt =
  (node.UpdateBindingTable()
   |> Async.AwaitTask
   |> Async.RunSynchronously)

printfn "UpdateBindingTable: %A" ubt
printfn "Binding table:"

Log.Information("Binding table: {0}", node.BidndingTable)

[matteobortolazzo]

@reinux hi, were you able to receive commands? I cannot receive anything from Ikea dimmers

[reinux]

No luck yet with either Philips or Ikea dimmers.

[DavidKarlas]

Maybe problem is this: https://github.com/WebThingsIO/zigbee-adapter/issues/159#issuecomment-536147269

[reinux]

Mine are working with Phoscon, which is the web app that's meant to be used with Conbee, so at least in my case I don't think that's the cause.

I'm sure it's something really simple that I'm missing. Hmm...

[reinux]

Is there something in the sample for demonstrating command listeners? Could there be some code that I could try following?

[Mr-Markus]

An example is here:

https://github.com/Mr-Markus/ZigbeeNet/blob/401318cb5310180e74e3b7e1b6749a59ebca6ea7/samples/ZigBeeNet.PlayGround/Program.cs#L581-L601

And here how it is used: https://github.com/Mr-Markus/ZigbeeNet/blob/401318cb5310180e74e3b7e1b6749a59ebca6ea7/samples/ZigBeeNet.PlayGround/Program.cs#L190

[reinux]

Nuts, I'm already doing exactly that.

Guess I'll keep investigating on my off-hours. Thanks!

[reinux]

I tried binding two lights and two switches directly using ZclOnOffCluster.Bind, and it still doesn't seem to want to work, so there must be something wrong with the binding as opposed to the listener...

If I bind, say, an on/off switch to a light, assuming they're both already on the same network, and I get the IEEE address, cluster (6) and endpoints (1) right, I should be able to just bind from the switch to the light to control it, right? Whether the coordinator is on or not?

   BindingTable [srcAddr=00178801086E2D79/1, dstAddr=500B91400001EDF4/1, clusterId=6]
     {ClusterId = 6us;
      DstAddr = 500B91400001EDF4;
      DstAddrMode = 3uy;
      DstGroupAddr = 0us;
      DstNodeEndpoint = 1uy;
      SrcAddr = 00178801086E2D79;
      SrcEndpoint = 1uy;}]

What's also weird is that I managed to get a message from my Ikea switch just once and never again:

Command received: ToggleCommand [On/Off: 1247/1 -> 0/1, cluster=6, transId=29]

[reinux]

I finally teased out the issues:

The Philips switch works fine; I just wasn't binding it correctly. It also needs full resets after you mess up binding.

The Ikea switches don't actually care about bindings even though you can set them; they just send everything to endpoint 0, I think it's probably a broadcast message, even though it's sending to endpoint 0:

2021-03-12 10:43:15.068 -08:00 [DBG] RX APS: ZigBeeApsFrame [sourceAddress=13575/1, destinationAddress=0/0, profile=260, cluster=6, addressMode=0, radius=0, apsCounter=0, payload=1 96 2]
2021-03-12 10:43:15.085 -08:00 [DBG] Unknown local endpoint for APS frame ZigBeeApsFrame [sourceAddress=13575/1, destinationAddress=0/0, profile=260, cluster=6, addressMode=0, radius=0, apsCounter=0, payload=1 96 2]

The message always gets stuck here:

https://github.com/Mr-Markus/ZigbeeNet/blob/a38d1c2ade02349e6f66ac9622e30d2b4a8bead9/libraries/ZigBeeNet/ZigBeeNetworkManager.cs#L775-L779

LOCAL_ENDPOINT_ID is 1 and BROADCAST_ENDPOINT_ID is 255.

Would it be possible to relax this check? Are there any situations where permitting messages outside of these two endpoints would cause a problem? If there are, could it raise another event instead so that it can be worked around?

[Mr-Markus]

It's a pitty, but manufacturers are implementing some stuff differently so that many things working differently. I do not know why ZigBeeAlliance allows this to it's members. Just take a look at Philips Hue, where only Philips devices will work with some features. And some manufacturers, like Ikea are not implementing all clusers or functions. And sometimes they are treating informations differently or ignore it. So it it not always possible to detect if it's a bug in the code or missing or wrong implementation on devices side.

Your problem is that it returns null, isn't it?

[reinux]

Yeah, it returns null, which ultimately means the message just gets discarded before it surfaces as an event. I wonder if the only "real" solution might be to expose more lower-level data...

For now, I've simply commented this whole block out on my local build, which seems not to have any ill effects -- unless this has something to do with the serial port dying, which I doubt.

It really is a shame what's happening in the home automation space. I've seen Philps WiFi bulbs that sell for nearly half the price of Ikeas, and they have way more sophisticated circuitry... the only difference being that they're 100% proprietary and require that you're always online, presumably so they can lock you in and gather data. The amount of anti-consumer behavior is pretty terrifying.