Feature Request - Send raw Xbee packets to a remote node

[JamesonBeebe]

I would love to use Xbee devices with my homekit/homebridge setup. Is there a quick and dirty way to discover all Xbee nodes locally, enumerate them, and then send them raw xbee packets?

[exsilium]

Hi @JamesonBeebe!

Let me get this.. you want to:

1. On startup, make a Network Discovery (ND)
2. Based on the discovered nodes, create a device list for homebridge and register them as Accessories?
3. What happens next? what devices are they? what signals should be sent? Are they switches? Some sensory equipment?

This plugin does Network Discovery on start but it doesn't build any device list based on the responses. Partly because you never know what type of device it is. That's why the device should be defined in the config.

Here we handle the ND response: https://github.com/exsilium/homebridge-xbee/blob/419a1188c213d341556282c8973f5430c3e9d570/xbee.js#L87

Here we are calling the discovery completed: https://github.com/exsilium/homebridge-xbee/blob/419a1188c213d341556282c8973f5430c3e9d570/xbee.js#L99

If you have specific usecase, you can hack it together quite easily:

1. on ND response, push the radio addresses to an array to save the list of radios that you have on the network during boot.
2. On finishing the discovery register the devices / Accessories as you require. Use the trigger() example to see how to send some frames to the registered devices.

[JamesonBeebe]

Thanks for the help, I additionally ran into problems installing from NPM, but I had to adjust some read/write permissions.

I'll give this a try tomorrow and report back. Thanks for the help!

[JamesonBeebe]

hello @exsilium, Thanks for the help. I think I'm almost there, I got the device to show up in Homekit, however, when I start homebridge, I get an unusual error:

Unable to retrieve NT value from XBee radio, XBee Platform init failed!

I can seem to interact with the buttons (however nothing in the physical world happens), but when I Ctrl+C to stop homebridge, I get a freeze up on my R-Pi and I am forced to power cycle.

I am going to look into this part because it seems I have some settings wrong. Any thoughts?

[exsilium]

Hi! Sad to hear you are having issues. The fact that the XBee Platform init fails is an indication that the communication with the locally connected XBee is failing. How have you configured the homebridge platform?

"platform":   "XBee",
"name":       "XBee Devices",
"port":       "/dev/ttyAMA0",
"baudrate":   9600,
"api_mode":   1,
"timeout":    20000

So I guess the port is correct but make sure the speed is the same as configured on your XBee. Also, make sure you are running the module in API mode and that the api_mode configuration reflects the escaping.

However, even if the XBee communication is not working correctly, your RPi shouldn't freeze up. Can you add more details, which RPi do you have and how have you wired the XBee connection with your board (or are you using an USB dongle/bridge?)

[JamesonBeebe]

Yes, you are correct, I originally had this connected to a USB port with FTDI chip, but I changed the connection to the UART, and also lowered the serial comms speed to 9600 and that seemed to do the trick (the Xbee device was already in API mode).

These changes got rid of the "NT" startup issue, but the homebridge still hangs when I shut it down using Ctrl+c. I can still start another SSH session to the Pi and reboot manually. The last line that I see when it hangs is quoted below:

[11/16/2017, 2:22:39 PM] [XBee Devices] Serial port open... ^C[11/16/2017, 2:22:43 PM] Got SIGINT, shutting down Homebridge...

What do you mean by:

the api_mode configuration reflects the escaping.

The Xbee module is in API mode, but maybe something else isn't configured properly?

[exsilium]

If you can still ssh to RPi, then it doesn't hang. Simply the Homebridge wont quit on SIGINT for some reason. Can you reproduce the behaviour, without homebridge-xbee plugin being installed/loaded? If only this plugin introduces the issue, I have to look more into it - but as far as I have tested it on my RPi2, I haven't had issues shutting homebridge down. However, I'm running Ubuntu as an OS.

As for the api_mode, it has to reflect the setting of your XBee radio's AP parameter as outline by the XBee documentation

If the XBee platform is initialised correctly and you also see in the console output the neighbours discovered, then I'd say the plugin is successfully loaded! 🙌😎

[JamesonBeebe]

hey @exsilium. After trying most of the day to get this to work, I'm afraid I have something going on. I have node 8x installed on my raspberry pi, and I'm testing homebridge with only the Xbee module installed. I am seeing the same problem though on a separate machine. I get a hang when i send the ctrl+c command and I get an output just like the one before.

This is the last component I have left to implement in a project i'm working on. Do you have any ideas or things I can try to troubleshoot?

[JamesonBeebe]

you know... sometimes you just gotta talk it out to solve the problem. So there is a reconfiguration in the mapping of the serial Comms in a Rpi3. you basically have to follow these instructuctions https://www.raspberrypi.org/forums/viewtopic.php?f=107&t=138223&start=50

All traffic on /dev/ttyAMA0 appears to be sent to the bluetooth module, so there has to be some configurations as stated in the link above. Next step is actually getting it to turn an LED on remotely and enumerating the I/O's.

Stand by for more Questions...

[JamesonBeebe]

I am pretty new to setting up homebridge in my house, so your instructions at the very beginning are a little hard for me to understand.

I basically have one Xbee device connected to my Pi, and another remote device (which is discovered during the ND step). I would like to turn a light on when I ask Siri to turn on the light or when I press the on screen button. It doesn't seem like my packets are being sent to the target remote device. I am monitoring that device through XCTU to see what exactly is being sent.

Would you mind guiding me a little more down the rabbit hole? :-)

[exsilium]

Hi!

1. Do you see the remote device being discovered during homebridge boot up? The discovered device address must appear in the console.
2. Can you show me your homebridge config file?
3. Have you added the device with the correct addresses to the config file?
4. Is the remote device also an XBee or something else?
5. What is the value of the AO parameter of your sending XBee and receiver XBee? For the sake of experiment make sure both radios have AO = 3

Note that the plugin trigger signal is an ZigBee command frame that gets sent:

https://github.com/exsilium/homebridge-xbee/blob/419a1188c213d341556282c8973f5430c3e9d570/xbee.js#L120

If AO = 0, it might not show up on the receiver side? You could modify the plugin to change frame type that gets sent.

[JamesonBeebe]

1. Yes, I see the remote Zigbee device being discovered now.

2. Here's my Json config: config.json.zip

3. I believe I added the correct address to the config file. If I'm not mistaken, the address in the the config file needs to be the remote Xbee device I'm communicating to correct?

4. I have 2 Xbee devices from Digi, and the remote device is a Xbee. It is the same model and series as the Xbee connected to the Pi. I am just working right now on toggling D0 as an LED, but I would like to add other functionality to the other pins on the remote device.

5. What do you mean by A0? Is that an Xbee parameter or part of the module?

[JamesonBeebe]

@exsilium I am still a bit lost with the AO paramenter. Is that part of the node module, or an Xbee config parameter? Just a bit of clarification... I am trying to send a REMOTE_AT_COMMAND_REQUEST to toggle a digital output (for a light).

Here are the parameters I would like to send in the Xbee frame: Remote device 64 bit ID = 0013a2004064c982 16 bit ID = 0x0003 Remote command options: 0x02 AT command: D0 Parameter Value: (toggle D0 on and off), so the commands would be 4 (low) and 5 (high)

Here's my output from homebridge:

[2017-11-19 03:21:34] Homebridge is running on port 51826.
[2017-11-19 03:21:34] [XBee Devices] Serial port open...
[2017-11-19 03:21:34] [XBee Devices] Discovery timer set to: 1000
[2017-11-19 03:21:35] [XBee Devices] Discovered device: 0013a2004064c982 (0003) (,Light)
[2017-11-19 03:21:35] [XBee Devices] Device discovery completed!
[2017-11-19 03:21:45] [XBee Devices] Power state for the 'Lightmodule' is 0
[2017-11-19 03:21:53] [XBee Devices] Set power state on the 'Lightmodule' to 1
buffer.js:183
    throw new TypeError(kFromErrorMsg);
    ^

TypeError: First argument must be a string, Buffer, ArrayBuffer, Array, or array-like object.
    at Function.Buffer.from (buffer.js:183:11)
    at new Buffer (buffer.js:158:17)
    at BufferBuilder.appendString (/usr/lib/node_modules/homebridge-xbee/node_modules/buffer-builder/buffer-builder.js:73:28)
    at Object.frame_builder.(anonymous function) (/usr/lib/node_modules/homebridge-xbee/node_modules/xbee-api/lib/frame-builder.js:55:11)
    at XBeeAPI.buildFrame (/usr/lib/node_modules/homebridge-xbee/node_modules/xbee-api/lib/xbee-api.js:87:28)
    at xbeeTrigger (/usr/lib/node_modules/homebridge-xbee/xbee.js:126:30)
    at Object.trigger (/usr/lib/node_modules/homebridge-xbee/xbee.js:133:54)
    at PushButtonAccessory.setPowerOn (/usr/lib/node_modules/homebridge-xbee/index.js:63:10)
    at emitMany (events.js:147:13)
    at Characteristic.On.emit (events.js:224:7)

I would really like to help you in the near future to create a more generic branch to this module because a lot of work has already gone into it and I have some remote power switches up and running with custom ruby on rails server, but I would love to use homebridge and control everything with Siri and homekit.

Just a sidenote, I'm monitoring an Xctu console on my remote Xbee device for received "Remote AT Commands" to see that the frame builder was able to assembly and successfully send the frame correctly. Once I can get this step up and running, I think I'll be able to close this issue and start making some progress and contribute a bit to the project when I'm done with the current project I'm working on.

[exsilium]

Hi! I've been AFK, thus the delay.

1. The AO parameter I was mentioning referred to XBee's parameter you can set via XCTU. However, as at the latet post you said that you are actually sending (or trying to send) REMOTE_AT_COMMAND_REQUEST, the AO can be default (0).

2. The config looks ok, but it doesn't match with the addresses in the later output. I assume you are aware of it and that you have done the changes accordingly. You can also set the 16bitnetwork to fffe to indicate broadcast. But the devices section the 64bitaddress needs to be the target device's full address. So based on the console output, that would be the discovered device: 0013a2004064c982

3. Based on the new accessory type "Lock" you are trying to implement, I can't say what is the error caused by your code. Needless to say, when you are toggling the power state of "Lightmodule" no signal is sent because the XBee frame cannot be correctly built (for whatever reason). Is the code you are running modified or not?

If you have modified code, I'd urge you to also use GitHub for the development of it. Otherwise it's hard to see what changes have been made and what might be the issue.

If you are running unmodified code, please let me know your exact node8 version, serialport and xbee-api versions so I can try to reproduce the error.

As for a generic XBee device handler, all Pull Requests are welcome! 👍 Cheers! 🍻

[JamesonBeebe]

Ah ok, @exsilium, I see what's happeneing. My Xbee modules are a bit old, (they were discontinued from Digi actually some of them still say Maxstream and a company I worked for discarded the ones that the technicians thought were "defective" but I knew they weren't). I have been Xbee-fying things for a Long time. Also, they are Series 1 Xbee modules, and they are not DigitMesh capable. So they do not have an A0 parameter. I tried to send the AT command to my device and a command not found error came back to me. That settles that problem for now....

You are quite observant because I did swap the xbee modules to see if maybe there was a TX error, so the addresses should indeed appear to be reversed. Sorry about the confusion and very observant catch. 👍

Let me do something... I modified the xbee.js file to send the commands that are needed for a remote AT request and the data that goes with it. The reason I modified the xbee.js was to force the module to send a DO low signal. I'm not sure if this is the right way of going about things, but let me post the changed part down here:

var xbeeTrigger = function(destination64, destination16) {
  if(xbeeAPI && serialPort) {
    // We send the trigger frame
    var dataArray = new Uint8Array(3);
    dataArray[0] = 0x44; //select DO
    dataArray[1] = 0x30;
    dataArray[2] = 0x04; //send DO Low command

    var frame_obj = {
      type: C.FRAME_TYPE.REMOTE_AT_COMMAND_REQUEST,
      destination64: destination64,
      destination16: destination16,
      options: 0x02,
      data: dataArray
    };
    serialPort.write(xbeeAPI.buildFrame(frame_obj));
  }
}

I also did a small modification to the Index.js (only the buttonName config variable to default to "light_name" for aesthetics). In this case, the default example module in the index.js is a pushbutton accessory, but I would like a Light, or a "Lock accessory."

So I have 3 questions:

1. Is there another library that lists the types of accessory that one can define triggers for?

2. Is there a library somewhere in the homebridge-xbee structure that defines which data commands need to be sent in the case of different accessories? (i.e. Digital Output, Analog Input, pwm, etc.....)

3. Since I have very very little experience with JavaScript, would it be possible for you write an example trigger in the index.js file for a simple remote light switch connected to an Xbee end node?

Trust me...your help is more appreciated than you know! 🍻 and the pull request is on its way....

[exsilium]

Hi! Oh, Series 1. I guess I'm guilty at assuming things...

But still, it should work. So based on the above it's clear now about the error you are receiving. For frame type: C.FRAME_TYPE.REMOTE_AT_COMMAND_REQUEST you build the frame differently:

{
    type: 0x17, // xbee_api.constants.FRAME_TYPE.REMOTE_AT_COMMAND_REQUEST
    id: 0x01, // optional, nextFrameId() is called per default
    destination64: "0013a20040401122",
    destination16: "fffe", // optional, "fffe" is default
    remoteCommandOptions: 0x02, // optional, 0x02 is default
    command: "BH",
    commandParameter: [ 0x01 ] // Can either be string or byte array.
}

Instead of the dataArray, use the command (String) and commandParameter attributes. Please see the xbee-api documentation for more examples how the frames of different types are built.

1. The included PushButtonAccessory is the only accessory currently implemented.
2. No, because of nr. 1, PushButtonAccessory is currently the only accessory
3. I think you're almost there 😄

I guess a light or similar such generic device could be coded as a supported accessory type. You specify the node's address and the signal you want to send when you operate the accessory. However, the PushButtonAccessory is truly a simplistic example. It doesn't have state, because it always returns to Off. So just defining an On action the config isn't enough, you must have at least 3 commands configured for: On/Off/Status.

[JamesonBeebe]

hey @exsilium, Thanks for the encoragement. I agree that I'm almost there, but you know the challange of diving into another developer's code and figuring out what is actually going on.

I must admit that as an EE with years of experience, I have never really taken a dive into Java or JS. I actually have this Xbee feature working as a python script, but your node module seems like a more elegant solution.

I know this is a big ask, but is there a possibility that you would be able to update your code with another example in the index.js and xbee.js of setting, reading, and turning a digital output on and off as well as the config.json that should go along with it. Honestly, the push button is a great place to start, but I think it is quite a jump to toggle a digital output or read an analog input.

I think in the future, it will help others as well. I was expecting to get up and running pretty fast with this module and it appears to be the only Xbee homebridge NPM module out there. Once I get the hang of that, I will be happy to work on making the module a little more general.

Much appreciated 😃

[exsilium]

I'll try to find some time to make a generic XBeeSwitch accessory for toggling the Outputs 👍

[JamesonBeebe]

Hey @exsilium. I was just wondering how things are going with the sample output toggling? I've tried on my end by adding a destinationPin and destinationCommand value to thee xbeeTrigger in the xbee.js file.

Also, in the index.js file, I modified the xbee.trigger functions to reflect the extra variables that are sent to the xbeeTrigger function. I have also modified the lightBulbService function to have a status, on, and off methods (not sure if I did that correctly though...

Unfortunately, I don't have access to the machine with the code on it right now, so I can't send you sample code, however does it sound like that's the way to go? Also, how is the progress coming on your end? I appreciate the help 👍

[exsilium]

Hi @JamesonBeebe ! It would be helpful to see the code up in GitHub prior to commenting on it 😃 I'm glad you're having progress, keep it going :). I haven't had time to dig into this myself - Hopefully soon.

[exsilium]

Hi @JamesonBeebe , can you try out this feature branch and see if the new Switch accessory works for you? https://github.com/exsilium/homebridge-xbee/tree/feature/xbeeSwitch

Still to be done:

• Status read back
• Handle properly callbacks (switch toggle only when we get the signal back)

One thing I noticed, however, is that in my case the XCTU Console did not show the incoming and outgoing frames for the command 🤷‍♂️ However, if you go and check the actual pin state (refresh in the radios settings), then the state is changed correctly. Might be a XCTU bug...