ZigBee Binding

[cdjackson]

I have an initial ZigBee binding running. Currently controlling Hue lights, but will also shortly add other devices.

[kaikreuzer]

Wow - can you already disclose some details? Are you using some library? What ZigBee dongles are supported?

[cdjackson]

I'm using a lib called zigbee4java. It's a 'port' of the ZB4O library - it's easier to manage than ZB4O and is on Git rather than (I think) a private server. https://github.com/tlaukkan/zigbee4java

Currently it only supports a TI Dongle (USB evaluation board) which is the same as ZB4O but I want to split this section out and am discussing this with the library maintainer.

Hue lamps are now working very nicely - auto discovery and control of on/off, brightness and color. I need to add color temperature yet. I don't intend to separate out all the different bulbs, although during the discovery process I do read the manufacturer and model.

My next goal is to get my temperature sensors working, but they are less compliant it seems to some of the subscription services, and being battery devices, they sleep (which has the same, or worse, consequences as ZWave battery devices).

[kaikreuzer]

I'm using a lib called zigbee4java. It's a 'port' of the ZB4O library

Interesting to use a fork that strips out OSGi to then package it in an OSGi application afterwards ;-) But I see the point, there is no need to have devices as OSGi services, so for embedding in openHAB, a pure Java version is easier to embed.

Hue lamps are now working very nicely

This now leads to the first architectural problem with the Thing concept: It is the same device as for the hue binding, so it would actually be nice to use the existing thing type definition. But there is no easy way to say that this thing type can be used through either a hue bridge or through a zigbee stack... I will have to think about this - if you have any good solution, please let me know!

[presslab-us]

Awesome, thanks a lot!

I gave this a shot last night and I was able to get my GE Link bulb working, but my Cree Connected wouldn't work. I loaded it up in the ZB4OSGI tester and I see the Cree bulb come up ZLL profile and not HA so it's not properly detected with either library I assume. ZLL is kinda weird as a ZLL device should be usable on any HA network but I think zigbee4java isn't responding with the HA profile so the Cree thinks it's a ZLL network.

I have a handful of other ZigBee HA devices I'd be happy to add support for.

[cdjackson]

Thanks fir the feedback - I'd certainly welcome any updates to devices (things) as well as the zigbee lib itself... I am working with the maintainer of the lib to fix a frw bugs so I do anticipate some updates but im not sure it will fix the issues you report.

The lib should respond to HA cluster as thats what im using for some temperature sensors. However im still reasonably new to the zigbee world having spent a loy of time with zwave so im still learning the system.

Anyway, any updates are welcome  - my current focus is on getting my temperature and humidity sensors running and solving some of the issues with battery devices...

Cheer  Chris 

Sent from Samsung Mobile

-------- Original message --------

From: presslab-us notifications@github.com

Date:24/02/2015 15:20 (GMT+00:00)

To: openhab/openhab2 openhab2@noreply.github.com

Cc: Chris Jackson chris@cd-jackson.com

Subject: Re: [openhab2] ZigBee Binding (#142)

Awesome, thanks a lot! I gave this a shot last night and I was able to get my GE Link bulb working, but my Cree Connected wouldn't work. I loaded it up in the ZB4OSGI tester and I see the Cree bulb come up ZLL profile and not HA so it's not properly detected with either library I assume. ZLL is kinda weird as a ZLL device should be usable on any HA network but I think zigbee4java isn't responding with the HA profile so the Cree thinks it's a ZLL network. I have a handful of other ZigBee HA devices I'd be happy to add support for. — Reply to this email directly or view it on GitHub.

[presslab-us]

After resetting the Cree bulbs they come up as a HA device ID now, 0x100 which is "OnOff Light". I think from new they are "set up" for a Wink hub. However they do have the dimming cluster (and ability), which seems like 0x100 is a bug with these new lights, sigh... The GE Link come up as 0x101 "Dimmable Light" which is correct. Maybe the best way forward is to lump these IDs into one type and then look at what clusters are supported to determine if it's dimmable.

Maybe a "Supported devices" wiki or Gdocs could be made somewhere? I've got working so far a Dresden color strip, a Centralite pluggable dimmer, GE Link bulb, and Cree Connected bulb. Next up is the thermostat. I will do a pull request when I get a few things together.

Thanks, Ryan

[cdjackson]

Great stuff Ryan. The Dresden strip converters are something I was going to get myself, so I’m pleased to hear they work :)

I suggest for the minute we stick the information in this issue - I can probably append it to the first message to build a list of devices? Until OH2 gets a bit more stable, then we can add the docs into the binding which I think is the ultimate goal…

Anyway, I do agree that the way I named the ‘things’ of using the device type doesn’t work. I’d already found this with the temperature sensors I’ve got…. I’ve just pushed an update that is hopefully a better solution - it uses a list of clusters, and to find the ThingUID it searches through the list looking for the best batch. So, to add a new thing, you can just provide the thing name, and the list of clusters the device supports, and the system ought to sort it out… I think this ought to work fine - I don’t think we should get more than one best match if we configure the clusters/things appropriately, but we’ll see :)

Cheers Chris

[presslab-us]

I checked out your update, seems like the best way forward.

There is one issue with this that I can see though. I've got the subscriptions working but if the zigbee4java device is an "OnOffLightDevice" it cannot subscribe to the "changedCurrentLevel" callback. So either this needs to be fudged in zigbee4java (not ideal) or I think there is a different way to subscribe, I'll look into it...

I'll work on getting my additions committed/pull request. Or I can just give you a diff and you can clean it up/change it as you like and commit it.

Thanks, Ryan

[cdjackson]

Hi Ryan, If you just want to push your changes to Github, I can merge them into my repo and push them back - there’s no need for a PR so long as I know your repository address - I can just pull directly…

I’ve not got subscriptions working on my devices - my thermostats don’t respond to them when they are asleep, and the hue comes back with a ‘queue full’ error (is that what you’re seeing when trying to subscribe?), so I’m nt sure what’s up here. I’ve been discussing stuff with the maintainer of the library and we’re going to make some changes to some of this, but again, any thoughts or changes are welcome…

Chris

[kaikreuzer]

I suggest for the minute we stick the information in this issue - I can probably append it to the first message to build a list of devices? Until OH2 gets a bit more stable, then we can add the docs into the binding which I think is the ultimate goal…

Actually, the idea behind the thing types is to be the database of the supported devices. So instead of "DimmableLight", there should be a "GE Bulb", a "CREE bulb" etc. For "generic" devices that we might not know yet, but which should nonetheless be supported, a "DimmableLight" type could still be kept - or maybe it would be just an "unknown Zigbee device" type, which does not list any channels in the xml, but simply gets its channels when being added by the handler. From these XMLs, we could then in future generate documentation about supported devices. And what's more, we can show the devices (maybe even with pictures in future) within the UI, incl. description texts, which might contain some specific information about them.

Would you agree that this approach makes sense or is there any reason against going down this path?

[cdjackson]

Ok, I hadn’t realised this was the idea - sorry. I do agree that it makes some sense, but I also fear this can become a maintenance nightmare and we’ll end up trying to support a ‘database’ of devices with very diverse features/requirements in the XML files… If you went down this route, can bindings add their own information into the XML files for device specific stuff? Currently I think xstream will throw an exception if there’s data in the XML that it doesn’t know about (unless you’ve managed to find some way to stop this?)… If so, it might be nice as in zwave I could merge the database with this, but the zwave database has a lot of stuff in it (not just simple names etc) so there would need to be freedom to add ‘anything’ the binding needed (somewhere), and then to be able to get it out again (somehow).

Maybe we’re getting off the topic of the zigbee binding. For now, I suggest with zigbee we leave the generic devices while the binding is still quite fluid, and once things firm up we can add more specific devices - if you’re ok with that? I’d suggest a separate discussion somewhere about the XML files though if you want to start that (or shall I - this will impact zwave when I start to port that, which I hope will be soon).

[presslab-us]

@kaikreuzer On the one hand I would vote it should be as generic as possible to ensure compatibility with future devices, this seems to be the spirit of ZigBee HA. On the other hand as I already see with my few devices that the Cree is not following standard but I'm not sure how big a problem this is. With specific devices like Nest, etc. it makes sense to have the thing strictly for the device in question. But with ZigBee I would say if possible leave it more generic and I agree with @cdjackson that for now keep it generic and see how things play out as more devices are tested.

[presslab-us]

@cdjackson Re the subscriptions I found a data type bug in zigbee4java that would trigger if the top bit of the address happened to be set. short org.bubblecloud.zigbee.network.packet.af.AF_INCOMING_MSG.getSrcAddr() should return int instead of short. The problem with changedCurrentLevel callback is that the OnOffLightDevice does not have the getLevelControl method to subscribe to this.

[cdjackson]

Thanks - I’ve just modified this to use int instead of short. I’m not sure why was a short - there were then a bunch of utility methods to print as an int, and then ultimately further up the stack it was converted to int anyway,,,

I’ve not merged this change from the library - let me know if you need this quickly, otherwise I’ll wait a couple of days as there are some library changes coming up to split out the USB driver...

[kaikreuzer]

but I also fear this can become a maintenance nightmare and we’ll end up trying to support a ‘database’ of devices

Isn't this what you have done with Z-Wave as well? I think it is nicer to have the database within the binding that just maintaining a long "supported devices list" on the wiki. The nice thing is that if a device needs some special handling, it is easy to provide a special ThingHandler for it to deal with the specialities.

And again: Yes, the idea of the ThingTypes is that they reflect a certain product/model. This is what you will find in most other bindings as well and I think all bindings should follow the same guidelines.

can bindings add their own information into the XML files for device specific stuff?

Do you have a concrete example? I think I see your point: Instead of having to implement specific ThingHandlers, you would still keep a single generic ThingHandler for all devices and have it dynamically adjust its behavior based on specific information in the thing descriptions. This is indeed not possible right now, but definitely worth to follow up upon. I have created https://www.eclipse.org/forums/index.php/m/1638625/#msg_1638625 for this.

On the one hand I would vote it should be as generic as possible to ensure compatibility with future devices

@presslab-us I agree, that's why I also said that it makes sense to keep a generic "DimmableLight" (and similar) thing type as a generic fallback option. Nonetheless, it would already give some valuable information to the user to see whether his device is explicitly supported (by having a dedicated thing type) or if it only falls in the "if it has implemented Zigbee the right way, it should in theory work" category.

[presslab-us]

Do you imagine an addition to the web GUI so that users can see what devices are supported before buying them? What would the motivation be for adding a specific Thing if the generic one works? Only for documentation?

[presslab-us]

@cdjackson What sort of problems are you seeing with battery devices? I am seeing Z_MAC_NO_ACK and timeout errors with my thermostat on battery power when directly reading/writing attributes. Sometimes it works though, and subscribe notifications work fine.

[cdjackson]

Yes - battery devices need more work. I’ve been talking with Tommi about this and we need to look at how messages ae handled (queued) to battery devices, but we’re also going to implement a save and restore feature as my battery devices disappear from the network when it’s restarted. We’ll effectively reload them into the stack on startup…

This areas needs a bit of work...

[cdjackson]

Hey Ryan, I've just pushed an update. This updates the library to the latest with the separation of the port from the main library and also has the various other fixes (eg short/int fix).

One point to note is that I also changed the thing naming and configuration. Previously I used the IEEE address in the Discovery service, but actually in the thing I was expecting the full endpoint ID - I'd not noticed this bug before as I'd configured my things manually and obviously hadn't checked it when I added the discovery service. So, you might (likely!) find you need to modify (or remove and re-add) your things and update your items accordingly.

Cheers Chris

[presslab-us]

Hi Chris, How about something like this for the device naming? It would take up less space in the items file and seems more consistent with the existing naming. String thingId = device.getEndpointId().toLowerCase().replaceAll("[^a-z0-9_/]", "").replaceAll("/", "_");

[cdjackson]

Does that end up concatenating the mac address and then putting _endpoint on the end? If so, it sounds fine (as you say, it’s a little shorter than I have now).

[presslab-us]

Yes, that's right.

I've got your update merged, I'll try to get some things committed to my repo tonight. On Mar 1, 2015 11:57 AM, "Chris Jackson" notifications@github.com wrote:

Does that end up concatenating the mac address and then putting _endpoint on the end? If so, it sounds fine (as you say, it’s a little shorter than I have now).

— Reply to this email directly or view it on GitHub https://github.com/openhab/openhab2/issues/142#issuecomment-76626981.

[presslab-us]

I pushed some of my changes to the repo here: https://github.com/presslab-us/openhab2/tree/zigbee

[cdjackson]

Thanks - all looks good :)

I’ve merged your changes into my branch so we can hopefully maintain a common source.

[presslab-us]

Great! Thanks.

I was thinking that it might be better to keep the light related methods all in ZigBeeLightHandler and I commited this https://github.com/presslab-us/openhab2/commit/56915899c20f97412a0798d4da9a5c4456a22d0f to show what I mean. What do you think?

[cdjackson]

I'll take a look tonight, but this sounds like a good move to keep the class sizes down and nicely distributed. Thanks.

[kaikreuzer]

Do you imagine an addition to the web GUI so that users can see what devices are supported before buying them? What would the motivation be for adding a specific Thing if the generic one works? Only for documentation?

@presslab-us: I have further contemplated about this and I guess you and @cdjackson are right - a rule such as "a thing type has to be a specific product/model" does not make sense. There are situations, where it is a good match (Z-Wave seems to be such a case), but others where mapping a thing type to a certain device category is the better option. It should be possible to both levels within the same binding as well. So having a "DimmableLight" as a generic type is fine and it can cover every bulb that behaves nicely wrt the specs. If you come across a bulb that you know that needs some special treatment and workarounds, you can define a specific thing type for this product. This also gives the possibility to assign a special thing handler (that might extend the generic thing handler) to it that holds the according specific logic. This could potentially even be added through a separate bundle, so that such a binding could be nicely extensible. One important feature through would be that the discovery service is extendible, too: If a bundle e.g. brings a specific description and handler for the Cree bulb, it should be able to hook into the discovery to make the discovery result not a DimmableLight thing type, but a Cree bulb thing type. This could possibly be achieved through a similar construct as we have in place for the UPnPDiscoveryService, which also delegates to UPnPDiscoveryParticipants.

Does this make sense to you?

[cdjackson]

Does this make sense to you?

Hi Kai, This makes sense - basically, the discovery needs to find the most appropriate ‘thing’ definitions - ie. it should look for specific things first, and then if nothing is found, we use the generic definition. Sounds good.

Chris

[cdjackson]

I've now updated the code to include the zigbee library as a jar (mvn) rather than having to copy in the source each time which was a pain (and wrong!) for a bunch of reasons. I've not merged it into my zigbee branch yet, but will do soon -: https://github.com/cdjackson/openhab2/tree/zigbee-mvn

[kaikreuzer]

I am wondering if we should move to bndtools for the build and dependency management - including external jars would probably much easier...

[smulikHakipod]

Hey, I am currently implementing the door lock and OTA clusters in the zigbee4java and I want to help in implementing the zigbee binding in openhab. I saw cdjackson and presslab-us implementation in their github profiles. I am new to openhab, and my question is how do you test your zigbee devices? from the osgi console? what commands do you use? with which configuration? sorry for the noob question.

[cdjackson]

Hi, Yes - I do most of my ‘playing’ and testing with the console - it’s easy(er) to hack about to get things working, and as it needs to be merged into the library, it’s the best place to get things working… Once things are working there, then modify the binding to add support for the new devices…

I’m currently looking to add support for 2 new devices - a plug in switch, and temp/humidity sensors.

I’m also likely to remove the link to the maven dependancy, and static linking the library as that will make it easier to add snapshots of the library into the binding. I’ll probably do the above couple of things over the next day or so (in fact the library change is done already).

Chris

[cdjackson]

OH - and I meant to add - great to have you aboard :)

[cdjackson]

I've now merged all changes back into the zigbee branch. The binding has been updated with the latest fixes from the zigbee library, and the updates to ESH for the latest thingStatus definition (which also means you need to update your environment to the latest OH2). See https://github.com/cdjackson/openhab2/tree/zigbee. I will remove the other branches in a few days.

[ubisys]

As a company making ZigBee HA devices, we would like to support this effort. We could provide documentation, devices and OTA images for a range of devices with different standard clusters. These will also be upgradable to ZigBee 3.0 shortly. If you are interested, please send an email to support@ubisys.de, referring to openHAB ZigBee binding project.

[kaikreuzer]

@ubisys (as Jeffrey's post is deleted, I also removed your answer to it - I think we should not worry about this),

Thanks a lot for your very kind offer - I think this could indeed help very much for development and testing; especially ZigBee 3.0 might be interesting as well as it can take a while before stable products can be found on the market. The development of the Zigbee binding is currently driven by @cdjackson and he isn't located in Germany - so I'll leave it up to him what kind of support he might want from you.

Best regards, Kai

[ubisys]

Thank you, Kai. We are already in touch with Chris... Best regards, Your ubisys Support Team

[sonycamera]

@cdjackson, I have a new Ti CC2531 dongle, should I re-flash it to ZStack HA firmware before it can work with this binding?

Best Regards, Fisher

[cdjackson]

I think so - I'm not 100% sure what firmware comes as standard so if you can flash new FW then it would put you in a known configuration.

[kgoderis]

Just wondering, Kai, if this binding-in-the-making would be a valuable route to for the Miele@Home stuff. The Miele XGW3000 exposes a lot of ZigBee-related stuff in a rather transparent way, e.g. there are jsonrpc methods provided to not only control and configure the gateway itself, but it also exposes part of the ZigBee stack. Maybe it is a bridge too far....

[kaikreuzer]

@kgoderis I doubt that it would be useful to treat the XGW3000 as a general purpose Zigbee bridge. It probably makes more sense to simply have support for the Zigbee dongle itself within the Zigbee binding.

[kgoderis]

Maybe I was not clear enough.... option 1. is to implement Miele binding interfacing with the XGW3000, using "half of the " zigbee stuff. option 2. is indeed to support Telstick (?) Zigbee dongle in the Zigbee binding. Obviously, doing away with the XGW in option 2 will also mean that the Miele iOS app etc will no longer function.....

[kaikreuzer]

You mean you might be able to spare some efforts by reusing code from the Zigbee binding? Well, I cannot judge whether that's feasible or not. But maybe it is better to discuss that further on the Miele issue, not here.

[dasfuu]

Just wanted to mention this rasperrypi zigbee module: https://shop.dresden-elektronik.de/wireless-2/boards-kits/boards/raspbee.html

I think this is very interesting and should be relatively easy to use. But depending on how we use it it may be better with its own binding. (Using the provided rest api for example)

[cdjackson]

I had looked at this, but as it's RPi dependent it's not something I would currently spend time supporting (maybe later). Unfortunately, unlike ZWave, the zigbee interface is not standard (every dongle supplier has a different interface, and it's not trivial to support different dongles in the library at the moment as it was written around the TI API. It is something I want to look at in future, as I have a couple of sticks in mind...

If there's an API for this board (I don't mean REST), it would be interesting to see in case it's similar to others.

[dasfuu]

There should be an API. The C++ software seems to be opensource, and there are many ressources on the website, even for people that want do develop their own firmware.

[Matticusguy]

Will this change with the implementation of zigbee 3.0, which according to the alliance is the standardisation of the protocol?

Sent from my Samsung device

-------- Original message -------- From: Chris Jackson notifications@github.com Date: 22/06/2015 11:29 (GMT+00:00) To: openhab/openhab2 openhab2@noreply.github.com Subject: Re: [openhab2] ZigBee Binding (#142)

I had looked at this, but as it's RPi dependent it's not something I would currently spend time supporting (maybe later). Unfortunately, unlike ZWave, the zigbee interface is not standard (every dongle supplier has a different interface, and it's not trivial to support different dongles in the library at the moment as it was written around the TI API. It is something I want to look at in future, as I have a couple of sticks in mind...

If there's an API for this board (I don't mean REST), it would be interesting to see in case it's similar to others.

— Reply to this email directly or view it on GitHub.

[cdjackson]

Good question! The reason it works with zwave is that Sigma sell / control all zwave chips made, so they can ensure the user API is consistent. With zigbee, that's not the case - there are a lot of manufacturers of SoC - Zigbee will (hopefully!) try with their type approval process, but we'll see what happens in time...

Fingers crossed! :) It would make life a LOT easier - the current situation is that each stick is often very different in it's computer interface and what/how it exposes the different Zigbee layers.

[dasfuu]

https://github.com/ninjasphere/go-zigbee is now opensource. Does it work similar to the mentioned TI zigbee usb adapter? They both seem to use the same TI Zigbee Z-Stack. The CC2530 and CC2531 seem to be the exact same thing.(31 has Serial over USB?)

[cdjackson]

Note that I've made some major changes to the binding.

• Things are now equivelant to physical devices - more in line with the ESH definition of things - rather than endpoints.
• Channels are now mapped to specific endpoints, and/or attributes within the endpoints
• I've added the ability to bind to devices endpoints
• I've added light sensor, temperature sensor and occupancy sensor
• This is supported by the Develco ZHT101 occupancy sensor (light/temperature/occupancy)
• I've added an intermediate 'converter' class to link between clusters and channels (this is similar to the way the zwave binding works).

This is obviously still work in progress and the architecture might change a little still, but it's now quite a flexible architecture.