Support for Hue motion sensor

[ebaauw]

I'm new to deCONZ, just received my RaspBee today. I would like to adapt my homebridge plugin for the Philips Hue bridge to work with deCONZ and maybe in the long run ditch the Hue bridge all together.

I managed to connect some of my lights, a Hue tap, a Hue dimmer switch and a Hue motion sensor to deCONZ. The latter isn't supported - I would like to add support for it. Any pointers on how to do that would be much appreciated.

[manup]

Hi, the motion sensor integration as well as general switch improvements like button events will be available in the github repo soon. We will also roll out basic Websocket support for events so that apps will be notified immediately when something happens and don't need to poll all the time.

[ebaauw]

the motion sensor integration as well as general switch improvements like button events will be available in the github repo soon

Thanks, I cannot wait.

I don't see the Hue tap ZGP switches in the deCONZ network view, but that's probably to be expected? They do show in the REST API

The Hue motion sensors show as HA? devices (note the question mark) in deCONZ and the rest API plugin shows them as ZHAPresence, whereas the Hue bridge shows them as ZLLPresence. Same for the Hue dimmer switch (ZHASwitch vs ZLLSwitch).

We will also roll out basic Websocket support for events

Cool! In my homebridge plugin for Sonos, I subscribe to events from the Sonos players rather than polling them. It would be übercool to do something similar for the ZigBee devices.

[manup]

The Hue Tap is a ZigBee Green Power (ZGP) device, only able to send but not to receive commands. In the node view only ZigBee PRO based devices are visible (ZHA, ZLL and ZB30), I'm afraid it will be that way for a while.

The ZHA/ZLL thing is a little mess, I would like to just remove the prefix and replace it with ZBSwitch or just Switch but this might break running apps. The ZLL and ZHA doesn't add any value here, soon there will be ZigBee 3.0 devices.

[ebaauw]

The ZHA/ZLL thing is a little mess, I would like to just remove the prefix and replace it with ZBSwitch or just Switch but this might break running apps.

How compatible do you want to be with the Philips Hue API? Do you want apps for the Philips Hue to work with deCONZ? If not, by all means, drop the ZLL/ZHA classification.

I noticed that uniqueid isn't.
A Hue motion sensor shows as two endpoints: 02 is a ZHA Occupancy Sensor endpoint with server clusters 0000, 0001, 0003, 0400, 0402, 0406; and 01 is a ZLL On/off sensor endpoint with only a 0000 server cluster and client clusters for light control. The Philips Hue bridge shows the Hue motion sensor as three sensor resources, one ZLLLightLevel (the 0400 cluster), one ZLLTemperature (the 0402 cluster) and one ZLLPresence (the 0406) cluster. The battery level is exposed in each sensor resource. The uniqueid value is derived from the mac address, endpoint and cluster: "00:17:88:01:02:xx:xx:xx-02-040y".
In my homebridge-hue plugin, I create a single HomeKit accessory for a Hue motion sensor with five services: Accessory Information, Motion Sensor, Light Sensor, Temperature Sensor, and Battery. I match the Hue bridge resources based on the mac address in the uniqueid.

The deCONZ REST API exposes the Hue motion sensor as three sensor resources as well, a ZHALight, a ZHAPresence, and a ZHASwitch, all with the same uniqueid value, based solely on the mac address: "0x0017880102xxxxxx". The battery seems exposed only in the ZHALight.
The ZHASwitch resource seems superfluous, it only has a lastupdated state attribute. The presence attribute is missing from the ZHAPresence resource's state, as are many config attributes. I'm excited to see a duration config attribute, though. The ZHALight resource's state contains an attribute lux, whereas Philips uses lightlevel, which is not in lux and Booleans dark and daylight (with thresholds in config).

Incidentally, deCONZ also created superfluous ZHASwitch sensor resources for my innr PL110 and UC110 lights. These expose a single ZLL Dimmable Light endpoint 01, which includes client clusters 0004, 0005, 0006, and 0008. Apparently the control boards for these lights can do a lot more than what they're using it for. Interesting: they also expose 0019 OTAU where innr told me explicitly that their firmware cannot be updated over the air.

[manup]

I think it's quite nice being inter-operable also on the app level. That's sadly not always possible, for example RaspBee supported Touchlink, scenes and groups prior to Hue and meanwhile they have rolled out their own flavor of it which is quite different.

You already figured out the lux vs. lightlevel difference which is the same story here - in this case the lux parameter will be replaced by lightlevel. The uniqueid issue for motion sensor is addressed in the upcoming update, I've stumbled over the same thing :) presence, button events and rules in general are also refactored.

The duration can be specified with the ontime parameter (1/10 s) in a rule action body like {"on":true, "ontime": 600} which will result in a ZigBee On With Timed Off command.

I'm not a big fan of it but I guess in future we might use a white-list for supported devices to prevent creation of sub-optimal sensors and lights in the API, it's always a pain in the neck to clean that up afterwards. The client on/off cluster (0x0006) for example automatically triggers the creation of an ZHASwitch.

[ebaauw]

I think it's quite nice being inter-operable also on the app level.

In that case, I think the deCONZ rest API should mimic the Hue bridge behaviour as much as possible. See also my remarks about UPnP discovery. It might provide additional attributes, but it should deliver the same attributes as the Hue bridge, possibly using different values. I would like to see a modelid in config, to recognise deCONZ vs the gen-1 (BSB001) and the gen-2 (BSB002) Hue bridge. Also, we should align the apiversion, which deCONZ now reports as 1.0.0.

RaspBee supported Touchlink, scenes and groups prior to Hue

deCONZ figured out my groups and scenes from the lights - I'm quite impressed. Of course it didn't get the group nor scene names, as they aren't stored in the lights. I ran into some funny issues regarding light names. Apparently, only some types of Philips lamps allow storing the name. When I set the names through the rest API, they appear correctly in the deCONZ gui, except for one (OSRAM) light, which gets changed to Kitchen Rightÿÿÿ.

presence, button events and rules in general are also refactored

Please let me know if I can help. For starters, I've documented the endpoints and clusters that deCONZ reports for my devices. Some of these aren't (yet) recognised by deCONZ, I'll try and create a ZCLDB XML source for these.

The duration can be specified with the ontime parameter

Sorry, I meant the duration attribute in the ZHAPresence sensor's config. Hoping this would indicate how long deCONZ reports presence as true, after motion was detected by the Hue motion sensor.

I'm not a big fan of it but I guess in future we might use a white-list for supported devices

Me neither, yet I started one for my homebridge-hue plugin anyway. The level of freedom within (and next to) the ZigBee standard seems too big for ZigBee (certified) devices to "just work". Different colour gamuts, colour temperature changes, older firmware in Osram lights than don't turn off with transitiontime > 0 (which is the default), innr lights that hang on "alert": "select".
The Osram lights allow setting the power-on default, I think they might use the FC0F cluster for that.

The client on/off cluster (0x0006) for example automatically triggers the creation of an ZHASwitch.

The Hue motion sensor provides this (and the client Level Control, Color Control, Groups, Scenes, Identify). Does this mean that (theoretically) it could work without a Hue bridge, interacting directly with the lamps/groups, like the Hue dimmer switch?

[ebaauw]

The client on/off cluster (0x0006) for example automatically triggers the creation of an ZHASwitch.

Indeed, I found a ZHASwitch sensor for my Hue bridge as well.

[manup]

I would like to see a modelid in config, to recognise deCONZ vs the gen-1 (BSB001) and the gen-2 (BSB002) Hue bridge.

Good suggestion, I'll figure something out.

Also, we should align the apiversion, which deCONZ now reports as 1.0.0.

That's sadly a complicated issue because even if some features of higher versions are implemented, not all features can be implemented, for example remote access to Philips server sure won't be possible. But I can image that the apiversion could be raised in future and missing features will gracefully return a not implemented error code.

Please let me know if I can help. For starters, I've documented the endpoints and clusters that deCONZ reports for my devices. Some of these aren't (yet) recognised by deCONZ, I'll try and create a ZCLDB XML source for these.

That's an impressive list, the biggest help would be if you can test sensors, switches and rules with your homebridge plugin and our next deCONZ beta release. As mentioned earlier the new WebSocket should provide near real time feedback for device events. We have also improved the ZCLDB XML file with missing descriptions for some devices but are keen to merge if you can provide further additions.

Sorry, I meant the duration attribute in the ZHAPresence sensor's config. Hoping this would indicate how long deCONZ reports presence as true, after motion was detected by the Hue motion sensor.

Right now deCONZ only forwards the value which is reported by the sensor and it's up to the application to implement the desired 'on' period after the impulse is received. I think in combination with the continuously updated lastupdated attribute (even if presence stays on do more received signals) this should be sufficient?

The Hue motion sensor provides this (and the client Level Control, Color Control, Groups, Scenes, Identify). Does this mean that (theoretically) it could work without a Hue bridge, interacting directly with the lamps/groups, like the Hue dimmer switch?

Yes the sensor can be used without bridge, you need to create a ZigBee binding between OnOff Cluster 0x0006 and the targeted group in the deCONZ Bind Dropbox. However I couldn't figure out to set the ontime which normally is achieved by setting the 0x0406 / PIR occupied to unoccupied delay attribute.

[ebaauw]

That's an impressive list, the biggest help would be if you can test sensors, switches and rules with your homebridge plugin and our next deCONZ beta release.

Happy to do so. Currently, my homebridge plugin does an unauthenticated GET /api/config to confirm the thing it discovered is indeed a Hue bridge, but deCONZ doesn't support that call. I'll try an create a pull request for that - probably easier than adding a new device.

We have also improved the ZCLDB XML file with missing descriptions for some devices but are keen to merge if you can provide further additions.

I need to figure out how to read them from the devices. Is there a debug switch that would make deCONZ dump the configuration of the devices it discovers? Or would I need to create something like an inspector plugin?

it's up to the application to implement the desired 'on' period after the impulse is received. I think in combination with the continuously updated lastupdated attribute (even if presence stays on do more received signals) this should be sufficient?

The Hue bridge supports a ddx operator in rules conditions, that fires the rule a specified time after an attribute has changed. The Hue app uses this for delayed turn off of the lights.

However I couldn't figure out to set the ontime which normally is achieved by setting the 0x0406 / PIR occupied to unoccupied delay attribute.

The Hue motion sensor supports the PIR Occupied to Unoccupied Delay attribute. When I try to write to it, I get an error. The new value seems to be written to the motion sensor anyway. And it seems to be working: the sensor continues to report Occupancy for the specified number of seconds. I've only been able to change the PIR Occupied to Unoccupied Delay value once, however, maybe I need to reset the motion sensor before it accepts a new value.

Yes the sensor can be used without bridge, you need to create a ZigBee binding between OnOff Cluster 0x0006 and the targeted group in the deCONZ Bind Dropbox.

The motion sensor appears to have a binding to the Hue bridge. I changed the attribute reporting maxInterval setting for the Occupany attribute is this is reflected in the rate at which the Hue bridge changes the lastupdated attribute.

I cannot create any (unicast) bindings, though - deCONZ reports failed: timeout. I think I did manage to delete a binding from the motion sensor to the Hue bridge: the Hue doesn't update lastupdated for that cluster anymore (so I assume Philips setup unicast bindings from the sensor to the bridge). Is their a way to view or query the bindings? Could the Hue motion sensor be limited to a single binding?

[ebaauw]

I've only been able to change the PIR Occupied to Unoccupied Delay value once, however, maybe I need to reset the motion sensor before it accepts a new value.

I could change them again, just now, without resetting the motion sensor. Maybe it just needs some time, maybe my ZigBee network is overloaded (I'm still running the RaspBee as a router on the Hue bridge network).

the Hue doesn't update lastupdated for that cluster anymore

Now it does, maybe I'm mistaken or maybe the Hue bridge re-establishes the binding. I definitely need to do some more testing...

[manup]

Hi, you can now try the new beta release, the sensor should now work similar to the original implementation:

https://github.com/dresden-elektronik/deconz-rest-plugin/releases

[ebaauw]

Thanks, Manual!

I got the beta running (still using the RaspBee as router on the network created by my Philips Hue bridge). First impressions:

• When starting deCONZ, it gives a messge:

  libEGL warning: DRI2: failed to authenticate

• While interacting with the GUI, deCONZ crashed once, with a segmentation fault. No additional message, no core dump.
• The rest API (as well as the Wireless Light Control web app) is now at port 8080 instead of port 80. Is there a way to reset them back to port 80? [Edit after rebooting the pi, port 80 is used again?]
• The Node List in the deCONZ GUI seems to be populated from the database instead of from the discovery: Name, Model, Vendor, Version are populated in the Node List, but not in the Basic Cluster attributes. I need to read the cluster attributes manually before the REST API creates the corresponding resources.
• The Hue Dimmer switches now have a state.buttonevent attribute, but the value is -1 and doesn't change when pressing a button. Also it still has the wrong uniqueid: 00:17:88:xx:xx:xx:xx:xx-01-0006 instead of 00:17:88:xx:xx:xx:xx:xx:-02-fc00. The config.battery attribute is still missing as well.
• The RESP API now creates a ZHATemperature sensor resource for the Hue Motion sensor, in addition to the ZHALight, ZHAPresence, and ZHASwitch sensor resources.
• The state.lux attribute to the ZHALight resource is gone, but no state.lightlevel [EDIT This is due to the sensor not yet being discovered in full. After reading the Illuminance measurement cluster attributes and searching for new devices, state.lightlevel appears]. Incidently, the Hue bridge calls this type ZLLLightLevel, so I thing deCONZ should use ZHALightLevel.
• The ZHASwitch resource is still there, but now also with a state.buttonevent attribute. Same for the Hue bridge and innr xx 110 light ZHASwitch resources. The unqueid for the Hue bridge ZHASwitch is weird: 00:17:88:xx:xx:xx:xx:xx-40-1000 instead of 00:17:88:xx:xx:xx:xx:xx-40-0006.
• There's still several Motion sensor attributes missing in deCONZ when comparing with the Hue bridge: for the ZXXLightLevel resource: state.daylight,state.dark, config.tholddark, config.tholdoffset; for the ZXXPresence resource: config.sensitivity, config.sensitivitymax; for al three resources: config.battery, config.alert, config.ledindication, config.usertest.

[manup]

Thanks alot for the report!

When starting deCONZ, it gives a messge: libEGL warning: DRI2: failed to authenticate

I've notices that too, not sure if that's a problem, the warning appeared since the switch to Qt5.

While interacting with the GUI, deCONZ crashed once, with a segmentation fault. No additional message, no core dump.

Is the ulimit for core dumps enabled on your RPi? Would be good to look deeper in the issue.

The rest API (as well as the Wireless Light Control web app) is now at port 8080 instead of port 80. Is there a way to reset them back to port 80?

deCONZ always tries to use the privileged port 80, 8080 is a fallback if 80 is used or not enough rights are set. If you start deCONZ from start menu or via $ deCONZ-autostart.sh it will try to give the executable the related network rights via setcap tool.

The Hue Dimmer switches now have a state.buttonevent attribute, but the value is -1 and doesn't change when pressing a button. Also it still has the wrong uniqueid: 00:17:88:xx:xx:xx:xx:xx-01-0006 instead of 00:17:88:xx:xx:xx:xx:xx:-02-fc00. The config.battery attribute is still missing as well.

Meanwhile I've adjusted the -02-fc00 suffix to match the hue ones.

The buttonevent -1 and missing of battery is very likely because the dimmer switch was already be registered before the 2.04.40 update. The problem is that your switch is missing two bindings which are created then registering the hue dimmer switch during setup.

I'm working on a solution which corrects the missing bindings on the fly for already registered switches.

Meanwhile, as a workaround, you can delete and re-run the setup for the switch, or just remove the battery and put it in again, after a few secs the proper bindings should be created.

The state.lux attribute to the ZHALight resource is gone, but no state.lightlevel [EDIT This is due to the sensor not yet being discovered in full. After reading the Illuminance measurement cluster attributes and searching for new devices, state.lightlevel appears]. Incidently, the Hue bridge calls this type ZLLLightLevel, so I thing deCONZ should use ZHALightLevel.

Same here, it seems that the bindings are not yet created, because the sensor was registered before 2.04.40 version. As workaround putting the battery out-and-in again should fix it till I figure out how to correct it on the fly, it's a bit more tricky than for the switch.

The ZHALight type is used for light sensors in some commercial projects, changing it would break some apps which are not under our control, so we decided to keep it as ZHALight.

The ZHASwitch resource is still there, but now also with a state.buttonevent attribute. Same for the Hue bridge and innr xx 110 light ZHASwitch resources. The unqueid for the Hue bridge ZHASwitch is weird: 00:17:88:xx:xx:xx:xx:xx-40-1000 instead of 00:17:88:xx:xx:xx:xx:xx-40-0006.

Ghosts ;) version 2.04.40 prevents the creation of those since there is now a white list for supported sensor creation. The hue motion sensor ZHASwitch will be discarded in the next update but the other ones need to be deleted manually for now.

There's still several Motion sensor attributes missing in deCONZ when comparing with the Hue bridge: for the ZXXLightLevel resource: state.daylight,state.dark, config.tholddark, config.tholdoffset; for the ZXXPresence resource: config.sensitivity, config.sensitivitymax; for al three resources: config.battery, config.alert, config.ledindication, config.usertest.

config.battery should appear in all sensors after the bindings are created. The ZXXLightLevel state.daylight, state.dark, config.tholddark and config.tholdoffset attributes might be implemented in future but personally I think this should be handled in the app, consider two apps with different opinions about these values.

The ZXXPresence attributes need some more ZigBee sniffing on how the hue bridge configures the sensor, these are mostly proprietary ZigBee commands.

[ebaauw]

Is the ulimit for core dumps enabled on your RPi? Would be good to look deeper in the issue.

ulimit reports unlimited under user pi (which runs deCONZ). Do I need to set something else? EDIT This was a bit tricky, but Google, man bash and find / limit.conf helped me find /etc/security/limits.conf. Now ulimit -c is also unlimited for user pi.

Ghosts ;)

Is there a way to clean the database and do a rediscovery from scratch? While we're at this, is there a way to hack the database, so I can assign the same numbers to the lights, sensors, etc as on my Hue bridge?

I plan to import my entire Hue bridge configuration to deCONZ and document all incompatibilities I encounter. I've started a Wiki page for that - I don't think listing every deviation as an issue would be manageable. Please let me know if that works for you or if you'd prefer issues.

The ZXXPresence attributes need some more ZigBee sniffing on how the hue bridge configures the sensor, these are mostly proprietary ZigBee commands.

Yeah, I couldn't find an attribute for config.sensitivity or config.sensitivitymax. I wander if they are implemented in the Hue bridge, as state.dark and config.tholddark for the Light sensor. On what attribute in what cluster do you base the buttonevent for the Dimmer switch? One of my reasons to buy the RaspBee was to do some ZigBee sniffing, but that seems a lot more work than I had hoped - needing a different firmware on the RaspBee, if I understand correctly.

Same here, it seems that the bindings are not yet created

Is there a way to view the bindings? Incidentally, Hue lights don't support attribute reporting. Will deCONZ poll these like the Hue bridge does?

[manup]

Is there a way to clean the database and do a rediscovery from scratch? While we're at this, is there a way to hack the database, so I can assign the same numbers to the lights, sensors, etc as on my Hue bridge?

Yes since the database is just a sqlite file, it can be cleaned by deleting it — it's recommended to make a backup first.

~/.local/share/data/dresden-elektronik/deCONZ/zll.db

For editing the file you can use tools like sqliteman which provides a nice GUI. Be aware that while editing or deleting the database deCONZ shall not be running.

I plan to import my entire Hue bridge configuration to deCONZ and document all incompatibilities I encounter. I've started a Wiki page for that - I don't think listing every deviation as an issue would be manageable. Please let me know if that works for you or if you'd prefer issues.

A Wiki for documenting differences is fine. I would suggest a column for classification, since some things wont be changed or treated as bug but as variation. For example the order of keys in a object should never be taken as granted since order is arbitrary in JSON and every parser and serializer might handle it differently. Thats an example of variation, which should have zero impact on applications.

Yeah, I couldn't find an attribute for config.sensitivity or config.sensitivitymax. I wander if they are implemented in the Hue bridge, as state.dark and config.tholddark for the Light sensor.

These aren't ZigBee attributes I assume Philips implements them as manufacturer specific extensions (which is allowed by ZigBee). For that the manufacturer specific clusters (>= 0xFC00) are used.

On what attribute in what cluster do you base the buttonevent for the Dimmer switch?

In fact the hue dimmer switch does the same to provide the buttonevents to the gateway. There is no attribute providing the data but when a binding between 0xFC00 cluster and gateway endpoint exists, the switch will fire commands which provides the info. The deCONZ version 2.04.40 already takes advantage of it.

One of my reasons to buy the RaspBee was to do some ZigBee sniffing, but that seems a lot more work than I had hoped - needing a different firmware on the RaspBee, if I understand correctly.

Yes the RaspBee is not made for sniffing in first place, for that the USB dongle ConBee is more appropriate, it can be used on desktop PC with the BitCatcher sniffer and soon Wireshark.

There are also other dongle solutions out there but I have no experience with them.

Is there a way to view the bindings? Incidentally, Hue lights don't support attribute reporting. Will deCONZ poll these like the Hue bridge does?

There is no GUI to view bindings yet. Only sniffer output and debug print might give some insight for example starting deCONZ with --dbg-info=1

Most lights don't support reporting, deCONZ will poll these from time to time, it's not that fast and will improve a lot in the next releases. Beside polling deCONZ tries to be clever and extracts on/off, brightness and scene calls from commands send by switches and sensors, and applies them to local lights attributes.

[ebaauw]

Thanks, once again!

I'm still struggling with getting a core file for the segmentation fault. ulimit -a for user pi returns:

core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 5813
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65536
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 95
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 5813
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

but still no core file. I'm starting deCONZ manually from the command line as user pi in /home/pi, which is writeable.

[manup]

I have not tried it yet but maybe the following guide helps:

https://pve.proxmox.com/wiki/Enable_Core_Dump_systemd

[ebaauw]

I have not tried it yet but maybe the following guide helps

It does. I now get a core dump when I kill the process. Now waiting for the next crash...

Yes since the database is just a sqlite file, it can be cleaned by deleting it — it's recommended to make a backup first.

Cool. I've also found the config for the deCONZ GUI, so I can change the name of the devices there as well.

For editing the file you can use tools like sqliteman which provides a nice GUI.

Brilliant. Had to compile it myself, but it's working.

I deleted the entire ~/.local/share/data/dresden-elektronik/deCONZ directory, unplugged my Hue bridge from power, and power-cycled the RasPi in an attempt to get rid of any ghosts. Once again, the deCONZ GUI is happily discovering my lights, Hue motion sensors, and Hue dimmer switches. The REST API is discovering the lights, groups, and scenes automatically, but no resources are being created for the Hue motion sensors and Hue dimmer switches. Before these resources where created automatically, without any search for devices.

[ebaauw]

In the deCONZ GUI, the Model, Vendor, and Version of the Hue motion sensor remain blank in the Node List, even after I manually read the attributes of the Basic cluster. Before, these fields would be populated, after which the REST API would create the corresponding resources. Also, the deCONZ GUI no longer shows a battery symbol on the motion sensors and dimmer switches.

In session.default in ~/.local/share/dresden-elektronik/deCONZ (the directory got recreated at a different path?) the motion sensor is listed like (I edited the name and masked the MAC address):

  {
    "ExtAddress": "0x0017880102xxxxxx",
    "NodeDescriptor": "AkCACxBZPwAAAD8AAA==",
    "NwkAddress": "0x5717",
    "SceneX": 782.0,
    "SceneY": -608.0,
    "SimpleDescriptors": [
      "AgQBBwEABgAAAQADAAYEAAQCBAEZAA==",
      "AV7AUAgCAQAABwAAAwAEAAYACAAAAwUA"
    ],
    "UserDescriptor": "Living Room Motion"
  }

I assume the SimpleDesriptors are the endpoint fingerprints?

For the dimmer switch, it looks like:

  {
    "ExtAddress": "0x0017880110xxxxxx",
    "NwkAddress": "0x7a8a",
    "SceneX": 781.0,
    "SceneY": -557.0,
    "UserDescriptor": "Living Room Dimmer"
  }

For two of my innr UC 110 lights, deCONZ doesn't read the Simple Descriptor(s). If I select them in the GUI, the checkmark disappears. I cannot select the endpoint dropdown for these lights, and they don't appear in the REST API. Technically, these lights use the same ZigBee module as the PL 110, which is recognised. I'll try and reset the UC 100s and see if that helps...

In session.default these lights show as:

 {
    "ExtAddress": "0x00158d0000xxxxxx",
    "NodeDescriptor": "AUCOZhF/ZAAAAGQAAA==",
    "NwkAddress": "0x7fef",
    "SceneX": 606.0,
    "SceneY": -64.5858,
    "UserDescriptor": "Kitchen Sink Left"
  }

The PL 110 shows as:

  {
    "ExtAddress": "0x00158d0000d041b2",
    "NodeDescriptor": "AUCOZhF/ZAAAAGQAAA==",
    "NwkAddress": "0x0031",
    "SceneX": 313.995,
    "SceneY": 574.917,
    "SimpleDescriptors": [
      "AV7AAAECCAAABAADAAYACAAFAP//GQAHAAAEAAMABgAIAAUAGQA="
    ],
    "UserDescriptor": "Bedroom 4"
  }

[manup]

In the deCONZ GUI, the Model, Vendor, and Version of the Hue motion sensor remain blank in the Node List, even after I manually read the attributes of the Basic cluster. Before, these fields would be populated, after which the REST API would create the corresponding resources. Also, the deCONZ GUI no longer shows a battery symbol on the motion sensors and dimmer switches.

Yes that's expected behaviour, since the entries are no longer in the database, the device is not known to the rest plugin. Albeit it is still in the network and hence visible in deCONZ GUI.

The resources will be recreated when you add the sensor again via find sensors REST API. While search is active you will have to either power-cycle the sensor (remove battery for a few seconds) or reset it in order to get the motion sensor discovered.

Note: Sensors and switches will only be created while search mode is active.

Also, the deCONZ GUI no longer shows a battery symbol on the motion sensors and dimmer switches.

Yes, that's because the ZDP power descriptor will no longer be queried in latest releases. The symbol will be refactored to show battery status of the ZCL Power cluster.

In session.default in ~/.local/share/dresden-elektronik/deCONZ (the directory got recreated at a different path?) Yes new installations always will use this path, older installations used different paths which were created by different Qt versions, which was a pain in the neck to discover :-) deCONZ takes care of it and will either use an existing older path or create a new one as above.

For two of my innr UC 110 lights, deCONZ doesn't read the Simple Descriptor(s). If I select them in the GUI, the checkmark disappears.

Normally these is done automatically, you can try to power-cycle the lights. Before that you might select the light node in deCONZ and press delete key, it should then reappear after power cycle.

[ebaauw]

The resources will be recreated when you add the sensor again via find sensors REST API. While search is active you will have to either power-cycle the sensor (remove battery for a few seconds) or reset it in order to get the motion sensor discovered.

OK, I got one of my dimmer switches recognised by the REST API. Pressing the four buttons simultaneously when search is active was enough. The sensor resource was created right away, but it took a while for the state to be populated. It now shows buttonevent and lastupdated, but these attributes aren't updated on pressing any of the dimmer button. Also, the dimmer switch's led turns red, typically indicating that it cannot talk to the bridge.

I tried to create a binding between the dimmer and the RaspBee, but I'm not sure I fully understand how that's done. I dragged the dimmer's On/Off cluster (0x0006) from the ZLL endpoint (0x01) to the Source box. Then I dragged the RaspBee's ZHA endpoint (0x01) to the Destination box. However, on Bind I get a failed: not supported message. Do I need to edit RaspBee's endpoint first and add a 0x0006 In cluster?

[ebaauw]

Device discovery remains a bit of a mystery to me...

I reset one of the dimmer switches (putting a pin in the back for 10 seconds) while deCONZ was searching for new devices. This switch was subsequently found by the REST API, and apparently it setup the bindings to the RaspBee. state.buttonevent and state.lastupdated are now updated when I press a button, and it even also shows config.battery. The Hue bridge is no longer updated on a button press. Apparently the switch can only handle one binding at a time?

While trying to attach the dimmer, two of my motion sensors where found spontaneously by deCONZ. When I read the attributes of the 0406, 0402, and 0400 clusters in the GUI, the state of the corresponding resource is updated. The sensor is still bound to the Hue bridge though: periodic updates only appear there. Also, config.battery is missing. I'll try and reset one of the motion sensors tomorrow.

The value of the sensorstate.lastupdated differs from the Hue bridge:

• The Hue bridge reports the UTC time; deCONZ reports the local time;
• The Hue bridge reports none when there is no value; deCONZ reports 1970-01-01T01:00:00.

Edit I'll leave the network open tonight (found /config/networkopenduration) and see if deCONZ finds my other motion sensors as well.

[manup]

The Hue bridge is no longer updated on a button press. Apparently the switch can only handle one binding at a time?

The relevant Binding here is between cluster 0xfc00 and the gateway endpoint, it might be that the dimmer switch only support one binding for this cluster. The thing is, after the reset the hue bridge binding was removed too, you can try and re-create it manually between hue bridge and dimmer switch cluster 0xfc00 as source.

The value of the sensorstate.lastupdated differs from the Hue bridge:

• The Hue bridge reports the UTC time; deCONZ reports the local time;
• The Hue bridge reports none when there is no value; deCONZ reports 1970-01-01T01:00:00.

Good catch, I put it on the list, should be easy to adjust.

[ebaauw]

For example the order of keys in a object should never be taken as granted since order is arbitrary in JSON and every parser and serializer might handle it differently. Thats an example of variation, which should have zero impact on applications.

You're right, of course. One of the charms of json over XML is that json is also human readable (sort of). Some humans like to have order, especially those, like myself, bordering on OCD ;-).
I've added a sort option to my own little json parser/formatter (which, of course, preserves the order) for easier comparison of the deCONZ output vs the Hue bridge output. Not being able to compare the output key by key, I actually missed the fact that deCONZ uses manufacturer whereas Hue uses manufacturername.

[ebaauw]

National holiday (King's birthday), so time for some serious testing...

Powered off the Hue bridge, started deCONZ, reset one of the motion sensors, opened the network through Wireless Light Control. The motion sensor is found and corresponding sensor resources are created (after hacking the SQL database to get the sensor numbers I want):

GET /sensors/20:

{
  "config": {
    "battery": 100,
    "on": true,
    "reachable": true
  },
  "ep": 2,
  "etag": "c27c18ccb78b6936155abe9049ecd292",
  "manufacturername": "Philips",
  "modelid": "SML001",
  "name": "Living Room Motion",
  "state": {
    "lastupdated": "2017-04-27T11:25:57",
    "presence": false
  },
  "swversion": "6.1.0.18912",
  "type": "ZHAPresence",
  "uniqueid": "00:17:88:01:02:xx:xx:xx-02-0406"
}

GET /sensors/30:

{
  "config": {
    "battery": 100,
    "on": true,
    "reachable": true
  },
  "ep": 2,
  "etag": "ad24a0060856b5fea7e379cf2582e066",
  "manufacturername": "Philips",
  "modelid": "SML001",
  "name": "Living Room Light Level",
  "state": {
    "lastupdated": "2017-04-27T11:24:47",
    "lightlevel": 29360
  },
  "swversion": "6.1.0.18912",
  "type": "ZHALight",
  "uniqueid": "00:17:88:01:02:xx:xx:xx-02-0400"
}

GET /sensors/40:

{
  "config": {
    "battery": 100,
    "on": true,
    "reachable": true
  },
  "ep": 2,
  "etag": "4b579f11fd08e1d3b4244421e2fe0e01",
  "manufacturername": "Philips",
  "modelid": "SML001",
  "name": "Living Room Temperature",
  "state": {
    "lastupdated": "2017-04-27T11:27:55",
    "temperature": 2135
  },
  "swversion": "6.1.0.18912",
  "type": "ZHATemperature",
  "uniqueid": "00:17:88:01:02:xx:xx:xx-02-0402"
}

Looking good! For all sensor resources, the state is updated periodically and when I manually refresh the corresponding cluster in the deCONZ, looking better! I receive events through WebSocket for all state updates, looking almost perfect!

Thu Apr 27 2017 11:19:05: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":18478},"t":"event"}
Thu Apr 27 2017 11:19:10: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":20833},"t":"event"}
Thu Apr 27 2017 11:19:39: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:19:39: {"e":"changed","id":"20","r":"sensors","state":{"presence":true},"t":"event"}
Thu Apr 27 2017 11:19:44: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":26230},"t":"event"}
Thu Apr 27 2017 11:19:49: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":29374},"t":"event"}
Thu Apr 27 2017 11:19:57: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:20:14: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:20:14: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:20:22: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":29360},"t":"event"}
Thu Apr 27 2017 11:20:23: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":29360},"t":"event"}
Thu Apr 27 2017 11:20:28: {"e":"changed","id":"40","r":"sensors","state":{"temperature":2107},"t":"event"}
Thu Apr 27 2017 11:20:38: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:21:12: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":29360},"t":"event"}
Thu Apr 27 2017 11:21:41: {"e":"changed","id":"40","r":"sensors","state":{"temperature":2107},"t":"event"}
Thu Apr 27 2017 11:22:56: {"e":"changed","id":"40","r":"sensors","state":{"temperature":2107},"t":"event"}
Thu Apr 27 2017 11:22:56: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:24:47: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":29360},"t":"event"}
Thu Apr 27 2017 11:24:56: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:25:57: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:27:53: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:27:55: {"e":"changed","id":"40","r":"sensors","state":{"temperature":2135},"t":"event"}
Thu Apr 27 2017 11:29:47: {"e":"changed","id":"30","r":"sensors","state":{"lightlevel":29231},"t":"event"}
Thu Apr 27 2017 11:29:55: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:30:16: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:31:00: {"e":"changed","id":"29","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:32:37: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}
Thu Apr 27 2017 11:32:54: {"e":"changed","id":"40","r":"sensors","state":{"temperature":2163},"t":"event"}
Thu Apr 27 2017 11:33:20: {"e":"changed","id":"20","r":"sensors","state":{"presence":true},"t":"event"}
Thu Apr 27 2017 11:33:30: {"e":"changed","id":"20","r":"sensors","state":{"presence":false},"t":"event"}

Powered on the Hue bridge and searched for new devices. It seems to reconnect to the motion sensor - the state attributes are updated and config.reachable becomes true. However, after the initial setting of the state, it not updated anymore. The sensor clusters remain bound to the RaspBee and the REST API resources continue to be updated automatically.

Now the funny part: the ZHAPresence sensor resource of one of the other motion sensors (which was previously discovered by the REST API) gets updated periodically in deCONZ, as if its still bound to the RaspBee, at least for periodic attribute reporting. Indeed, the state on the Hue bridge doesn't get updated periodically. However, when I trigger the motion sensor, the Hue bridge state is updated and the REST API state isn't. I don't understand how that's possible - I thought attribute reporting is configured per cluster/attribute not per binding? Or is the REST API somehow polling the other motion sensor?

Restarted deCONZ. No change in behaviour for the Living Room motion sensor, good! The other motion sensor's state.lastupdated is no longer updated periodically in the REST API. The Hue bridge state still isn't updated periodically either, and when triggering the sensor, only the Hue bridge state is updated.

Shutdown en powered off the Raspberry Pi (to reset the RaspBee board). deCONZ is started automatically after booting up the Raspberry Pi, but the RaspBee is not in the Network. It has lost the ZLL Link Key, but the other network parameters (PAN ID, channel, NWK address) have been saved to non-volatile memory. Re-entered the ZLL Link Key, and joined the network. No change in behaviour of the motion sensors, good!

(to be continued in next post)

[ebaauw]

(continued from previous post)

Now, for the bindings... Restarted deCONZ --dbg-info=1. Indeed, that's a whole lot of messages. It looks like the REST API sets up attribute reporting bindings for Living Room motion sensor for clusters 0x0001, 0x0400, 0x0402, and 0x0406.

In the deCONZ GUI, opened the Bind Dropbox. Dragged the Power Configuration cluster (0x001) from the Kitchen motion sensor to the Source, and the HA endpoint 0x01 from the RaspBee to the Destination. Leave the Binder at the sensor and press Bind... failed: timeout, and later a failed: not supported. Bummer.

However, WebSocket all of a sudden sends two events for Kitchen motion sensor ZHAPresence resource. And state.lastupdated has been updated. Let's try the other clusters. Occupancy sensing (0x0406): same errors. Illuminance measurement (0x0400): same errors. Temperature measurement (0x0402): same errors. The presence sensor state.lastupdated is still updated periodically, but the light level and temperature sensors remain unchanged. Triggering the sensor is reflected in the Hue bridge, not in the REST API.

This does lead to some new insight: the changed event of the other motion sensor is probably caused by of the 0x0001 cluster attribute reporting - not by the 0x0406 attribute reporting. That would also explain why I see two events per 5 minutes for the presence sensor and only one event per 5 minutes for the light level and temperature sensors (all clusters have a maximum reporting value of 300).

Updated the Kitchen motion attributes from the deCONZ GUI. As expected, the REST API states are updated and WebSockets issues events. Still no luck in binding, even though the deCONZ debug output seems happy:

13:36:23:705 Force read attributes for SensorNode Kitchen Temperature
13:36:23:705 0x0017880102xxxxxx (SML001) create binding for attribute reporting of cluster 0x0001 on endpoint 0x02
13:36:23:705 0x0017880102xxxxxx (SML001) create binding for attribute reporting of cluster 0x0402 on endpoint 0x02
13:36:23:705 Force binding of attribute reporting for node Kitchen Temperature
  ...
13:36:25:290 Bind response success
13:36:25:307 Bind response success

Let's try and unbind the temperature sensor from the Hue bridge: ... failed: no entry ... success. And then bind it to the RaspBee: ... success. That looks promising. The other clusters seem to like this approach as well. And.. BINGO the attributes are updated periodically on the REST API and when triggering the sensor. The Hue bridge attributes are not longer updated.

I see no messages in the deCONZ output for any of the GUI binding. Apparently the above messages were caused by the GUI discovering the motion sensor.

Conclusion: The Hue motion sensor only allows one binding per cluster at a time.

Next steps:

• Try and automate the binding/unbinding through the RESP API rules, so I can switch my sensors between the Hue bridge and the deCONZ rest API;
• Repeat this test for the Hue dimmer switch.

[ebaauw]

Repeat this test for the Hue dimmer switch.

Seems to work similarly, except that the communication between deCONZ (GUI as well as REST API) and the Hue dimmer switch seems more flaky. Lots of timeouts, or simply no reply at all. In the debug log, I only see messages for binding the 0x0001 cluster of the dimmer switch. I know that 0xFC00 needs to be bound as well, but no message? @manup, do you know if 0x000F needs to be bound as well?

The WebSocket picks up the button events brilliantly. When I press (and release) the on Button, I get exactly what I expected:

Thu Apr 27 2017 16:07:29: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1000},"t":"event"}
Thu Apr 27 2017 16:07:29: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1002},"t":"event"}

And when I press and hold, I get:

Thu Apr 27 2017 16:07:31: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1000},"t":"event"}
Thu Apr 27 2017 16:07:32: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1001},"t":"event"}
Thu Apr 27 2017 16:07:35: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1001},"t":"event"}
Thu Apr 27 2017 16:07:35: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1001},"t":"event"}
Thu Apr 27 2017 16:07:36: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1001},"t":"event"}
Thu Apr 27 2017 16:07:36: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1001},"t":"event"}
Thu Apr 27 2017 16:07:37: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1001},"t":"event"}
Thu Apr 27 2017 16:07:38: {"e":"changed","id":"55","r":"sensors","state":{"buttonevent":1003},"t":"event"}

Functionally this is correct, but I'm not too happy about the 3-second delay between the first and second 1001 event. It only happened this time, though.

Occasionally, the WebSocket events seem to go wrong - e.g. I see two 2002 events in a row, with a missing 2000 event. Not sure if this is a communication glitch or a deCONZ REST API bug.

One of my dimmers is bound to the RaspBee, but when I hold the On button, it goes into touchlink modus and linked with the two nearest lights. I'm still receiving the buttonevents in WebSocket, and the dimmer controls the lights directly at the same time. Apparently there's more to commissioning the dimmer than setting up attribute reporting - it needs to be told not to go into touchlink mode.

Try and automate the binding/unbinding through the RESP API rules

That's not going to work, of course. The REST API only supports bindings for the On/Off, Level and Scene clusters.

After unbinding the sensor or switch from the Hue bridge, deCONZ will create the bindings to the RaspBee automatically after a restart. Still need to test if the Hue bridge will do so as well

[sfeigl]

One of my dimmers is bound to the RaspBee, but when I hold the On button, it goes into touchlink modus and linked with the two nearest lights. I'm still receiving the buttonevents in WebSocket, and the dimmer controls the lights directly at the same time. Apparently there's more to commissioning the dimmer than setting up attribute reporting - it needs to be told not to go into touchlink mode.

I noticed this too. Some weeks ago I sniffed a pairing of a Hue dimmer switch with a Hue bridge. (using BitCatcher with Conbee).

There i noticed a "write attribute" from bridge to dimmer switch. If I read the packet correctly , it is sent to cluster 0. Attribute Id seems to be 0x31, Datatype 0x19 and data "0x08 0x00"

I did not try to write this attribute through deCONZ though. Might this be the missing bit to setup the switch?

[manup]

In my tests the dimmer switch works well after a clean setup which involves:

• create binding between power configuration cluster and gateway (to start battery attribute reporting)
• configure the power configuration attribute reporting for the related attribute
• create binding between 0xfc00 cluster and gateway (custom Philips cluster which sends button events)

deCONZ REST plugin does all this automatically since version 2.04.40 when the switch is newly installed and joins the network for the first time. Meaning the switch is deleted from database (DELETE REST method) and is in factory new state (via setup button on the back).

I've worked on some improvements to also configure switches which are already part of the network. The extra attribute writes are not needed for proper operation, still I don't know what they are for. The changes will be available in a few days.

However, I really don't think it's possible to get the switch work smoothly with multiple gateways at the same time. Switching the configuration over between gateways on the fly won't work either, because Philips hue bridge as well as RaspBee gateway assume to configure the switch in the initial setup phase (which makes sense). For lights it's much easier since they are always listing to RF signals but battery powered devices are sleeping most of the time after setup phase — reconfiguring after that will most likely fail or be error prone.

[ebaauw]

Thanks for the info, @manup. I found the same procedure is needed to reconnect the dimmer to the Hue bridge: delete the sensor resource, reset the dimmer and connect it as a new device. Probably the same for the motion sensor as well.

However, I really don't think it's possible to get the switch work smoothly with multiple gateways at the same time.

I would only use this while transitioning from the Hue bridge to deCONZ. My ultimate goal is still to ditch the Hue bridge, once I've moved all its functionality to deCONZ (including the homebridge plugin). Testing will be more elaborate (resetting the sensors and dimmers) or more expensive (getting an additional sensor or dimmer), but it's not the end of world.

The changes will be available in a few days.

Thanks, I'm looking forward to that!

[ebaauw]

Yes the RaspBee is not made for sniffing in first place, for that the USB dongle ConBee is more appropriate, it can be used on desktop PC with the BitCatcher sniffer and soon Wireshark.

OK, I got myself a ConBee. I connected it to my Raspberry Pi and, after removing the RaspBee, deCONZ found it and updated its firmware. I no longer have a PC (I moved to Mac in 2009) and I only have one Raspberry Pi. When both the RaspBee and ConBee are connected to it, GCFFlasher_internal -l only shows the RaspBee - I don't suppose you can use the RaspBee and ConBee concurrently on the same Raspberry Pi?

I managed to run deCONZ on my MacBook Pro, in a Windows 10 virtual machine under Parallels. The ConBee didn't seem to communicate with the ZigBee network at first, when connected through a USB hub (or maybe two USB hubs in series), but it seemed happy when connected to a USB port on the MacBook. I've found earlier (before abandoning the PC) that Parallels does a brilliant job passing through all sorts of obscure USB devices to Windows VMs.
By the way, I prefer the Windows GUI, with the curved lines, over the Raspbian GUI. It's also way more responsive than using VNC to the Raspberry Pi.

macOS recognises the ConBee as a FT230X Basic UART modem. Any chance of compiling and running deCONZ natively on macOS? Qt5 seems to be supported on macOS...

What's the status on Wireshark? I'm running it regularly on my MacBook for IP sniffing. Looking under Preferences, Protocols, it already knows IEEE 802.15.4, ZigBee Network Layer and ZigBee Green Power Profile. Would I need to configure the ZigBee network key? How do I retrieve that from deCONZ? How would I add the ConBee as capture interface?

The link to download BitCatcher in the ConBee User Manual is broken. I did find a Java-based, v1.0.1 from Aug 14, 2013. In the README, it mentions JRE 1.6 and Windows XP (or newer) or Linux. What would you recommend? I still have an old VM image of WinXP somewhere, which no longer connects to the Internet since Microsoft ended WinXP support. I don't think I have a Windows 7 key, I stayed on XP for as long as it was supported and then moved to Windows 8.1.

[manup]

I don't suppose you can use the RaspBee and ConBee concurrently on the same Raspberry Pi?

deCONZ will very likely pick the RaspBee for connection. There is not yet a good way to prevent that.

By the way, I prefer the Windows GUI, with the curved lines, over the Raspbian GUI. It's also way more responsive than using VNC to the Raspberry Pi.

The more simple interface on Raspberry Pi was introduced to let RPi 1 perform well, since RPi 3 gains more momentum I guess in mid future the interface will be adjusted to be on par with desktop.

The lag is caused by VNC, when the Pi is attached via HDMI it should perform better.

macOS recognises the ConBee as a FT230X Basic UART modem. Any chance of compiling and running deCONZ natively on macOS? Qt5 seems to be supported on macOS...

In fact there was a version for macOS (2.04.00) it was pulled back because it had some issues with GCFFlasher to update the firmware. We plan to re-release the mac version in the next stable release but there is no ETA available yet.

What's the status on Wireshark?

We work on this and have a internal prototype running, a first beta will be released within the next months. A Qt5 based helper tool will part of the release to select the channel and transfer sniffed traffic to Wireshark.

The link to download BitCatcher in the ConBee User Manual is broken.

I'll forward that to our QA, meanwhile the link here should work to download BitCatcher. V1.0.1 was the last BitCatcher release from Luxoft and seems to be longer maintained. That's also the reason to add support for Wireshark, in order to have a decent up-to-date sniffer, albeit BitCatcher is quite nice too.

https://www.dresden-elektronik.de/funktechnik/products/usb-radio-sticks/derfusb-analyzer-bitcatcher/?L=1

I've got it running in Windows 7 with latest official JRE and also in a Arch Linux with latest JRE.

[NathanSweet]

Does DeCONZ support the Philips Hue motion sensor? From the above it sounds like it does, but how do I add the motion sensor to DeCONZ? I am using 2.04.55.

[ebaauw]

Just reset the motion sensor by pushing the reset button on the back for 10 seconds - the led on the front will blink red. Then open the network in the deCONZ web app. The sensor stops blinking as it joins the network and it appears as a new node in the deCONZ GUI. After that, the REST API plugin should create three sensor resources (presence, lightlevel, and temperature). If you listen to the websocket, you should see notifications when these are created. Otherwise, check using the API (do a GET of /api//sensors).

If the resources aren't created automatically, double-check that the network is still open and in the deCONZ GUI read the attributes of the Basic cluster, making sure the manufacturer and model have been retrieved.

The motion sensor doens't show in the Web app, but if you open the Menu on the top right of the screen, a new option Daylight control appears after a view seconds. There you can configure the light sensor. To use the motion sensor, you have to create rules through the API.

[manup]

Whoopsi, the Daylight control was not really made for the Philips motion sensor and is very limited since it only sets light level based on lightlevel input. It wasn't ment to be visible and will be hidden again.

The new, not yet public, app is going to have proper sensor control editors (better name to be found..) for motion sensor and daylight brightness control. I will provide the early access to the app to interested users in the next days. There are still many todos in the new app but it is developed at a fast pace :)

[NathanSweet]

Thanks guys! It was very confusing since I was expecting to see something in the app rather than have the sensor automatically added to DeCONZ. I do see a new node in DeCONZ named ZHATemperature 4 as well as the Daylight Control webpage. I plan to respond to the sensor's presence events from the websocket and I am seeing those events, so I'm good to go now. Cheers!

[posixx]

Hello manup; the new, not yet public, app; do you refer to the web-based version? or the deCONZ interface on the RPi?

[manup]

Hello manup; the new, not yet public, app; do you refer to the web-based version? or the deCONZ interface on the RPi?

Hi, I mean the web-based version, deCONZ and REST-API are already available in the most recent version due the beta releases.

For the wep app it won't be a hard update, the old (current) web app will stay functional in parallel for a long time.

[donnib]

Hi, i am using master branch together with hue motion sensor and websockets but it sends out rarely events. How often should it send morion events, is there a grace period or ? Can i change it if there is one ?

[ebaauw]

You should get at least four events per 5 minutes: two presence, one lightlevel, and one temperature. Also, you should see events when significant changes happen (motion detected, more than x difference). This is configured by ZigBee attribute reporting on the motion sensor's battery, occupance, light measurement and temperature measurement clusters - you can change this in the deCONZ GUI. The motion sensor seems to take approx. 15 seconds to report no occupancy after it last detected motion. This can be prolonged by changing the corresponding attribute in the deCONZ GUI.

[NathanSweet]

@ebaauw I'm interested in making the Hue motion sensor more sensitive. Here's what I see in DeCONZ: The values are 0 already?

[ebaauw]

So attribute 0 indicates whether motion is detected. Of course, it's read-only. Attribute 10 allows you to set the duration, how long until the sensor reports attribute 0 as 0, after motion is no longer detected. Note that the Hue API doesn't expose this. There is a config.duration in the deCONZ API, but that isn't shown for the Hue motion sensor. @manup, could that be added? I was able to change the value from deCONZ after a couple of tries, and indeed, it worked. However, I could not set it back, until iI reset the sensor. Not sure if the reset was needed, or if I just needed to be more persistent in trying...

The Hue API has a config.sensitivity setting, to set the sensitivity of the motion sensor. I haven't been able to figure out to which ZigBee attribute that maps. I'm not even sure if that's implemented on the Hue bridge or on the sensor (using a custom attribute that the deCONZ GUI doesn't know?). As far as I can tell, it seems to revert to the most sensitive setting on sensor reset.

If you double-click on an attribute, you'll get a pop-up window to configure the attribute reporting, but that won't change the sensitivity (only the rate at which the battery depletes ;-)

You might want to play around with placing the sensor. They are more sensitive to lateral movement than to moving towards or away from the sensor.

[ebaauw]

Reading back the earlier part of this issue (it's been a while):

However I couldn't figure out to set the ontime which normally is achieved by setting the 0x0406 / PIR occupied to unoccupied delay attribute.

So Philips use this attribute differently. Guess there's no such thing as "normal" in the wonderful world of ZigBee.

[donnib]

So i am still confused, what does the lightlevel indicate ? Should/can i use it for something or is it just a different value than lux?

[ebaauw]

The Philips Hue API uses lightlevel, which is a logarithmic scale, see https://developers.meethue.com/documentation/supported-sensors, section 2.7. The deCONZ API used lux, until I complained - now it provides both.
I think the Hue motion sensor itself uses the same uint16 value as lightlevel. lux is computed by the API. deCONZ reports a (slightly) different value than my homebridge plugin (HomeKit wants lux) - I need to double-check if I screwed up reversing the formula given by Philips, or if deCONZ uses a different formula.

[donnib]

@ebaauw ok so i have made the formula which is : round(10 ** (float(lightlevel - 1) / 10000),1), it's not 100% the same as lux that deConz is reporting but i think it's ok as long it's close enough ;)

[ebaauw]

That's the same formula as I use: https://github.com/ebaauw/homebridge-hue/blob/master/lib/HueSensor.js#L32

deCONZ uses (10 ** (lightlevel /10000)) - 1 instead: https://github.com/dresden-elektronik/deconz-rest-plugin/blob/17998cd832bd0447250eca40e8f34e388fd1f08b/de_web_plugin.cpp#L2988. They do mention the correct formula from lux to lightlevel in the comment, though.

[dghh]

Hi, is the new web app already available for testing purposes ?

[manup]

Not yet we need to get a few rough edges smooth, it will be available on request in a few days.

[ebaauw]

We've been having a lot of useful discussions in this issue over the last couple of months. Thanks everybody!

I feel like closing this issue, since the Hue motion sensor, dimmer switch, and tap seem to be working well with the current deCONZ beta. However, I still have the following issues:

1. Support for state.dark, state.daylight, config.tholddark, and config.tholdoffset for the ZHALight resource. Not sure if these are actually implemented on the Hue motion sensor or solely on the Hue bridge. Could be worked around by using conditions on state.lightlevel instead, but I like to configure the thresholds explicitly rather than hide them in rule conditions;
2. Support for config.sensitivity and config.sensitivitymax for the ZHAPresence resource. Again, not sure if these are implemented on Hue motion sensor or solely the Hue bridge;
3. Support for config.duration for the ZHAPresence sensor, mapped to the PIR Occupied to Unoccupied Delay attribute of the Occupancy Sensing cluster. Note that this is not implemented on the Hue bridge;
4. The attribute reporting on the Battery Percentage Remaining attribute of the Power Configuration cluster results in additional state.presence events on the ZHAPresence resource;
5. The computation of state.lux from state.lightlevelfor the ZHALight sensor is wrong;
6. ZHALight should be called ZHALightLevel. Only an issue if, like me, you're still recovering from OCD ;-)

@manup, what's you take on these issues? Do you want to put them on the backlog, or should I just learn to live with them? Can I help with additional ZigBee sniffing for 1 and 2 or a PR for 5?