dresden-elektronik / phoscon-app-beta

Access to Phoscon app beta
78 stars 5 forks source link

Support for Aqara Smart Vibration Sensor DJT11LM #119

Closed Cavemanz closed 4 years ago

Cavemanz commented 5 years ago

This seems to be a new item just released. Model: DJT11LM. Any plans to get support for it in Deconz?

https://www.gearbest.com/smart-home-controls/pp_009661787808.html?wid=1433363 https://www.aliexpress.com/store/product/Xiaomi-Aqara-Smart-Motion-Sensor-International-Edition/329388_32912246474.html

https://www.youtube.com/watch?v=odnaG2I387U

jo-me commented 5 years ago

Cool, could be interesting for the classic washing machine use case.. Simpler as this: https://community.home-assistant.io/t/laundry-sensors-with-nodemcu-and-home-assistant/42533

sumo-89 commented 5 years ago

I've just ordered 3 of these, I also hope they will be supported in Deconz soon. :)

ebaauw commented 5 years ago

I would expect these to be IAS devices, just as the Xiaomi leak sensor. In that case, adding support should be straightforward. Please post the usual deCONZ GUI screenshots:

Please also try to capture the Xiaomi special packet for this device from the dCONZ log.

paolotremadio commented 5 years ago

I've got mine today. It's my first time on deCONZ so please help me out getting the details šŸ˜Š

Device:

3 4

Node info:

1 2

Cluster info (0000 Basic)

0000 basic

Cluster info (0101 Door lock)

0101 lock 1 0101 lock 2 0101 lock 3

Also, some pics of the device:

img_8486 img_8487

ebaauw commented 5 years ago

Door lock cluster ?! That doesn't make any sense to me, even for Xiaomi. Do any of the Door lock attributes change when the sensor detects vibration?

I wonder if deCONZ read it correctly. Could you please reset the device (hold the reset button/hole for 10 seconds), remove it from deCONZ, power down all your lights (ZigBee routers), move it close to the RaspBee/ConBee and re-pair it, while shaking it (to keep it awake)?

The leak sensor doesn't show an IAS Zone cluster either. I've copied the logic of that sensor to create a ZHAVibration sensor resource for the lumi.vibration.aq1, assuming a similar of Xiaomi magic packet.

paolotremadio commented 5 years ago

I can confirm. I've paired a brand new one, by standing next to the Conbee USB device. I have no lights on this network (totally separate from the Hue network I have, even the channel is different). The new device is still a door lock... how can I test your changes? Do I have to pull your branch and build?

ebaauw commented 5 years ago

I can confirm.

Thanks. I've seem some other strange, non-functional clusters reported by Xiaomi devices. Do you see any changes in the Door lock attributes when the sensor detects vibration?

Do I have to pull your branch and build?

Yes. Or wait for @manup to publish the next release. I see he just merged the open PRs.

paolotremadio commented 5 years ago

I'm tilting and shaking the sensor as the vibration detection might not be accurate (see https://community.home-assistant.io/t/new-aqara-vibration-shock-sensor/63211/8). The lock is in "not fully locked" and it does not change if I keep moving the sensor. Also, no changes in any of the other params.

manup commented 5 years ago

Latest PRs are now available in 2.05.37, however please make a backup and read the changelog before updating, the release has some larger changes under the hood, hopefully nothing will break.

https://github.com/dresden-elektronik/deconz-rest-plugin/releases

paolotremadio commented 5 years ago

Also, check this for reference too: https://github.com/Koenkk/zigbee2mqtt/issues/295

paolotremadio commented 5 years ago

I've deleted two comments. After upgrading the firmware too and deleting and repairing (several times), here's where i've got:

1 pairing after upgrading 2 after upgrading

I shake the sensor, the dot next to the sensor in the map on the GUI flashes blue but no changes in value. When I shake it I can see:

image

When I stop shaking I see: image

manup commented 5 years ago

Interesting, you may also try command line parameters --dbg-zcl=2 --dbg-aps=2 to get more details when receiving data.

paolotremadio commented 5 years ago

deconz.log

I've added some empty line before and after shaking the sensors (shaked first one sensor, added more empty lines, than the second one)

ebaauw commented 5 years ago

That means it does send attribute reports for the Door lock cluster (0x01). Currently deCONZ uses this cluster only for /lights resources (the door lock is exposed as a light), but it can be added to the /sensors logic as well.

According to the zigbee2mqtt thread, the sensor issues reports for attributes "85" (0x0055), "1283" (0x0503), and "1288" (0x0508) - that matches pl[0] of 55 and 8. I have no clue as to what these attributes mean, they're not defined for the Door lock cluster, but 0x0055 is Present value for many of the input and output clusters.
The Basic cluster report with pl[0] of 5 and 1 matches attributes 0x0005 (Model Identifier) and 0xff01 (Xiaomi special).
The Xiaomi special package also contains a lot of info. If I were a betting man, I'd place my money on tags 98, 99, and 9A. According to the gearbest page, the sensor detects vibration, tilt, and fall, so I'd expect different values for these three.

paolotremadio commented 5 years ago

@ebaauw here's the log after compiling and running your master branch (including f68d511):

new.log

I've tilted the sensor first, then pressed the button to capture the special packet

ebaauw commented 5 years ago

That doesn't look good - too many unsupported tag messages. I haven't actually compiled and tested the commit myself yet, still upgrading to v2.05.37 and new firmware.

The asdu message gives the raw package payload. Dissecting the magic package:

                                            1                       2                       3                      4
                     1 2  3 4  5 6  7  8 9  0 1  2 3  4 5  6 7  8 9 0 1 2  3 4  5 6  7 8  9 0  1 2  3 4  5 6  7 8  9 0  1 2 3 4 5 6
         attr tp  l   battery    temp         ?   RSSI dB             LQI         ?         ?         ?         ?                 ?
         ---- -- -- --------- ------- --------- --------- --------------- --------- --------- --------- --------- -----------------
         01ff 42 2e 0121 030c 0328 1f 0421 a813 0521 3b00 0624 0000000000 0821 0803 0a21 0000 9821 2800 9921 7400 9a25 8000fffbb300
vibration:       46    3.075V    31Ā°C  u16       u16       u40              u16      u16       u16       u16       u48
ebaauw commented 5 years ago

Ah there was an if in the original code which should have been an else if.

paolotremadio commented 5 years ago

new2.log Pulled the new commit, recompiled, updated firmware

ebaauw commented 5 years ago

I'll add the tag 0x0A. Next step is to figure out how these values change for the various actions: tilt, shake/vibrate, and drop.

Looking at the reports for the Door lock cluster, it reports attributes 0x0055 and 0x0503, both u16. I'll add them to general.xml, so you should be able to monitor them in the deCONZ GUI. You need to copy this file to /usr/share/deCONZ/zcl (or create a symlink to where you download the git repository). I'm guessing the 0x0508 attribute is u48, just like the 9A tag. They showed it as two numbers in zigbee2mqtt, I'm guessing u16 and u32 to make up u48. Would be nice if you manage to capture the asdu for that as well.

paolotremadio commented 5 years ago

I think i've cracked the meaning of 0x0055: Value "1": shake Value "2": tilt Value "3": drop

The value 0x0503 seems to be a number between 1 and 100 (but at some point I saw a value like 108, 169, 200 and 230). It doesn't seem to represent an angle. Might be the strength/intensity of the motion.

The value of 0x0508 is usually quite a long number (sometimes 20 digits like 18446744073707061293 or 18446744073708372031 but also 1507201 and 65339396).

The 0x0503 and 0x0508 might stay to the same value, even if the value of the 0x0055 changes.

ebaauw commented 5 years ago

I think i've cracked the meaning of 0x0055

Cool! I couldnā€™t have been more wrong assuming it would be an IAS zone device. Maybe best to expose the sensor as a switch, re-using the buttonevent values for shake, flip, and drop, from the cube.

Did you get the values from the deCONZ GUI or from the log? If from the log, could you post the asdu for the 0x0508?

20 digits is too long for a u48, more like a u64. In hex, the values are 0xFFFFFFFFFFDA002D and 0xFFFFFFFFFFEE003F.

paolotremadio commented 5 years ago

Cool! I couldnā€™t have been more wrong assuming it would be an IAS zone device. Maybe best to expose the sensor as a switch, re-using the buttonevent values for shake, flip, and drop, from the cube.

I think that is a good idea

Did you get the values from the deCONZ GUI or from the log? If from the log, could you post the asdu for the 0x0508?

From the UI. Is there a way to run deCONZ to log the output only of a single device? I usually get a stream of events and it's quite difficult to isolate the ones relating to the vibration sensor.

20 digits is too long for a u48, more like a u64. In hex, the values are 0xFFFFFFFFFFDA002D and 0xFFFFFFFFFFEE003F.

I'm really curious to find out what that number means :)

ebaauw commented 5 years ago

Is there a way to run deCONZ to log the output only of a single device?

Not really. I usually just grep -i for the mac address, but that doesnā€™t match the asdu nor Xiaomi special messages. It does provide the time of the preceding message, so I use that to search the log in an editor.

paolotremadio commented 5 years ago

Ok, will dump a larger set of logs on Wed once I'm back home.

ebaauw commented 5 years ago

@paolotremadio, my latest commit (https://github.com/ebaauw/deconz-rest-plugin/commit/bca9cf23fc8db1c7c69c68ec146b948daf385dc4) should support the lumi.vibration.aq1 as ZHASwitch sensor. The values for state.buttonevent are:

@manup, I kept the code for ZHAVibration and CLIPVibration, even though currently we don't support any _IASZone vibration sensors. I suppose it won't do any harm and it might come in handy later.

paolotremadio commented 5 years ago

I've been digging into GitHub and I've found this comment:

https://github.com/Danielhiversen/PyXiaomiGateway/issues/86#issuecomment-418135852

Also, here's a detail of the clusters (translated from French): https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Ffaire-ca-soi-meme.fr%2Fdomotique%2F2018%2F09%2F03%2Ftest-xiaomi-aqara-vibration-sensor%2F%23Compatibilite_ZiGate&edit-text=&act=url

To recap

0x0055 is clear: Value "1": shake / vibrate Value "2": tilt Value "3": drop / free fall

0x0503 should be the bed activity:

bed_activity: this reports a number. It seems to be related to vibration. It might correspond to how much movement was seen during the last reporting period. When sitting on my clothes dryer while it was running, it reported the same number for many reports which seem to come about one minute apart. The numbers reported from 72 to 214. Often the same number would report for multiple minutes. A 0 is never reported.

0x0508 is still a mystery. According to the french article:

This frame makes it possible to avoid flooding of actions. As long as this frame is not started, you can not start any other actions.

but according to the GitHub comment:

coordination: this is three numbers separated by commas. It is reported without any regularity. When in the same spot on my running dryer, it reported different sets of numbers that were fairly close to one another: "51,-24,1207" and "47,-31,1233". When I then walked it to another room and placed it on my desk it reported "-935,40,220", but also, at another time, while near my desk, "5,46,1233"

Implementation

The device continues to report "vibrate" every minute or so when it is on a vibrating surface. However, there is no report when it stops vibrating. There is not a "no_motion" report like we see with the PIR sensors. This means that reporting when vibration stops will require a timer of slightly over 1 minute, most likely implemented in the sensor/xiaomi_aqara Entity code.

Shall we go ahead and implement it as a switch with buttonevent? The problem will be: the sensor doesn't report any "no action" so we have to start a timer and stop the "button pressed" after 1 minute or so (unless more changes have been sent over ZigBee). Would be nice to surface the other data in the rest API.. but not a deal breaker

Screenshot from app

image

manup commented 5 years ago

@manup, I kept the code for ZHAVibration and CLIPVibration, even though currently we don't support any IAS_Zone vibration sensors. I suppose it won't do any harm and it might come in handy later.

Yes I guess sooner or later more these sensors will be available, so It's wise to keep the code :)

ebaauw commented 5 years ago

Shall we go ahead and implement it as a switch with buttonevent? The problem will be: the sensor doesn't report any "no action" so we have to start a timer and stop the "button pressed" after 1 minute or so (unless more changes have been sent over ZigBee).

That's precisely why I think it should be supported as a switch, rather than a sensor. Switches don't report that the button is pressed. Switches report events (button was pressed, button was released). Sensors, on the other hand, report a current state (door is open, door is closed, temperature is 21Ā°C). Sensors are stateful, switches are stateless. You can query a sensor for its current state; you cannot query a switch for its current event.
Granted, this is a subtle difference, as a new state value (door is now open) is very similar to a state change event (door was opened).

My point is that the vibration sensor sends events. For tilt and drop (sensor was tilted, sensor was dropped), this is very intuitive - you don't expect a state (sensor is falling), since the drop only takes a brief moment. For vibration (sensor was shaken) it is maybe less intuitive, as vibration sounds more like a state (sensor is vibrating), which lasts longer.

Incidentally, we have the same confusion with motion sensors, which basically issue an event (motion was detected), but are implemented as ZHAPresence, suggesting a state (some-one is present). Only some motion sensors actually report that motion is no longer detected. For the others, deCONZ provides a timer, config.duration, to reset state.presence. Then of course, "motion is no longer detected" is something entirely different than "no-one is present".

I don't think we can technically expose the vibration sensor as two different resources for tilt/drop (ZHASwitch) vs vibration (ZHAVibration).

paolotremadio commented 5 years ago

Sounds good to me, I will test it tomorrow.

Just one question: let's say I have the vibration sensor connected to deCONZ and I connect deCONZ to homebridge-hue. Homebridge-hue will expose the switch. We also know that the vibration sensor reports motion from time to time (let's say every 30 seconds).

With that setup, I shake the aqara sensor, the switch gets the event and I can connect an automation in Homekit. If I keep shaking the aqara sensor, will it trigger a new automation after 30 seconds, when deCONZ gets the status update from the sensor?

ebaauw commented 5 years ago

Yes.

paolotremadio commented 5 years ago

log3.log

I've shaken, tilted and dropped the sensor

paolotremadio commented 5 years ago

The sensor is now displayed as a switch, but I had to re-added it via the Phoscon UI

ebaauw commented 5 years ago

Either I screwed up, or deCONZ hasnā€™t yet fully recognised the sensor. Could you try to read the attributes of the Basic cluster in the GUI, while Phoscon is searching for new devices? Briefly press the reset button or shake the sensor, to make sure itā€™s awake.

paolotremadio commented 5 years ago

Done it. Turns out it doesn't get correctly recognised if I add it via the deCONZ GUI. But it works over the Phoscon UI. and it is reporting the correct buttonevent :)

paolotremadio commented 5 years ago

Shall I create a separate issue for adding the support on homebridge-hue?

Here's the output from the REST api (shown as "unsupported" in Homekit):

{
        "config": {
            "battery": null,
            "on": true,
            "reachable": true,
            "temperature": 0
        },
        "ep": 1,
        "etag": "010f388f53a1178a50b280aa6752ee54",
        "manufacturername": "LUMI",
        "mode": 1,
        "modelid": "lumi.vibration.aq1",
        "name": "lumi.vibration.aq1 (2)",
        "state": {
            "buttonevent": 1009,
            "lastupdated": "2018-09-05T18:40:04"
        },
        "type": "ZHASwitch",
        "uniqueid": "00:15:8d:00:02:a8:ad:09-01-0101"
    }

img_8539 img_8540

ebaauw commented 5 years ago

Yeah, opening the network from the GUI doesnā€™t trigger the REST API plugin to create new resources. You need to use Phoscon or the old web app.

Iā€™ll create a PR for the changes.

Note that I still need to whitelist the sensor in homebridge-hue. If I expose it as a switch, I need to map the buttonevents to Single Press, Long Press, and Double Press.

ebaauw commented 5 years ago

Oh, looks like our posts crossed. Please, feel free to open an issue in homebridge-hue, so I wonā€™t forget. Will be this weekend.

oltman commented 5 years ago

Hi there-- eagerly following this to build a smartthings device handler. I think I have decoded 0x0508-- it is the final resting tilt of the device after motion. This needs to be represented by 3 values for x,y,z and acceleration (since they use an accelerometer to detect tilt). On the smartthings log, I get 12 hex characters. Based on my testing I have decoded as follows:

First 4 hex characters: 2's complement encoded Z axis (+ squigly line side up yields pos values on this, flip to get negative) Second 4 hex characters: 2's complement encoded Y axis (+ button side up yields pos values on this, flip to get negative) Last 4 hex characters: 2's complement encoded X axis (+ right side of device yields pos values, opposite yields negative)

Testing: Side Up, 0x0508 raw return, decode: 1st digit, 2nd digit, 3rd digit 1 (squiggle), 04ae002f001e, 1198, 47, 30 2 (battery), fcb4002a0046, -843, 42, 70 3 (right), 0086003803e5, 134, 56, 997 4 (left), 00690015fc91, 105, 21, -878 5 (opp button), 006afc2d0073, 106, -978, 115 6 (button), 00600445ffff, 96, 1093, 0

If you then mix an angle (say 45Ā°) you get an appropriate mix of tilts... use your geomotry here...

Tested this with two brand new devices...

paolotremadio commented 5 years ago

I can confirm that 0x0503 is the rotation angle when ā€œtiltā€. It will report how many degrees the sensor has been tilted (it will report the difference from the previous position)

Kane610 commented 5 years ago

Could you perhaps summarise what values we can get from the sensor and what they mean?

oltman commented 5 years ago

@paolotremadio -- I don't get any returns from 0x0503-- did you mean 0x0508?

paolotremadio commented 5 years ago

General

The sensor reports tilt and drop/free-fall as soon as the action has completed (e.g. as soon as you stop tilting or as soon as the sensor has landed). Vibrations get reported every 30-60 seconds. The values get reported on the Door lock cluster (0x0101).

Attribute 0x0055

Value 1: shake / vibrate Value 2: tilt Value 3: drop / free fall

Attribute 0x0503

If 0x0055 is 1 - bed activity; it might correspond to how much movement was seen during the last reporting period (keep in mind that vibrations are reported every 30-60 seconds). If 0x0055 is 2 - tilted angle, in degrees; if the sensor has been was laying down and it gets tilted 90 degrees, the value will be 90 degrees. Please note: this is the difference from the previous sensor position and not an absolute measurement of rotation. If 0x0055 is 3 - unknown / no update.

Attribute 0x0508

Still not sure. Likely to be:

First 4 hex characters: 2's complement encoded Z axis (+ squigly line side up yields pos values on this, flip to get negative) Second 4 hex characters: 2's complement encoded Y axis (+ button side up yields pos values on this, flip to get negative) Last 4 hex characters: 2's complement encoded X axis (+ right side of device yields pos values, opposite yields negative)

Usage examples

paolotremadio commented 5 years ago

@ebaauw don't forget to update the Wiki to add the sensor to the list of supported devices šŸ˜‰

oltman commented 5 years ago

Any idea which register(s) program sensitivity and any other settings?

ebaauw commented 5 years ago

Still puzzled about 0x0508. @paolotremadio did you manage to capture an adsu message for this attribute? You reported 16 hex characters before (which suggests u64), but @oltman (thanks for helping out, BTW!) only mentions 3x4 = 12 hex characters (which suggests u48). Or is the acceleration also encoded in this attribute? Or could it be a bug in deCONZ GUI, not displaying a u48 value correctly?

It would be nice if we can figure out the relationship between 0x0503 an 0x0508 values and the tags (I would suspect 98, 99, and 9A) in the Xiaomi special attribute.

paolotremadio commented 5 years ago

Is sensitivity something set on the device or at the hub/gateway level? How is it implemented on Xiaomi and/or Philips motion sensors?

veeceeoh commented 5 years ago

@oltman - From this webpage by the creator of ZiGate, the sensitivity level is set with a write command as follows:

Cluster: 0x0000, Attr: 0xFF0D, DataType: 0x20 (8bit Integer), Data: ā€¢ 0x01 = High sensitivity level ā€¢ 0x0B = Medium sensitivity level ā€¢ 0x15 = Low sensitivity level

By the way, I am already working on SmartThings device handler, and will be adding an initial beta to the bspranger/Xiaomi GitHub repository soon.

Thanks to everyone on this thread for all the helpful information!

paolotremadio commented 5 years ago

@ebaauw can we support sensitivity in deCONZ? @veeceeoh, you're welcome. It's a team work, we are all helping reverse-engineering this device :)

ebaauw commented 5 years ago

can we support sensitivity in deCONZ?

Sure. We already have config.sensitivity for the Hue motion sensors.

@veeceeoh, do you know if this is implemented as a manufacturer-specific attribute?

veeceeoh commented 5 years ago

@ebaauw - It must be, as I don't see Attribute 0xFF0D mentioned in the ZigBee documentation. On that webpage, he states that the Manufacturer Code is 0x115F (Xiaomi).