dresden-elektronik / deconz-rest-plugin

deCONZ REST-API plugin to control ZigBee devices
BSD 3-Clause "New" or "Revised" License
1.89k stars 496 forks source link

Xiaomi Mijia (HonneyWell) smoke detector #152

Closed rtenklooster closed 4 years ago

rtenklooster commented 7 years ago

Hi, Just received my RaspBee today and started playing around. I've upgraded to the latest beta in order to get my xiaomi switches and motion detectors running.

My smoke detectors are able to connect en they show up like this: rookmelder

With debugging enabled this event comes by when pushing the test button. 20:39:01:991 APS-DATA.indication srcAddr: 0x00158d00014d315c, dstAddrMode: 2, profile: 0x0104, cluster: 0x0500, lqi: 199, rssi: -66 20:39:02:218 APS-DATA.request id: 13, addrmode: 0x02, addr: 0x0000, profile: 0x0000, cluster: 0x0031, ep: 0x00 queue: 0 len: 2 20:39:02:292 APS-DATA.confirm id: 13, status: 0x00 SUCCESS And: 20:39:05:011 APS-DATA.indication srcAddr: 0x00158d00014d315c, dstAddrMode: 2, profile: 0x0104, cluster: 0x0500, lqi: 199, rssi: -66 20:39:05:100 APS-DATA.request id: 40, addrmode: 0x02, addr: 0x16c2, profile: 0x0000, cluster: 0x0031, ep: 0x00 queue: 0 len: 2 20:39:05:189 APS-DATA.confirm id: 40, status: 0x00 SUCCESS What should I do next to get my smoke detector supported?

manup commented 7 years ago

The cluster 0x0500 indicated that this sensor is a IAS Zone, I figure therefore it's easy to support it. However question is how should the sensor be represented by the API?

Either ZHAPresence or better ZHAOpenClose or maybe a new type like ZHAAlarm?

rtenklooster commented 7 years ago

ZHAAlarm seems the most appropriate type. There is a Gas sensor as well, and a water leakage sensor. They all could use the ZHAAlarm type I think.

snozzlebert commented 7 years ago

Presence of smoke/gas/water, so ZHAPresence?

ebaauw commented 7 years ago

However question is how should the sensor be represented by the API?

Tough one. I'm not too thrilled at the prospect of supporting another half a dozen of ZHA sensor types (and corresponding CLIP sensors). On the other hand, I would want a hint through the API what kind of sensor this is. So at least, I'd want different state attributes for presence of people, smoke, gas, water. My suggestion: if there's different ZigBee device types (not that Xiaomi would care for these), use different sensor resource types; if they're all just an IAS Zone device, use a single sensor resource type.

rtenklooster commented 7 years ago

Do you guys know if, once this detector is supported, it would be possible to control the siren from deconz?

Thought it would be really cool to use my 8 smoke detectors as sirens for my alarm... ;) (when not at home of course.. else I would scare the shit out of me in the middle of the night...)

ebaauw commented 7 years ago

else I would scare the shit out of me in the middle of the night

LOL. We could implement a state.scaringtheshitoutofme attribute for that.

On a more serious note, does it implement cluster IAS WD (0x0502) or provide another means to turn on (and, hopefully, off) the siren over ZigBee?

rtenklooster commented 7 years ago

does it implement cluster IAS WD (0x0502) or provide another means to turn on (and, hopefully, off) the siren over ZigBee?

It might be my lack of knowledge regarding Deconz, but I don't see any endpoints besides 01. smoke

I hope those endpoints will show up when the detector is being supported? Or else; how can I do some more testing / debugging?

Using the MI gateway you are able to enable and reset the alarm. This should mean using Deconz it should be theoretically possible to do this as well.

Those sensors also allow testing / identifying. You can send a command witch makes the smoke detector test itself sounds a short beep, and reports it's status.

Furthermore they report their battery status.

@ebaauw I suddenly realized the Xiaomi Switches and Motion sensors are not reporting their battery levels. I know they report the battery state. How can we solve this?

ebaauw commented 7 years ago

I hope those endpoints will show up when the detector is being supported? Or else; how can I do some more testing / debugging?

An endpoint is like a logical device. Is has an ID (0x01) a profile (in this example ZHA or Home Automation) and a device type (0xffff).
An endpoint might provide multiple clusters. A cluster has a direction indicated by the colour (blue for in or server clusters; grey for out or client clusters), and a type (0x0000 for the Basic) cluster. A server cluster maintains state attributes, which might be read and/or written from clients, or reported to clients, and it accepts commands. (Note: this is not the correct ZigBee terminology, but it helps me in understanding). The Cluster Info panel in the deCONZ GUI shows the standard commands and attributes for known clusters (this is actually configurable in the ZCLDB file, by default /usr/share/deCONZ/zcl/general.xml).

The problem with battery-operated nodes is that they sleep most of the time, causing queries from the gateway to discover their endpoints and clusters to fail. It would seem that this sensor hasn't been fully read by deCONZ, as only the Basic cluster is shown. You might want to try and open the network and press the test button (hoping this will wakeup the smoke sensor). For a motion sensor, you would wave, for a switch, you would press a button, for door sensors, you would move the magnet. If this doesn't result in more clusters being discovered, try deleting the node (select it and press DEL) and repairing it (while pressing the test button to keep it awake throughout the discovery).

Furthermore they report their battery status.

This is typically done thought the Power Configuration cluster 0x0001. Then again, nothing about the Xiaomi sensors is standard.

I suddenly realized the Xiaomi Switches and Motion sensors are not reporting their battery levels.

That's because we haven't found any evidence of the sensors reporting their battery level. We have seen some non-standard messages that we haven't figured out yet (see https://github.com/dresden-elektronik/deconz-rest-plugin/issues/138#issuecomment-325141455).

I know they report the battery state. How can we solve this?

Unless Xiaomi have open-sourced their gateway: Sniff the ZigBee communication between the Mi gateway and the sensors, reverse engineer the Mi gateway behaviour, and implement something similar in deCONZ.

rtenklooster commented 7 years ago

@ebaauw Thanks for your explanation. I have found some guy who appears to have reverse engineered some Xiaomi sensors. For example the Xiaomi Motion sensor:

fingerprint profileId: "0104", deviceId: "0104", inClusters: "0000, 0003, FFFF, 0019", outClusters: "0000, 0004, 0003, 0006, 0008, 0005, 0019", manufacturer: "LUMI", model: "lumi.sensor_motion", deviceJoinName: "Xiaomi Motion"

This line shows the used (in and out? ) clusters, is this the needed info? His code seems well documented, so proberly some info could be used. I don't know exactly where to add the clusters for this sensor in the XML file though.

https://github.com/a4refillpad/Xiaomi/blob/master/devicetypes/a4refillpad/xiaomi-motion-sensor.src/xiaomi-motion-sensor.groovy

ebaauw commented 7 years ago

This line shows the used (in and out? ) clusters, is this the needed info? His code seems well documented, so proberly some info could be used.

This would be the needed info, but it doesn't look like any of the Xiaomi devices I've seen. My guess would be he copied these from another sensor. Or he has a very different (hardware/firmware) version.

Looking at the code it looks like he extracts the battery value from the 0x0000 message indeed.

I don't know exactly where to add the clusters for this sensor in the XML file though.

Most are in there already. The XML file is used to build the Cluster Info panel; any clusters missing from the XML file will display in the GUI as unknown. As far as I know, if the clusters don't show in the GUI, the device doesn't have them and there's no way in deCONZ to add them (with the exception of correcting incomplete reads during pairing).

rtenklooster commented 7 years ago

Unless Xiaomi have open-sourced their gateway: Sniff the ZigBee communication between the Mi gateway and the sensors, reverse engineer the Mi gateway behaviour, and implement something similar in deCONZ.

Ok. that's what I did. foto

I sacrificed one of my Xiaomi push buttons and flashed it into a packet sniffer. I sniffed the transport key so I'm able to sniff and see all packages :)

I've tested my push button: push

Pretty nice I think!

manup commented 7 years ago

Nice, which sniffer is that?

rtenklooster commented 7 years ago

The hardware is a Xiaomi push button ( All Xiaomi devices have onboard ftdi pads) Inspired by this guy: http://faire-ca-soi-meme.fr/hack/2017/04/24/hack-xiaomi-mi-smarthome-decouverte-de-clef/

Ubiqua: https://www.ubilogix.com/ 1000 Packages per capture free for 21 days. Allows on the fly decoding and gives a pretty good inside view I think.

Now lets checkout the smoke detector in order to get them supported :)

manup commented 7 years ago

Looks really good :)

If possible capture as many use cases and setup traffic as possible and safe the log files, maybe Wireshark can open them as well.

rtenklooster commented 7 years ago

Yes, I can export to PCAP? (somethink like that, I believed Wireshark uses that format

ebaauw commented 7 years ago

Cool! What did you use to flash the button? And where did you get sniffer firmware? I take it the round black board is the late button? And the green board? Is a regular USB to serial converter or something more?

rtenklooster commented 7 years ago

Ok, What I did:

smoke

The image above shows the package capture ID from when I did the tests. I exported the capute to native ubiqua format, to wireshark and new wireshark.

Xiaomi Mijia Honeywell smoke detector.zip

I tested to import the capture. The downsite of opening those in wireshark is, the data is encrypted. So I guess the best thing would be if I'd shared the capture, and my decription key so you could view them in Ubiqua?

For example the test using the button gives this info: alarm I think you developers could not wish for more information do you?

ebaauw commented 7 years ago

Inspired by this guy

Unfortunately the Excel file he captured his sniffing conclusion in doesn't show us anything new. No battery info ;-(

rtenklooster commented 7 years ago

Cool! What did you use to flash the button? And where did you get sniffer firmware? I take it the round black board is the late button? And the green board? Is a regular USB to serial converter or something more?

The black board is the inside of the push button. I soldered wires onto the FTDI header. I connected those wires to my serial -> FTDI converter Pulled down D0 in order to get the Xiaomi Push button into programming mode. Flashed NXP sniffer sketch (JennickSniffer) included in the NXP sdk. Started capturing

His Excel sheet is pretty useless indeed. I will connect one of my motion sensors now and see what info comes around.

rtenklooster commented 7 years ago

It's up to you guys to let me know how the data should be delivered. Exporting the raw data could be done. Exporting only relevant frames with some explanation is possible like: After adding the motion sensor, the sensor sends this announcement: https://pastebin.com/VSTh1dab Or just screenshots or something like that.

For your info, I currently have:

For those devices I can supply extensive debugging if wanted.

manup commented 7 years ago

Unfortunately the Excel file he captured his sniffing conclusion in doesn't show us anything new. No battery info ;-(

I think the battery info is in the 0xff01 report send to basic cluster after an hour or so. I see this for the door/window contact sensor.

It's up to you guys to let me know how the data should be delivered. Exporting the raw data could be done. Exporting only relevant frames with some explanation is possible like: After adding the motion sensor, the sensor sends this announcement: https://pastebin.com/VSTh1dab Or just screenshots or something like that.

The most interesting frames are the APS layer ones, like ZCL reports and commands which the gateway sends to the devices.

For your info, I currently have:

Xiaomi Mijia Smoke detector Xiaomi smart socket Xiaomi push butten gen. 1 Xiaomi potion sensor gen. 1 Xiaomi Mijia door / windows sensor (ordered, will arrive soon) For those devices I can supply extensive debugging if wanted.

Xiaomi push butten gen. 1 and Xiaomi Mijia door / windows sensor are already supported.

rtenklooster commented 7 years ago

About the round switch. Yes, it's supported. However I did fount the following. It appears to be reporting double / triple / quad clicks. See this attached picture: button_events

rtenklooster commented 7 years ago

Ok. to get back on topic. I've sampled frames of all events I am aware off. I've documented them and pasted them on pastebin. I hope this information will provide enough information to get the smoke sensors supported :-) That's the aim of this issue.

https://pastebin.com/59rUTJKv

ebaauw commented 7 years ago

About the round switch.

Cool. I only have the Aqara switch (round button, square-ish outside), which doesn't send Press, only Release. I'll double-check on the 0x8000 attribute. Is it also on cluster 0x0006 (see APS header)? And a Report Attributes command (see ZCL header)? And is the Manufacturer Specific flag set? If so which Manufacturer Code is used?

rtenklooster commented 7 years ago

Ok cool, found the battery reporting as well. Will open a new issue regarding this push button.

rtenklooster commented 7 years ago

Added the attribute AD's and endpoints of the smoke detector to my Excel sheet.

https://tenklooster-my.sharepoint.com/personal/richard_tenklooster_net/_layouts/15/guestaccess.aspx?docid=0745e883aa555494cb8885a2d8eb48f3e&authkey=AQ08dGXdAz2r02LBPoyxdMw

lajo-osd commented 6 years ago

I just got this smoke detector today. Does it work with deCONZ in any way right now? Also, what do I need to press on the detector to pair it? :)

rtenklooster commented 6 years ago

To pair you should open the network and push the button three times.

And no, the smoke detectors are not supported yet. The smart plugs and round buttons multiple press events and battery data haven’t been added yet.

lajo-osd commented 6 years ago

Thanks rtenklooster that worked! So right now we cannot get anything out of the the smoke detector? I use the other Xiaomi buttons, pir and magnets to Node-Red and then to Home Assistant.

rtenklooster commented 6 years ago

You’re welcome. Yes it’s true.. you can’t use it until the captured addresses have been added to the xml file.

Have sniffed all packets and posted the result in an excel spreadsheet above. Feel free to give it a try, it should contain all needed info. Currently I’m way to short on time to do it myself.

The (Dresden) developers are currently focusing on other things I guess. Maybe they will find time in the near future to add my captured data in order to get the sensors fully supported. Including battery states etc. I have sacrificed one of my push buttons in order to provide them all data needed. Documented it and published the result so I’ve put my fingers crossed and hope they will find the time :-)

manup commented 6 years ago

Documented it and published the result so I’ve put my fingers crossed and hope they will find the time :-)

Absolutely, it's not forgotten I can't provide an ETA but the sniffer logs will be used to proceed on the integration of the detector :)

lajo-osd commented 6 years ago

Thanks rteenklooster, really nice work!! And thanks manup for hopefully integrating this detector in the not to distant future :)

AntiHeld889 commented 6 years ago

@ebaauw is it possible to get the Xiaomi Smoke detector working with Homekit (Homebridge-hue Plugin)

ebaauw commented 6 years ago

Probably. I need to see what the sensor resource(s) look(s) like in the REST API. Homebridge-hue should issue a warning message with all the info when it encounters an unknown sensor.

ebaauw commented 6 years ago

I totally missed PR #310. Could some-one please post what the sensor resources for the Heiman sensors look like, so I can add support for them to homebridge-hue? Thanks.

zbzhs20 commented 6 years ago

I've got a Schwaiger smoke sensor which seems be a Heiman sensor. I posted some information about it in issue #352 , maybe this helps. Unfortunately I don't have a zigbee sniffer ...

ebaauw commented 6 years ago

@rtenklooster, @AntiHeld889, I've got the Xiaomi Aqara flood sensor working with deCONZ (and with homebridge-hue). It too reports alarms through the IAS Zone cluster, but doesn't advertise the cluster. I think the smoke sensor could be supported in the same way. Can you test my latest PR?

AndreasRavn commented 6 years ago

I think the smoke sensor could be supported in the same way. Can you test my latest PR?

I'm still quite new to this, but I tried connecting one of my Xiaomis with 2.05.04 (which include your latest PR?), no luck. I even tried activating the alarm with a couple of blown matches, while the network was still open, to push it to transmit something, but no zhafire in the REST api.

smokecluster
ebaauw commented 6 years ago

No, the PR for the Xiaomi smoke sensor has been merged into deCONZ, but it is not included in v2.05.04. You need to wait for the next release, or compile the REST API plugin yourself (see README).

Could you please double-check on the Node Info panel that the Manufacturer Code is 0x1037 and that the MAC address starts with 0x00158d?

You need to open the network from the web app (or the Phoscon app). deCONZ won't create any API resources when you open the network from the GUI. When pairing the other Xiaomi devices, it's best to press/release the rest button (hole) regularly after opening the network, and then try and read the attributes of the Basic cluster. Typically, they support Manufacturer Name, Date Code and SW Build ID, but it takes a few attempts to read these.

rtenklooster commented 6 years ago

Opening the network from the webapp did the trick. When pushing the test button: image It's listed as fire instead of smoke. image image image So it looks great 🥇 Is it possible to set the fire state to on in order to make the sensor beep? Like a PUT request to /sensor/id/state fire: true?

Gave it a try but get error 400. Can imagine it's not designed to work like that. However it would be very nice to be able to sound the siren from home asistant. To extent my home assistant alarm with my 8 fire alarm's... can imagine the whole street would wake up when someone tries to break into my house in the middle of the night... 🗡

The rest api reports this: { "config": { "on": true, "reachable": true }, "ep": 1, "etag": "748cfe1cfdba9ee12a9db46213a286e5", "manufacturername": "LUMI", "modelid": "lumi.sensor_smoke", "name": "Fire 29", "state": { "fire": false, "lastupdated": "2018-02-19T22:26:06" }, "swversion": "20161018", "type": "ZHAFire", "uniqueid": "00:15:8d:00:01:4d:31:5c-01-0500" }

ebaauw commented 6 years ago

It's listed as fire instead of smoke.

That’s the name of IAS Zone Type.

Did you manage to activate the siren remotely or does it sound only when the sensor itself has an alarm?

rtenklooster commented 6 years ago

It havent managed to activate it remotely. Haven’t tested it either.

Maybe I can give it a try tomorrow. I believe I have seen a YouTube video suggesting when connected to the same mi gateway the alarm of all smoke detectors are linked. That would suggest the functionality is there..

Time to give it a try tomorrow. Let’s make some fire and see what happens with the other sensors.

Outlook voor iOShttps://aka.ms/o0ukef downloaden


From: Erik Baauw notifications@github.com Sent: Monday, February 19, 2018 11:48:13 PM To: dresden-elektronik/deconz-rest-plugin Cc: Richard; Mention Subject: Re: [dresden-elektronik/deconz-rest-plugin] Xiaomi Mijia (HonneyWell) smoke detector (#152)

It's listed as fire instead of smoke.

That’s the name of IAS Zone Type.

Did you manage to activate the siren remotely or does it sound only when the sensor itself has an alarm?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/dresden-elektronik/deconz-rest-plugin/issues/152#issuecomment-366820741, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGNMwi919ZT6dtPIKGRA-JnBGSDZKT0tks5tWfotgaJpZM4PH0WK.

AndreasRavn commented 6 years ago

No, the PR for the Xiaomi smoke sensor has been merged into deCONZ, but it is not included in v2.05.04. You need to wait for the next release, or compile the REST API plugin yourself (see README).

Ok. Got it working today with 2.05.05! :)

No luck activating the siren remotely for me neither. When paired to the xiaomi hub and app, it's possible to send a test signal to the siren from the app, but afaik not trigger the siren. The xiaomi hub can be configured as a repeater siren though, maybe that is what you have seen in the youtube video @rtenklooster ?

QempZor commented 6 years ago

Got the same smokedetectors connected today. Working like charm, except on turning on the sirene remotely.

That would be awesome to create your own home alarm/security system 👍

fasmide commented 6 years ago

I started off by ordering one of these smoke detectors just to verify that i could get them working, and the first one i got connected right away, very nice - not sure what pattern i pressed but i remember the pairing as super easy...

3 weeks fast forward, another 10 of these devices arrives by mail, the first one i pick up i'm having some trouble pairing to deconz, after googling around for a few seconds, reading other users pairing advice in this issue, i found out it seems to pair if the button it has was pressed 3 times in a row...

All was good however :)

I seem to have hit some kind of limit for these smoke detectors in the way it is absolutely impossible for me to pair a third smoke detector, I've tried not only one but 3 different directly from the original packaging and i cannot get them to pair with deconz - the Phoscon app just says failed to add device ( and i can confirm, the rest-api did indeed not have a sensor added to it...)

Currently using 2.05.12, some not too old raspbian on the Pi 3 with a raspbee - is there anything i can do to debug this?

QempZor commented 6 years ago

I've got 5 of them paired to deconz. Took me some time but it looks like you gave to open the network en then press the button 3 times on the smoke sensor

rtenklooster commented 6 years ago

My guess is you have reached the device limit per router. Experienced the same a while ago. There is a limited nummer of child devices per router. So adding a extra router, like a Ikea bulb or Xiaomi outlet could solve youre issues. Outlook voor iOShttps://aka.ms/o0ukef downloaden


From: QempZor notifications@github.com Sent: Wednesday, April 4, 2018 8:00:01 AM To: dresden-elektronik/deconz-rest-plugin Cc: Richard; Mention Subject: Re: [dresden-elektronik/deconz-rest-plugin] Xiaomi Mijia (HonneyWell) smoke detector (#152)

I've got 5 of them paired to deconz. Took me some time but it looks like you gave to open the network en then press the button 3 times on the smoke sensor

Op 3 apr. 2018 23:29 schreef Kristian Mide notifications@github.com:

I started off by ordering one of these smoke detectors just to verify that i could get them working, and the first one i got connected right away, very nice - not sure what pattern i pressed but i remember the pairing as super easy...

3 weeks fast forward, another 10 of these devices arrives by mail, the first one i pick up i'm having some trouble pairing to deconz, after googling around for a few seconds, reading other users pairing advice in this issue, i found out it seems to pair if the button it has was pressed 3 times in a row...

All was good however :)

I seem to have hit some kind of limit for these smoke detectors in the way it is absolutely impossible for me to pair a third smoke detector, I've tried not only one but 3 different directly from the original packaging and i cannot get them to pair with deconz - the Phoscon app just says failed to add device ( and i can confirm, the rest-api did indeed not have a sensor added to it...)

Currently using 2.05.12, some not too old raspbian on the Pi 3 with a raspbee - is there anything i can do to debug this?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dresden-elektronik/deconz-rest-plugin/issues/152#issuecomment-378405607, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AiHjwmGjm1Jj6VZgwR1d9LHK2GCS7fsoks5tk-nOgaJpZM4PH0WK.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/dresden-elektronik/deconz-rest-plugin/issues/152#issuecomment-378489383, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGNMwvTZi2849z76CSO1YewiR3l6Z_0Yks5tlGFhgaJpZM4PH0WK.

martikainen87 commented 6 years ago

I have 5 smoke detectors connected so just keep trying. Some devices can really be a pain in the butt, my first water sensor took about 100 tries to get connected but I haven't been succesfull with the second one, must have done over 300 tries with it so far :P

fasmide commented 6 years ago

@QempZor @rtenklooster @martikainen87 Thanks for your suggestions :)

Today i finally managed to add another one, it went something like this: I tried some sensors that i was 100% sure about how should be paired - these suddenly could not pair either which made me think it must have been deCONZ that somehow didn't really open the network when it should. An upgrade to the latest version didn't help But, a good old hard reboot with the power token off did the trick, now im able to pair both the smoke detectors and other sensors i have laying around...

To those wondering: The xiaomi mijia smoke detectors should pair when the only button they have is pressed 3 times somewhat quickly in a row

Straeng commented 6 years ago

Hello! Great work with these sensors! I added two today successfully. They are listed ny the /sensors resource and show up in HASS - however they do not show up in Phoscon app. Should they? How can I change their names/ids?