Closed Magalex2x14 closed 4 years ago
Hello @Magalex2x14 :)
Thanks! even though i don't have near as much time as i'd like to explore more (still have the amazfit watches and ihealth devices in WIP).
I've already looked a bit about these new sensors (the square ones with the emoji thing for comfort level), sadly, when i've visited the xiaomi store in champs elysees earlier this month they didn't had any (only the "old" round ones), and as i'm moving quite a lot lately, i can't really order one from china and wait 3 weeks until it comes (i'll try in the xiaomi store in valencia, near where i am now if they got any).
The ideal thing would be to get the mi home "app" (mi home is a modular system as each OEM makes it's own "mi home app" that is what's launched when you tap on a device in mi home, hence the differences in UI), as these app are only downloaded from xiaomi's servers when you pair one of the devices, so, ideally somebody that has the device paired in mi home and a rooted phone could get a copy of the app (it's midnight here so i don't have time to charge and check my rooted phone, but i'll give the path of the modules tomorrow :) ). It's often the easiest way to reverse engineer the protocol, as apps are made by OEMs they are rarely obfuscated, and quite small (as their one and only job is to manage the device, it's a way smaller target for RE than the whole mi home). If i can get my hands on one of these in valencia i'll definitively reverse engineer it.
I'll also take a look tomorrow on the data you have on your side, especially the ble data :)
Oh, I did not know about the modularity of the mihome app (I'm an iOS user). Apparently, therefore, I could not find anything useful in the downloaded apk :) I will dig in the direction pointed by you, thank you! I did not shared some information (such as the https log with the cloud exchange and the corresponding hci_snoop.log) as it contains confidential information (one of the users sent it in a personal message, there are tokens, etc.) In the near future I plan to get these logs by myself (in my account there is nothing but a pair of lywsd03mmc sensors) and I want to collect all that I have in one place the other day. I will definitely share it.
I collect information here
I think it's the same in ios (though i don't know much about ios in general), as mi home is basically a "homekit-like" with multiple 3rd party having a SDK to have their devices appear and work in mi home (multiple SDKs even, with one for mi home "apps", miot for the devices themselves, and probably one for the mi home backend as well).
The http log between mi home and AWS (mi home servers are on AWS) isn't probably going to be useful, beside for getting the token, if it's ever needed (rarely on BLE devices, the token is more something for wifi devices using the miio protocol). Though a really useful part (that i didn't had time to look into yet) would be the app downloading http log, maybe it's actually easy to download an app using a script, that would be useful to be able to download all apps at once instead of having to buy the devices to get the app with it :smile:
You can find the plugin files in /data/data/com.xiaomi.smarthome/files/plugin/install
, the most interesting ones are probably /data/data/com.xiaomi.smarthome/files/plugin/install/mpk/(id_of_the_plugin)/oat/arm64/(???).odex
, which should be the actual plugin. I don't know the id for the square thermometer though (if somebody have multiple plugins, it's even better though, that may give informations on more than just the square thermometer!). It should be safe to share, as it's the app code only, there shouldn't be any personal data in it.
It works in a bit different way now. Modern devices have plugins written on React Native and often rely on the main app's code. I believe, Bluetooth pairing one of such things. But if it will be helpful - i've attached RN-plugin for this sensor. And btw, why you considering that log between app and cloud is useless? Cloud should have the ability to decrypt BLE advertising translated through BLE-gateways, so the app definitely sends key info. I believe bind_key being used for this somehow, which could be found in the mentioned log :)
I don't think they switched this dramatically, as moving from their existing platform to react native would probably break several hundreds of devices, including some important brands for them (like yeelight), some plugins may be in react native, as AFAIK there's no big limitations on the plugins, but i don't think mi home switched to react native as a requirement. Bluetooth pairing (with the rc4-based auth protocol) was always dependent on mi home, as the plugins are executed on mi home's context, they inherit it's JNI and several classes, i do believe that xiaomi have a SDK to use these given what i've seen in the plugins i've reverse engineered, the bluetooth auth is a JNI, and there are several others (especialiy for miot).
BLE traffic is mostly non encrypted, especially for these kind of devices where there's actually nothing sensitive inside, even though it's suprising to find a challenge-response auth even for things like toothbrushes, it's mostly because they probably used some xiaomi-made boilerplate code more than something else.
These sensors aren't made by the same company than the round one though, the "round one" is made by clearglass, this one is made by miaomiaoce (this model seems to be the "chameleon mini 2"), they seem to be a general health/wellbeing company, so i guess there's not a ton of engineering that went into simple temperature sensors, especially that they aren't focused on making BLE devices.
The cloud connection, for BLE devices is mostly for data synchronisation, sometime they are used for some calculations, but overall, it seems that they try to minimize the processing done on the cloud (which is normal, as it's more costly for xiaomi to do calculations on their EC2s machines than on the devices, and there's not much "secret sauce" that needs to be hidden there), the keys/tokens are the standard ones and AFAIK are generated for every mi home device, even if the device doesn't do anything with it (which is the case of the clearglass sensor, for example).
Thanks for the app, i'll look into it when i'll have some free time :)
True, some devices still use APK-based plugins, but MI forces developers to switch to RN-based ones. Btw, Yeelight almost completely switched to RN already :) And as I said - this sensor uses RN too.
BLE traffic encryption is not the issue which we struggle to. The main issue that BLE ADV-packets (connection-free broadcasts) are encrypted, which is something new for such devices. Miaomiaoce also made the e-ink clock-hygrometer, which doesn't use BLE-ADV encryption.
I believe you didn't get what's the point :) "Data synchronization" - the thing we are talking about. MI BLE-gateways are passive, they just collect and pass ADV packets to the cloud, no data processing is done at this point. So cloud itself - the only place where data could be decrypted, which is necessary to getting readable values and sending it to the status cards.
Oh, and I talking about bind_key, not tokens ;)
@vevsvevs can it be that the decryption takes place on the ble-gateway itself? That is, the gateway receives token/keys from the cloud, and in combination with the information already available (mac address, device id) this enough to decrypt?
Thanks for RN-plugin. Yesterday I watched it, and did not find any broadcasts processing there...
It turns out that we have no choice but to dig towards the ble-gateway and its cloud traffic?
Why do you think the ble advertisement frames are encrypted? that would be really surprising given the device and the company. If really it is encrypted it's probably a change from xiaomi's SDKs, and therefore, would probably be in some JNI (or in mi home, but they seems to prefer JNIs for these kind of stuff)
Data synchronisation really don't matter, if it's sent encrypted and it's decrypted on the servers, well, we aren't going to do much with it, if it's not, we're probably not getting a lot to work with either, so... given that the focus for BLE devices is actually the communication between the BLE device and mi home, the rest is not going to be helpful there i guess.
Given it's advertisement frames, it isn't possible to just capture these frames using bluetooth or gattool? given that the goal is to understand how the device works, i don't see much point in having the ble gateway in the equation as well.
there is an example of advertisements: in appearance, we assumed that they are encrypted. I'm not an expert, of course, but for me it looks that way...
Not the entire packet is encrypted, but only Service Data payload (we think that is encrypted :)
That part would actually surprise me, it wouldn't make any sense, given the costs to make and even the processing cost on the mcu (it's designed to be cheap, why put encryption where it's not needed?), especially for a thermometer (which doesn't really contain any private data). if they really did encrypt their advertisement frames, it would be kinda the same as going to the grocery store with a military APC, it works, it's more secure, but.. not much point in doing that :)
Though, given the sizes and the triple 0x00 in between, i highly doubt it's encrypted, if it was, it would defintiively look more random than that :)
Yes, I share your surprise. Zeros in the middle are not always zeros. After some time, when the packet counter overflows (if memory serves me right), the right of the three "zero" bytes increases by one. I will clarify this point...
It is also interesting that there are other devices sending such packets - yesterday another user wrote to me - Cleargrass CGD1.
Clearglass devices also send advertisement frame with it's data, but given it's a completly other company than miaomiaoce, it would be strange that they would share code... well, not completly suprising either, it's shenzhen after all :sweat_smile:
Do you have the temperature and humidity for the frames from your repo?
I have these sensors. I can collect a dump of any duration. I will do it and share it.
That would be awesome :)
Updated example.
Dump duration - 30 minutes
Temperature: 22.2 to 21.8
Humidity: 36 to 37
len AD Xiaomi ? Device packet MAC (LE) ----------------PAYLOAD-------------- RSSI
type UUID type № counter
0F 16 95 FE 30 58 5B 05 BF 6B 87 2F 38 C1 A4 08 E3
1A 16 95 FE 58 58 5B 05 C0 6B 87 2F 38 C1 A4 63 02 92 E6 37 21 00 00 E5 E9 32 F7 E2
0F 16 95 FE 30 58 5B 05 C1 6B 87 2F 38 C1 A4 08 E3
1A 16 95 FE 58 58 5B 05 C2 6B 87 2F 38 C1 A4 11 B4 AF 05 28 21 00 00 8B C7 85 E9 E3
0F 16 95 FE 30 58 5B 05 C3 6B 87 2F 38 C1 A4 08 E2
1A 16 95 FE 58 58 5B 05 C4 6B 87 2F 38 C1 A4 E3 28 24 BC 5F 21 00 00 BE 1E 41 1A E2
0F 16 95 FE 30 58 5B 05 C5 6B 87 2F 38 C1 A4 08 E2
1A 16 95 FE 58 58 5B 05 C6 6B 87 2F 38 C1 A4 9C 02 27 6B E4 21 00 00 D5 5E 58 EF E2
0F 16 95 FE 30 58 5B 05 C7 6B 87 2F 38 C1 A4 08 DC
19 16 95 FE 58 58 5B 05 C8 6B 87 2F 38 C1 A4 8E 95 EE 1F 21 00 00 BF 6A C6 94 DC --??
0F 16 95 FE 30 58 5B 05 C9 6B 87 2F 38 C1 A4 08 E2
1A 16 95 FE 58 58 5B 05 CA 6B 87 2F 38 C1 A4 4F DB 8C 38 30 21 00 00 6E 64 71 E6 DC
0F 16 95 FE 30 58 5B 05 CB 6B 87 2F 38 C1 A4 08 E2
1A 16 95 FE 58 58 5B 05 CC 6B 87 2F 38 C1 A4 D6 02 B0 D8 A4 21 00 00 03 7B C1 69 E2
0F 16 95 FE 30 58 5B 05 CD 6B 87 2F 38 C1 A4 08 E2
* "counter" starts from 00 00 00 and increases by 1 when "packet №" overflows
From what i've seen so far, the advertisement frame is not used at all in the app, everything happens when connected to the device.
The good news is that pretty much everything is in the app, without much (or even any on the mioamioce parts) obfuscation, from firmware OTA updates to configuration and getting the data, even the baby mode and the toothbrush data thing (?!?) so it should be pretty simple to document the device :)
I have a few of the square sensors and some prior experience poking about with BTLE.
I have an old android dev phone that allows for HCI snooping so I may be able to get, effectively, a pcap
if somebody can link me to an APK and walk me through what you'd like me to capture in the app.
Let me know if/how i can be of assistance :).
@vevsvevs can it be that the decryption takes place on the ble-gateway itself?
I bet no. At least - because I have an un-updated device with BLE-gateway functionality (built much-much earlier before ADV-encryption came to the scene) which transmits data from this sensor to the cloud without any issue. At most - because it would be a much more time-cost job for the cloud to form a response for every BLE-gateway with BLE-credential currently bound to it and joined to the corresponding account.
Why do you think the ble advertisement frames are encrypted?
Well, I think we collected enough information to make this assumption: MCU with built-in hardware AES encryption, previously unseen (meaningless) object data in ADV payload, some info about mi-beacon :)
That part would actually surprise me, it wouldn't make any sense, given the costs to make and even the processing cost on the mcu (it's designed to be cheap, why put encryption where it's not needed?), especially for a thermometer (which doesn't really contain any private data).
They didn't put anything aux, this MCU has built-in encryption core. As for thermometer's data - without encryption it could be easily spoofed for causing some (unwanted) automation triggering, , so making this data private is a good idea, imo.
Though, given the sizes and the triple 0x00 in between, i highly doubt it's encrypted, if it was, it would defintiively look more random than that :)
Obviously, it depends on the data sub-structure.
so it should be pretty simple to document the device
Direct connection abilities and data structures are already known, afaik. The main goal is getting which way to decrypt ADV-data. Cloud has MAC, token, bind_key and DID for doing this, we have to figure out which way these parameters tangled with encryption.
Sorry for grammar, too much English for me :)
For more information about Mi-Beacon and why we assumed that it could be encrypted - see the link from this message.
@prototux I updated my repository with all the information that I managed to dig up. My problem is solved - the ADV payload is decrypted. In case of work with the BT-gateway, data is decrypted on the gateway itself.
Thank you for your attention!
Hello, @prototux!
Your work on organizing knowledge of Xiaomi ecosystem devices is impressive and gives hope!
I am one of the authors of the custom component for Home Assistant, which collects data from advertising messages from Xiaomi sensors. Not so long ago, component users requested support for a new sensor - LYWSD03MMC. As a result of some research, it turned out that BLE advertisements from this sensor are encrypted.
Since I am not an expert in solving the problems of reverse engineering such devices, after a while I found myself in a dead end (I can’t even call myself a programmer, but just have a little understanding of how this works :)
In this regard, I wanted to ask if I can on behalf of users of this sensor, ask you for support in finding a method for decrypting advertisements from this sensor? Here is our discussion and our sad progress... But we collected some information (the type of chip used, its firmware, message dumps, logs of http exchanging between the official Xiaomi application and its cloud service, and so on). I am will be happy to share all the available information here, and I think that other users will help as much as they can. I have not yet given up, but hope is diminishing, and the help of a specialist like you would help us a lot...
Sorry for my bad English.