BLE Protocol Reverse Engineering

[nccchirag]

Inputs from @matthias-schulz

yee-rc detected as F8:24:41:C1:D1:1F (Yeelink) -67 dBm.

 │   Handles    │ Service > Characteristics │  Properties   │         Data         │
├──────────────┼───────────────────────────┼───────────────┼──────────────────────┤
│ 0001 -> 001a │ fe95                      │               │                      │
│ 0003         │     0001                  │ WRITE, NOTIFY │                      │
│ 0007         │     0002                  │ READ          │ 0000                 │
│ 000a         │     0004                  │ READ          │ O993yDåo8f04X        │
│ 000d         │     0005                  │ WRITE, NOTIFY │                      │
│ 0010         │     0007                  │ WRITE         │                      │
│ 0013         │     0010                  │ WRITE         │                      │
│ 0016         │     0013                  │ READ, WRITE   │ ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ │
│ 0019         │     0014                  │ READ, WRITE   │ L8e"ëad<01cúýéû 

LL Data: 05 22 ea 7f 8e f9 d1 c2 e0 ab df 41 24 f8 bb 3d e9 b1 d9 16 42 08 06 00 43 00 00 00 d0 07 ff ff ff ff 1f 10
[i] Got CONNECT_REQ packet from c2:d1:f9:8e:7f:ea to f8:24:41:df:ab:e0
 |-- Access Address: 0xb1e93dbb
 |-- CRC Init value: 0x4216d9
 |-- Hop interval: 67
 |-- Hop increment: 16
 |-- Channel Map: 1fffffffff
 |-- Timeout: 20000 ms

LL Data: 13 09 08 e1 00 00 00 00 00 00 00
LL Data: 0b 09 09 01 00 00 00 00 00 00 00
LL Data: 06 10 0c 00 05 00 12 01 08 00 10 00 20 00 00 00 c8 00
LL Data: 0a 0c 08 00 04 00 11 06 01 00 1a 00 95 fe
LL Data: 13 0c 00 08 06 00 24 00 00 00 c8 00 08 00
LL Data: 1e 0a 06 00 05 00 13 01 02 00 00 00
LL Data: 12 0b 07 00 04 00 10 1b 00 ff ff 00 28
LL Data: 0a 09 05 00 04 00 01 10 1b 00 00
LL Data: 12 0d 09 00 04 00 06 01 00 ff ff 00 28 95 fe
LL Data: 0a 09 05 00 04 00 07 01 00 1a 00
LL Data: 12 0d 09 00 04 00 06 1b 00 ff ff 00 28 95 fe
LL Data: 0a 09 05 00 04 00 01 06 1b 00 0a
LL Data: 12 0b 07 00 04 00 08 01 00 1a 00 02 28
LL Data: 0a 09 05 00 04 00 01 08 01 00 00
LL Data: 12 0b 07 00 04 00 08 01 00 1a 00 03 28
LL Data: 06 1b 17 00 04 00 09 07 02 00 18 03 00 01 00 06 00 02 07 00 02 00 09 00 02 0a 00 04 00
LL Data: 1e 0b 07 00 04 00 08 0b 00 1a 00 03 28
LL Data: 06 1b 17 00 04 00 09 07 0c 00 18 0d 00 05 00 0f 00 08 10 00 07 00 12 00 08 13 00 10 00

LL Data: 05 22 08 e4 ad a2 ac c8 1f d1 c1 41 24 f8 60 58 ac 0b 72 86 a0 08 06 00 43 00 00 00 d0 07 ff ff ff ff 1f 10
[i] Got CONNECT_REQ packet from c8:ac:a2:ad:e4:08 to f8:24:41:c1:d1:1f
 |-- Access Address: 0x0bac5860
 |-- CRC Init value: 0xa08672
 |-- Hop interval: 67
 |-- Hop increment: 16
 |-- Channel Map: 1fffffffff
 |-- Timeout: 20000 ms

LL Data: 13 09 08 e1 00 00 00 00 00 00 00
LL Data: 0b 09 09 01 00 00 00 00 00 00 00
LL Data: 12 0b 07 00 04 00 10 01 00 ff ff 00 28
LL Data: 0a 0c 08 00 04 00 11 06 01 00 1a 00 95 fe
LL Data: 13 0c 00 08 06 00 24 00 00 00 c8 00 08 00
LL Data: 1e 0a 06 00 05 00 13 01 02 00 00 00
LL Data: 12 0b 07 00 04 00 10 1b 00 ff ff 00 28
LL Data: 0a 09 05 00 04 00 01 10 1b 00 00
LL Data: 12 0d 09 00 04 00 06 01 00 ff ff 00 28 95 fe
LL Data: 12 0d 09 00 04 00 06 1b 00 ff ff 00 28 95 fe
LL Data: 0a 09 05 00 04 00 01 06 1b 00 0a
LL Data: 12 0b 07 00 04 00 08 01 00 1a 00 02 28
LL Data: 0a 09 05 00 04 00 01 08 01 00 00
LL Data: 12 0b 07 00 04 00 08 01 00 1a 00 03 28
LL Data: 0a 1b 17 00 04 00 09 07 02 00 18 03 00 01 00 06 00 02 07 00 02 00 09 00 02 0a 00 04 00
LL Data: 12 0b 07 00 04 00 08 0b 00 1a 00 03 28
LL Data: 0a 1b 17 00 04 00 09 07 0c 00 18 0d 00 05 00 0f 00 08 10 00 07 00 12 00 08 13 00 10 00

LL Data: 05 22 40 9f ce 64 21 c3 a3 d5 c1 41 24 f8 12 8c e2 7b eb 6e 0f 08 06 00 43 00 00 00 d0 07 ff ff ff ff 1f 05
[i] Got CONNECT_REQ packet from c3:21:64:ce:9f:40 to f8:24:41:c1:d5:a3
 |-- Access Address: 0x7be28c12
 |-- CRC Init value: 0x0f6eeb
 |-- Hop interval: 67
 |-- Hop increment: 5
 |-- Channel Map: 1fffffffff
 |-- Timeout: 20000 ms

LL Data: 13 09 08 e1 00 00 00 00 00 00 00
LL Data: 0b 09 09 01 00 00 00 00 00 00 00
LL Data: 12 0b 07 00 04 00 10 01 00 ff ff 00 28
LL Data: 0a 0c 08 00 04 00 11 06 01 00 1a 00 95 fe
LL Data: 0a 09 05 00 04 00 01 10 1b 00 00
LL Data: 12 0d 09 00 04 00 06 01 00 ff ff 00 28 95 fe
LL Data: 0a 09 05 00 04 00 07 01 00 1a 00
LL Data: 12 0d 09 00 04 00 06 1b 00 ff ff 00 28 95 fe
LL Data: 0a 09 05 00 04 00 01 06 1b 00 0a
LL Data: 12 0b 07 00 04 00 08 01 00 1a 00 02 28
LL Data: 0a 09 05 00 04 00 01 08 01 00 00
LL Data: 0a 1b 17 00 04 00 09 07 02 00 18 03 00 01 00 06 00 02 07 00 02 00 09 00 02 0a 00 04 00
LL Data: 12 0b 07 00 04 00 08 0b 00 1a 00 03 28
LL Data: 0a 1b 17 00 04 00 09 07 0c 00 18 0d 00 05 00 0f 00 08 10 00 07 00 12 00 08 13 00 10 00
LL Data: 12 0b 07 00 04 00 08 14 00 1a 00 03 28
LL Data: 0a 14 10 00 04 00 09 07 15 00 0a 16 00 13 00 18 00 0a 19 00 14 00
LL Data: 12 09 05 00 04 00 04 04 00 05 00
LL Data: 0a 0e 0a 00 04 00 05 01 04 00 02 29 05 00 01 29
LL Data: 12 09 05 00 04 00 04 08 00 08 00
LL Data: 0a 0a 06 00 04 00 05 01 08 00 01 29
LL Data: 12 09 05 00 04 00 04 0b 00 0b 00
LL Data: 0a 0a 06 00 04 00 05 01 0b 00 01 29
LL Data: 0a 0a 06 00 04 00 05 01 0e 00 01 29
LL Data: 0a 0a 06 00 04 00 05 01 11 00 01 29
LL Data: 12 09 05 00 04 00 04 14 00 14 00
LL Data: 0a 0a 06 00 04 00 05 01 14 00 01 29
LL Data: 12 09 05 00 04 00 04 17 00 17 00
LL Data: 0a 0a 06 00 04 00 05 01 17 00 01 29
LL Data: 12 09 05 00 04 00 04 1a 00 1a 00
LL Data: 0a 0a 06 00 04 00 05 01 1a 00 01 29
LL Data: 12 0b 07 00 04 00 12 13 00 90 ca 85 de
LL Data: 0a 05 01 00 04 00 13
LL Data: 12 09 05 00 04 00 12 04 00 01 00
LL Data: 0a 05 01 00 04 00 13
LL Data: 12 13 0f 00 04 00 12 03 00 8c d1 cf 62 43 fb b1 d3 f8 2a f2 b9
LL Data: 1a 05 01 00 04 00 13
LL Data: 06 13 0f 00 04 00 1b 03 00 5e 6a 72 c9 52 b1 95 a9 2c 0f 1f 51
LL Data: 1e 0b 07 00 04 00 12 03 00 99 7b 30 c5
LL Data: 06 05 01 00 04 00 13
LL Data: 1e 07 03 00 04 00 0a 19 00
LL Data: 06 11 0d 00 04 00 0b 4c 0a 2a 21 a8 c9 4a 69 63 4c e7 31
LL Data: 1f 02 02 13

[a-andreyev]

any progress here?

Nope. As you could see, newcomers like me are able to reproduce the progress of the past year (to complete the pairing and listen to encrypted BLE adv packets), but not able to handle the decryption logic.

[Magalex2x14]

I can't remember the reason why I subscribed to this thread, didn't read it in full, but I have a feeling that I have the information you need. I am working on integrating Mijia ecosystem devices into Home Assistant (ble monitor integration) and I was able to figure out encryption in due time. In a nutshell, here's everything I know on the subject:

1. The encryption key is generated when the device is activated (added) in the MiHome application.
2. All devices in the ecosystem known to me at the moment use a common encryption method.
3. The key is 16 bytes long and appears in the application traffic as bind_key or beaconkey
4. All currently known methods of obtaining a key are described in our FAQ
5. Decryption procedure can be seen in the integration code

I don’t own the device discussed here, so I don’t know how it works, but theoretically together we can try to implement its support in the HomeAssistant ble_monitor custom integration.

[madprogrammer]

The problem with this particular dimmer is that it is not paired with the Mi Home app (which allows to dump the beaconkey / bind_key) but instead it pars directly with a supported ceiling light.

One theoretically possible way of reverse engineering this could be to intercept a firmware update sent from the Mi Home app targeted to a supported ceiling light (although that firmware binary is probably encrypted), and then try to reverse engineer that firmware.

Another way is to play the guessing game and try to figure out the encryption with trial and error (seems not feasible)

The last and probably "cheapest" way is to ditch the original firmware of the dimmer and ether write a completely new firmware from scratch for the same board (which will not be compatible with Mi Home / ceiling light) or layout a new PCB which will fit into same enclosure, again with a custom firmware.

[ominitay]

It would probably be easiest to write custom firmware, considering this attempt has been going for 1 and a half years now...

There is an IDE provided on the MCU manufacturer's website which may be of use. It is Windows only, so unfortunately I can't be of use with testing this software...

[Ernst79]

We solved the decryption of the YLKG08YL, thanks to @rezmus. It's using the MiBeacon V2/V3 (legacy) format, in stead of the newer MiBeacon V4/V5 format. More info can be found in this issue of the BLE monitor repository. Support is added in BLE monitor 2.1.x (currently beta). The encryption key is 24 characters long, in stead of the usual 32 (in the decryption process, 4 bytes (8 characters) are added).

The code can be found here. To get the encryption key, check these instructions

[Busyrev]

The way to get encription key is described here. Succeeded with way 6. You should use India Location. Any account, fake ore real. Remove existing mi home before installing mod. Got keys. Next step is to connect it to raspberry.

[Busyrev]

@Ernst79 links in your previous post is 404 too

[Ernst79]

Thanks, I've updated the links (it got merged in the mean time, unfortunately the links weren't updated automatically).

BTW. Final version of BLE monitor (Home Assistant integration) is released, with support for YLKG07YL and YLKG08YL dimmers.

If any of you know an easier way to get the key, let me know, than I'll add that to the instructions.

For those who are interested, in the code, dimmer messages are (after decryption) parsed as obj0110(xobj) This message type 0110 is used for both remotes (YLYK01YL) as well as the dimmers, and consists of 3 bytes, in the code called button, value, press. Combinations of these three bytes are converted to output, like dimmer, press_type, remote_command and remote_binary. Dimmer sensors only use dimmer and press type output, the others are for remotes.

The decryption part of the code is at the end of the file, in decrypt_mibeacon_legacy(self, data)

[rezmus]

@Busyrev these 2 methods are working (pair with special mi home app or send miio cmd to ceiling paired with remote/dimmer), but the point is to auth device and read beaconkey directly via bt. some ppl in this issue were able to do that, maybe they can share working code. should be something similar to mikettle with a few adjustments.

https://github.com/drndos/mikettle

[gogui63]

@Ernst79 Do you know how to get the encryption key without ceiling light ? I do not have any BT Xiaomi gateway and I can't connect to the dimmer directly

[Ernst79]

Method 6 should work without dimmer, see the FAQ

[gogui63]

Unfortunately, doesn't work :(

[Ernst79]

Did you use the modified version of the MiHome mod app (the one with dimmer at the end)?

[gogui63]

Yes mi home mod downloaded on step 6, I see on device list my dimmer but unsucessfully pair it

[Ernst79]

And bluetooth is enabled on your phone? @rezmus Any idea?

[rezmus]

kill app, restart bluetooth, use different phone. just paired mine, no issues.

https://ibb.co/M6x38KM https://ibb.co/3mcbttj https://ibb.co/z80S9WV

[dezaxe69210]

Hello guys thanks a lot for your very hard work! Can you tell me please what are the steps to test the device with another app? Jeedom to be more precise

[Ernst79]

The links I gave are referring to BLE monitor, which is an "custom integration" for Home Assistant. Support for other apps/software has to be developed by others, you will have to ask the developers of the Jeedom software (Bluetooth advertisement plugin) to implement the code in their software. But the hard part is done (getting the BLE messages decrypted and translating the messages to human readable text).

If you have specific questions regarding the code, let me know. Basically you/they should develop something that listens to BLE advertisements, decrypt the advertisements and translate the payload to something readable. The last two steps are (for BLE monitor) done in this file

[Busyrev]

Sebastian at telegram channel https://t.me/xiaomi_gw_v3_hack got link to get key without app or any other shit. https://github.com/rexbut/mikettle/blob/master/get_beacon_key.py not tried for now. Will post here if it works.

[rezmus]

@Busyrev you can read more here

https://github.com/custom-components/ble_monitor/issues/353

[Busyrev]

Works. posted results here https://github.com/custom-components/ble_monitor/issues/353#issuecomment-846398163

[Ernst79]

Script to get the beaconkey is added in the BLE monitor documentation (including updated script for more devices).

[Busyrev]

Confirming total success. Got encryption key using get_beacon_key, installed HomeAssistant with https://github.com/custom-components/ble_monitor and cofigured device using key. It works.

[wouterf11]

@Ernst79 could you elaborate on how the BLE Monitor works for the YLKG08YL dimmer? How should the data on https://github.com/archaron/docs/blob/master/BLE/ylkg08y.md be combined into decoding a message, say: Click once 58 30 b6 03 | c1 | 69 44 c2 41 24 f8 | 17 4a bb 49 78 d3 02 | 00 00 | 63

with the beaconkey: 10 d8 99 8c 8e cd 72 9d e4 21 02 8d

From what I gather from xiaomi.py: key = "10 d8 99 8c 8e cd 8d 3d 3c 97 72 9d e4 21 02 8d" (beacon(1:6), 8d3d3c97, beacon(6:end)) nonce (iv) = "58 30 b6 03 c1 02 00 00 69 44 c2 41 24" (frame, dev, count, reverse mac(:-1) where count = (packet id, msg(-4:-1)) cipherdata = "17 4a bb 49 78" authdata = "11"

Is (any of) that correct? Many thanks in advance!

[rezmus]

@wouterf11 beaconkey is encrypted in this example (was not decrypted with token). use this sample data for dimmer to check your code.

https://github.com/custom-components/ble_monitor/issues/289#issuecomment-840572486

[Ernst79]

Here is a little python script that shows the decryption. Your message starts at frame ctrl and stops before rssi.

from Cryptodome.Cipher import AES

data_string = "043e25020103008b98c54124f819181695fe5830b603368b98c54124f88bb8f2661351000000d6ef"
aeskey = "b853075158487ca39a5b5ea9"

#                                       frame dev ct ---mac------ ----encrypted payload- rssi
#                                       ctrl  id                  cipherpayld- -cnt-- tk 
#  043e25020103008b98c54124f819181695fe 5830 b603 36 8b98c54124f8 8bb8f2661351 000000 d6   ef

data = bytes(bytearray.fromhex(data_string))
key = bytes.fromhex(aeskey)

key_1 = key[0:6]
key_2 = bytes.fromhex("8d3d3c97")
key_3 = key[6:]
key = b"".join([key_1, key_2, key_3])
print("key: ", key.hex())

xiaomi_index = data.find(b'\x16\x95\xFE')
xiaomi_mac_reversed = data[xiaomi_index + 8:xiaomi_index + 14]
print("reversed mac: ", xiaomi_mac_reversed.hex())
# reversed mac: 8b98c54124f8

framectrl_data = data[xiaomi_index + 3:xiaomi_index + 5]
print("frame ctrl: ", framectrl_data.hex())
# frame ctrl: 5830

device_type = data[xiaomi_index + 5:xiaomi_index + 7]
print("device type (product id): ", device_type.hex())
# device type (product id): b603

encrypted_payload = data[xiaomi_index + 14:-1]
print("encrypted payload: ", encrypted_payload.hex())
# encrypted payload: 8bb8f2661351000000d6

packet_id = data[xiaomi_index + 7:xiaomi_index + 8]
payload_counter = b"".join([packet_id,  encrypted_payload[-4:-1]])
print("payload counter: ", payload_counter.hex())
# payload_counter: 36000000

nonce = b"".join([framectrl_data, device_type, payload_counter, xiaomi_mac_reversed[:-1]])
print("nonce: ", nonce.hex())
# nonce: 5830b603360000008b98c54124

aad = b"\x11"

token = encrypted_payload[-1:]
print("token: ", token.hex())
# token: d6

cipherpayload = encrypted_payload[:-4]
print("cipher payload: ", cipherpayload.hex())
# cipher payload: 8bb8f2661351

cipher = AES.new(key, AES.MODE_CCM, nonce=nonce, mac_len=4)
cipher.update(aad)

decrypted_payload = cipher.decrypt(cipherpayload)
print("decrypted payload: ", decrypted_payload.hex())
# decrypted payload:  01100300ff04

The decrypted payload can be read as follows. 0110 = Button (= type of message according to the MiBeacon protocol) 03 = length of data 00 = button ff = value 04 = press

button, value and press are the names I use in BLE monitor, depending on the device type, they are translated to a message. See the def obj0110(xobj): function in https://github.com/custom-components/ble_monitor/blob/master/custom_components/ble_monitor/ble_parser/xiaomi.py. In this example, press 04 + button = 0 means "rotate left" with (256 - 255(= ff) = 1 steps.

I also tried your BLE advertisement + beaconkey, but it doesn't seem to be right, I get this as result.

decrypted payload: ab330e5cbc82

[rezmus]

I also tried your BLE advertisement + beaconkey, but it doesn't seem to be right, I get this as result.

he used sample data from old github, where beaconkey was not decrypted using token.

beacon_key = cipher(token, peripheral.readCharacteristic(HANDLE_BEACON_KEY)).hex()

[wouterf11]

Thanks a lot!! I will have a look.

[kirilldobr]

I successfully got dimmer beakonKey using method 6 from FAQ, but after adding the integration to HA I am getting weird errors:

2021-05-23 13:30:03 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Data binary sensor received: {'rssi': -55, 'mac': 'F82441C371CD', 'type': 'YLKG07YL/YLKG08YL', 'packet': 1, 'firmware': 'Xiaomi (MiBeacon)', 'data': True}
2021-05-23 13:30:03 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Data measuring sensor received: {'rssi': -55, 'mac': 'F82441C371CD', 'type': 'YLKG07YL/YLKG08YL', 'packet': 1, 'firmware': 'Xiaomi (MiBeacon)', 'data': True}
2021-05-23 13:30:16 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 5 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2021-05-23 13:30:16 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 5 BLE ADV messages processed for 1 measuring device(s).
2021-05-23 13:30:16 DEBUG (Thread-4) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2021-05-23 13:30:16 DEBUG (Thread-4) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2021-05-23 13:30:16 DEBUG (Thread-4) [custom_components.ble_monitor] 3121 HCI events processed for previous period.
2021-05-23 13:30:16 DEBUG (Thread-4) [custom_components.ble_monitor] HCIdump thread: Run
2021-05-23 13:30:16 DEBUG (Thread-4) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-05-23 13:30:16 DEBUG (Thread-4) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2021-05-23 13:30:31 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:31 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 13:30:32 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:32 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 13:30:33 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Data binary sensor received: {'rssi': -58, 'mac': 'F82441C371CD', 'type': 'YLKG07YL/YLKG08YL', 'packet': 1, 'firmware': 'Xiaomi (MiBeacon)', 'data': True}
2021-05-23 13:30:33 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Data measuring sensor received: {'rssi': -58, 'mac': 'F82441C371CD', 'type': 'YLKG07YL/YLKG08YL', 'packet': 1, 'firmware': 'Xiaomi (MiBeacon)', 'data': True}
2021-05-23 13:30:33 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:33 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 13:30:35 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:35 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 13:30:35 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:35 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 13:30:37 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:37 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 13:30:37 ERROR (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 13:30:37 DEBUG (Thread-4) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed

My configuration.yaml:

ble_monitor:
  devices:
    - mac: 'CD:71:C3:41:24:F8'
      name: 'myDimmer'
      encryption_key: '346B958D1E040ED240AB84B7'

They seem to happen every time dimmer action is sent... Does anyone know what can cause such a behaviour?

[Ernst79]

Did you restart HA?

[kirilldobr]

Yes I did, but the result is the same. It’s state changes between unknown (dark gray) and unavailable (light gray, during events).

[Ernst79]

Can you show the log from right after the start of HA?

[kirilldobr]

That's debug log enabled, of course.

[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] udev.sh: executing... 
[cont-init.d] udev.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
2021-05-23 16:38:36 WARNING (MainThread) [homeassistant.loader] You are using a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2021-05-23 16:38:36 WARNING (MainThread) [homeassistant.loader] You are using a custom integration ble_monitor which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2021-05-23 16:38:36 WARNING (MainThread) [homeassistant.loader] You are using a custom integration yandex_smart_home which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2021-05-23 16:38:42 DEBUG (MainThread) [custom_components.ble_monitor] Initializing BLE Monitor integration (YAML): {'devices': [{'mac': 'CD:71:C3:41:24:F8', 'name': 'Крутилочка', 'encryption_key': '346B958D1E040ED240AB84B7', 'reset_timer': 35, 'restore_state': 'default', 'decimals': 'default', 'use_median': 'default'}], 'bt_interface': ['DC:A6:32:F7:CD:74'], 'log_spikes': False, 'restore_state': False, 'active_scan': False, 'hci_interface': [], 'discovery': True, 'report_unknown': False, 'period': 60, 'decimals': 1, 'rounding': 1, 'batt_entities': True, 'use_median': False, 'is_flow': False, 'ids_from_name': True}
2021-05-23 16:38:42 DEBUG (MainThread) [custom_components.ble_monitor.config_flow] async_step_import: {'devices': [{'mac': 'CD:71:C3:41:24:F8', 'name': 'Крутилочка', 'encryption_key': '346B958D1E040ED240AB84B7', 'reset_timer': 35, 'restore_state': 'default', 'decimals': 'default', 'use_median': 'default'}], 'bt_interface': ['DC:A6:32:F7:CD:74'], 'log_spikes': False, 'restore_state': False, 'active_scan': False, 'hci_interface': [], 'discovery': True, 'report_unknown': False, 'period': 60, 'decimals': 1, 'rounding': 1, 'batt_entities': True, 'use_median': False, 'is_flow': False, 'ids_from_name': True}
2021-05-23 16:38:42 DEBUG (MainThread) [custom_components.ble_monitor.config_flow] async_step_user: {'devices': '--Devices--', 'bt_interface': ['DC:A6:32:F7:CD:74'], 'log_spikes': False, 'restore_state': False, 'active_scan': False, 'hci_interface': [], 'discovery': True, 'report_unknown': False, 'period': 60, 'decimals': 1, 'rounding': 1, 'batt_entities': True, 'use_median': False, 'is_flow': False, 'ids_from_name': True}
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] Initializing BLE Monitor entry (config entry): <homeassistant.config_entries.ConfigEntry object at 0x7fa942d460>
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: domain {'devices': [{'mac': 'CD:71:C3:41:24:F8', 'name': 'Крутилочка', 'encryption_key': '346B958D1E040ED240AB84B7', 'reset_timer': 35, 'restore_state': 'default', 'decimals': 'default', 'use_median': 'default'}], 'bt_interface': ['DC:A6:32:F7:CD:74'], 'log_spikes': False, 'restore_state': False, 'active_scan': False, 'hci_interface': [], 'discovery': True, 'report_unknown': False, 'period': 60, 'decimals': 1, 'rounding': 1, 'batt_entities': True, 'use_median': False, 'is_flow': False, 'ids_from_name': True}
2021-05-23 16:38:46 WARNING (MainThread) [custom_components.ble_monitor] Available Bluetooth interfaces for BLE monitor: ['DC:A6:32:F7:CD:74']
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] async_setup_entry: {'devices': [{'mac': 'CD:71:C3:41:24:F8', 'name': 'Крутилочка', 'encryption_key': '346B958D1E040ED240AB84B7', 'reset_timer': 35, 'restore_state': 'default', 'decimals': 'default', 'use_median': 'default'}], 'bt_interface': ['DC:A6:32:F7:CD:74'], 'log_spikes': False, 'restore_state': False, 'active_scan': False, 'hci_interface': [0], 'discovery': True, 'report_unknown': False, 'period': 60, 'decimals': 1, 'rounding': 1, 'batt_entities': True, 'use_median': False, 'is_flow': False, 'ids_from_name': True}
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] HCI interface is [0]
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] Spawning HCIdump thread
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] HCIdump thread: Init
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] 1 encryptors mac:key pairs loaded.
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] whitelist: []
2021-05-23 16:38:46 DEBUG (MainThread) [custom_components.ble_monitor] 0 whitelist item(s) loaded.
2021-05-23 16:38:46 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2021-05-23 16:38:46 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-05-23 16:38:46 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2021-05-23 16:38:47 WARNING (MainThread) [homeassistant.config_entries] Config entry 'MFC-J2510 E71354E3F408895' for brother integration not ready yet: Bad IPv4/UDP transport address BRW1C3E84A7D7C3.local@161: [Errno -2] Name does not resolvecaused by <class 'socket.gaierror'>: [Errno -2] Name does not resolve; Retrying in background
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Starting binary sensor entry startup
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialization
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] BLE binary sensors updater initialized
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary sensor entry setup finished
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Starting measuring sensor entry startup
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialization
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.sensor] BLE sensors updater initialized
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Measuring sensor entry setup finished
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Binary entities updater loop started!
2021-05-23 16:38:48 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Entities updater loop started!
2021-05-23 16:38:50 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for remote.xiaomi_miio_192_168_1_82 (<class 'homeassistant.components.xiaomi_miio.remote.XiaomiMiioRemote'>) took 0.639 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+xiaomi_miio%22
2021-05-23 16:39:09 ERROR (MainThread) [pyhap.characteristic] SecuritySystemCurrentState: value=0 is an invalid value.
2021-05-23 16:39:09 ERROR (MainThread) [pyhap.characteristic] SecuritySystemTargetState: value=0 is an invalid value.
2021-05-23 16:39:40 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'click' when rendering '{{ value_json.click }}'
2021-05-23 16:39:41 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'click' when rendering '{{ value_json.click }}'
2021-05-23 16:39:41 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'click' when rendering '{{ value_json.click }}'
2021-05-23 16:39:48 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] 0 MiBeacon BLE ADV messages processed for 0 binary sensor device(s) total. Priority queue = 0
2021-05-23 16:39:48 DEBUG (MainThread) [custom_components.ble_monitor.sensor] 0 BLE ADV messages processed for 0 measuring device(s).
2021-05-23 16:39:48 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: main event_loop stopped, finishing
2021-05-23 16:39:48 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Scanning will be restarted
2021-05-23 16:39:48 DEBUG (Thread-5) [custom_components.ble_monitor] 3085 HCI events processed for previous period.
2021-05-23 16:39:48 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: Run
2021-05-23 16:39:48 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: connected to hci0
2021-05-23 16:39:48 DEBUG (Thread-5) [custom_components.ble_monitor] HCIdump thread: start main event_loop
2021-05-23 16:39:51 WARNING (MainThread) [homeassistant.components.media_player] Updating samsungtv media_player took longer than the scheduled update interval 0:00:10
2021-05-23 16:40:15 ERROR (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 16:40:15 DEBUG (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 16:40:16 ERROR (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 16:40:16 DEBUG (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 16:40:16 DEBUG (MainThread) [custom_components.ble_monitor.binary_sensor] Data binary sensor received: {'rssi': -44, 'mac': 'F82441C371CD', 'type': 'YLKG07YL/YLKG08YL', 'packet': 1, 'firmware': 'Xiaomi (MiBeacon)', 'data': True}
2021-05-23 16:40:16 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Data measuring sensor received: {'rssi': -44, 'mac': 'F82441C371CD', 'type': 'YLKG07YL/YLKG08YL', 'packet': 1, 'firmware': 'Xiaomi (MiBeacon)', 'data': True}
2021-05-23 16:40:16 DEBUG (MainThread) [custom_components.ble_monitor.sensor] Sensor device with mac address F8:24:41:C3:71:CD has the following settings. Name: F82441C371CD. Temperature unit: °C. Decimals: 1. Use Median: False. Restore state: False. Reset Timer: 35.
2021-05-23 16:40:16 DEBUG (MainThread) [custom_components.ble_monitor.sensor] async_added_to_hass called for ble dimmer F82441C371CD
2021-05-23 16:40:17 ERROR (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 16:40:17 DEBUG (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed
2021-05-23 16:40:19 ERROR (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Decryption MiBeacon V2/V3 advertisement failed: No encryption key found
2021-05-23 16:40:19 DEBUG (Thread-5) [custom_components.ble_monitor.ble_parser.xiaomi] Invalid data: Data decryption failed

[kirilldobr]

Btw, i'm running on a RPI B4, Home Assistant OS 5.13

[Ernst79]

You have entered the MAC address reversed (per two characters in your config

[kirilldobr]

Script told me that mac address is

{'mac': 'cd71c34124f8', 'evtid': 4097, 'pid': 950, 'beaconkey': '346b958d1e040ed240ab84b7'}

I double checked mac address in the config, it's the exact same. I am not seeing something obvious, aren't I (

[kirilldobr]

I got it! The script returns MAC address in the reversed order! Everything works now.

Thank you for your help and your amazing work, I've been following the YLKg08YL thread closely, it inspired me to create a proper home automation ✨

[Ernst79]

I will change the script to return the real MAC. I can understand its confusing.

[rezmus]

@Ernst79 mac is reversed in miio cmd response (method 6 he used). just bold in faq to reverse it during remote/dimmer setup in HA.

afaik mac for dimmer/remote (not from bundle) always starts with F8:24:41.

[Ernst79]

Ah, you’re right. I’ll highlight it in the FAQ

[psylity]

I've made a full python code that pairs and reads Yeelight dimmer data. For those who want the working code - check it out https://github.com/psylity/yeelight-dimmer-python

[psylity]

Also I'm glad to present ESP-IDF component that adds Yeelight YLKG08Y support to your ESP32 projects - yeelight-dimmer-esp32

[Busyrev]

@psylity @AlexxIT do you know any way to read battery level for this dimmers?

[onlysunjun]

kill app, restart bluetooth, use different phone. just paired mine, no issues.

https://ibb.co/M6x38KM https://ibb.co/3mcbttj https://ibb.co/z80S9WV

Dose the xiaomihome app need the old version? the web url is cannot work,https://github.com/custom-components/ble_monitor/issues/353 ,could you share the xiaomihome app? thx

[AlexxIT]

@Busyrev Does any Xiaomi software show the battery value?

[Busyrev]

AlexxIT

Does any Xiaomi software show the battery value?

No idea, I`m not using xiaomi software at all.

[psylity]

@psylity @AlexxIT do you know any way to read battery level for this dimmers?

There is no place left in standard ble packet to store the battery level information.

[Artoria2e5]

https://stackoverflow.com/questions/70426330/decrypt-aes-128-ccm-on-nodejs-without-auth-tag reports that the mac_len=4 is actually incorrect: the message contains no MAC at all. I believe this means that the encryption scheme is not regular CCM, but CCM* in ZigBee 802.15.4 Annex B. The 13-byte nonce seems to match.

As a result:

• The correct mac_len should be 0. Of course, this is not allowed in regular CCM, so the library won't accept it; you have to use a wrong one. (It is a mistake to decrypt without verify in CCM too, but sometimes two wrongs make a right haha)
• The update with AAD b'\x11' is unnecessary. An update is only needed to make the MAC match. If there is no MAC to check against, no update is needed.
• Some libraries do not allow you to make the mistake of decrypting without verifying in CCM. For that, the StackOverflow question has an answer: use a different library.

(Also, I don't think you should bother libraries to support a mac_len=0. It's reckless and weird -- it's overall hard to justify.)

The C code with aes_ccm_auth_decrypt works because MBEDTLS allows you to make the same mistake. It gives a return value MBEDTLS_ERR_CCM_AUTH_FAILED, but the decrypted data is already there!