Closed boriszweimueller closed 6 years ago
I’m facing the same problem, looking forward to dig it a little deeper to see what’s the problem here. Also the connection bond itself is not correctly settled somehow, every time I will have to rebound to the device after it disconnects
Same problem here with esp-idf version v3.0.
+1
I didnt test it against latest esp-idf yet, but with arduino code i have no issues. Will try with latest esp-idf now.
It does still not work with latest IDF. I already tried different versions (3.0.1, 3.1rc, master)...
I have to disagree. I have to mention im not using plain esp-idf but im using Neil Kolban's library. Ive got just one single issue with bonding but like i said it was just one time.
My procedure:
E (213466) BT_GATT: GATT_INSUF_ENCRYPTION
D (213611) BLEUtils: Received a GAP event: ESP_GAP_BLE_SEC_REQ_EVT
@boriszweimueller Let me know if im doing something wrong to perform test.
Just in case, here is part of logs when devices getting bonded. Just ignore that part is pointing class and look at part of message from bluedroid (like this: ESP_GAP_BLE_PASSKEY_NOTIF_EVT):
I (11818) BLEDevice: ESP_GAP_BLE_PASSKEY_NOTIF_EVT
I (11823) BLEDevice: passKey = 114885
I (11827) SampleServer: On passkey Notify number:114885
D (11833) BLEServer: BLEServer ... handling GAP event!
D (11960) BLEUtils: Received a GAP event: ESP_GAP_BLE_UPDATE_CONN_PARAMS_EVT
D (11960) BLEUtils: [status: 0, bd_addr: 76:e0:bb:76:56:75, min_int: 0, max_int: 0, latency: 0, conn_int: 6, timeout: 500]
D (11967) BLEServer: BLEServer ... handling GAP event!
W (19873) BT_SMP: FOR LE SC LTK IS USED INSTEAD OF STK
D (19907) BLEUtils: Received a GAP event: ESP_GAP_BLE_KEY_EVT
D (19907) BLEUtils: *** dumpGapEvent: Logger not coded ***
I (19907) BLEDevice: ESP_GAP_BLE_KEY_EVT
I (19912) BLEDevice: key type = ESP_LE_KEY_LENC
D (19917) BLEServer: BLEServer ... handling GAP event!
D (19922) BLEUtils: Received a GAP event: ESP_GAP_BLE_KEY_EVT
D (19928) BLEUtils: *** dumpGapEvent: Logger not coded ***
I (19937) BLEDevice: ESP_GAP_BLE_KEY_EVT
I (19938) BLEDevice: key type = ESP_LE_KEY_PENC
D (19943) BLEServer: BLEServer ... handling GAP event!
D (19949) BLEUtils: Received a GAP event: ESP_GAP_BLE_KEY_EVT
D (19954) BLEUtils: *** dumpGapEvent: Logger not coded ***
I (19960) BLEDevice: ESP_GAP_BLE_KEY_EVT
I (19965) BLEDevice: key type = ESP_LE_KEY_LID
D (19970) BLEServer: BLEServer ... handling GAP event!
D (19975) BLEUtils: Received a GAP event: ESP_GAP_BLE_KEY_EVT
D (19981) BLEUtils: *** dumpGapEvent: Logger not coded ***
I (19986) BLEDevice: ESP_GAP_BLE_KEY_EVT
I (19991) BLEDevice: key type = ESP_LE_KEY_LCSRK
D (19996) BLEServer: BLEServer ... handling GAP event!
D (20003) nvs: nvs_open_from_partition bt_config.conf 1
D (20007) nvs: nvs_set_blob bt_cfg_key0 526
D (20015) nvs: nvs_close 4
D (20016) nvs: nvs_open_from_partition bt_config.conf 1
D (20019) nvs: nvs_set_blob bt_cfg_key0 526
D (20027) nvs: nvs_close 5
D (20027) nvs: nvs_open_from_partition bt_config.conf 1
D (20032) nvs: nvs_set_blob bt_cfg_key0 526
D (20039) nvs: nvs_close 6
D (20040) nvs: nvs_open_from_partition bt_config.conf 1
D (20044) nvs: nvs_set_blob bt_cfg_key0 526
D (20051) nvs: nvs_close 7
D (20052) nvs: nvs_open_from_partition bt_config.conf 1
D (20056) nvs: nvs_set_blob bt_cfg_key0 526
D (20101) nvs: nvs_close 8
D (20101) BLEUtils: Received a GAP event: ESP_GAP_BLE_KEY_EVT
D (20102) BLEUtils: *** dumpGapEvent: Logger not coded ***
I (20104) BLEDevice: ESP_GAP_BLE_KEY_EVT
I (20109) BLEDevice: key type = ESP_LE_KEY_PID
D (20114) BLEServer: BLEServer ... handling GAP event!
D (20119) BLEUtils: Received a GAP event: ESP_GAP_BLE_AUTH_CMPL_EVT
D (20126) BLEUtils: [bd_addr: 76:e0:bb:76:56:75, key_present: 0, key: ***, key_type: 0, success: 1, fail_reason: 0, addr_type: ***, dev_type: ESP_BT_DEVICE_TYPE_BLE]
I (20140) BLEDevice: ESP_GAP_BLE_AUTH_CMPL_EVT
@boriszweimueller I just found something in my logs:
E (11480) BT_GATT: GATT_INSUF_AUTHENTICATION: MITM Required
Its log before bonding and is different than yours.
Hi.
I don't know Neil's library in detail (I know his book though!), but:
I can provide code so you can test it yourself.
@chegewara E (11480) BT_GATT: GATT_INSUF_AUTHENTICATION: MITM Required
This is good. Same here - but it does not work after rebooting..
code would be great. I don't care if you use Neil's lib - I can pick the relevant parts out of it.
@chegewara
The logs I posted above:
I (16062) SEC_GATTS_DEMO: ESP_GATTS_CONNECT_EVT E (18472) BT_GATT: GATT_INSUF_AUTHENTICATION
appear AFTER a reboot. When the devices are bonded and access to protected characteristics should still be possible - but it's not.
Ok, i didnt try with reboot. After reboot i have the same issue on read:
E (13514) BT_GATT: GATT_INSUF_AUTHENTICATION: MITM Required
E (13611) BT_GATT: GATT_INSUF_AUTHENTICATION: MITM Required
E (13709) BT_GATT: GATT_INSUF_AUTHENTICATION: MITM Required
Yes, i can confirm its something wrong (i would say it can be my code if not your issue).
EDIT i just want to mention that there is no more logs that can be reported here
@chegewara: Thanks for the reboot:-)! And thanks for helping and trying this out!
I assume, that there is missing some information in the NVS which specifies that the key has been exchanged/calculated with secure pairing mechanisms (providing MITM then).
Actually the bonding works - but the ESP recognizes the connection as BLE legacy connection instead of a secure connection.
I performed one more test. This time with:
esp_ble_set_encryption(param->connect.remote_bda, ESP_BLE_SEC_ENCRYPT_MITM);
and i got exact same result. But in this case esp32 recognizes bonded android devices and does not request pairing/bonding, only there is issue with insuficient authentication:
D (6779) BLEUtils: Received a GAP event: ESP_GAP_BLE_AUTH_CMPL_EVT
D (6780) BLEUtils: [bd_addr: 50:4a:4d:45:e8:59, key_present: 0, key: ***, key_type: 0, success: 1, fail_reason: 0, addr_type: ***, dev_type: ESP_BT_DEVICE_TYPE_BLE]
yes. the call to esp_ble_set_encryption
does not change anything. It simply tells the stack what it's possibilities are.
Actually I don't understand why this should be necessary. Normally the characteristic permissions define the security constraints which have to be met to access them.
Maybe its small offtopic, just want to explain some thing (at least how i understand it):
Some devices requires to connect with authentication (passkey, yes/no confirm) and it that case you can but you dont have to protect characteristics with this flag ESP_GATT_PERM_WRITE_ENC_MITM
, and some devices allow to connect but requires authentication to read and/or write characteristic.
You can allow to read without authentication but request authentication when you try to write. In that case you wont use esp_ble_set_encryption
.
There is one more thing ive learned very recently. BLE peripherals that could work with iOS devices must to implement security in very special way. You cant use esp_ble_set_encryption
, because device should connect without authentication and characterictics needs to be secured the way you are trying to do (just MITM is not required). At least this solution helped to solve issues with esp32 BLE HID keyboard.
Oh yes, of course - my comment was bad (i had esp_ble_gap_set_security_param
in mind).
With esp_ble_set_encryption
you switch security on. If I remember correctly, the standard defines, that the central should switch security on, not the peripheral. And i think iPhone & Co does this correctly.
My experience is: iOS & Co is very stable (together with esp32) once you have managed to make it work. With android we encounter many issues (spontaneous disconnections all the time).
Hi, I am having the same problem.
I did a test - paired, bonded, disconnected. Connect, pair - log the central info (read works), reset, connect, pair - log the central info (insufficient authentication). In both cases all the information exchanged during the pairing was exactly the same. The keys are maintained during the power off.
There is another case, if we set ESP_LE_AUTH_REQ_SC_MITM (no bonding), but we keep characteristic permissions ESP_GATT_PERM_READ_ENC_MITM. It always gives the insufficient authentication error. Comparing the Central logs of Espressif and Cypress peripherals pairing I see that in the Espressif is missing this message:
[16:09:38:444] : 'Misc Event Notification' received
[16:09:38:444] : Event Code: 0x002C
[16:09:38:444] : Event Data: [0A:2D:F4:65:E3:BD:7B:49:1E:B4:C0:95:95:13:46:73]
I assume it has something to do with encryption or authentication key? However, if we enable bonding, this message will be exchanged. Maybe that is why it works only if the bonding is enabled?
Moreover, even though I set pairing to ESP_LE_AUTH_REQ_SC_MITM and characteristic permissions to ESP_GATT_PERM_READ_ENCRYPTED and ESP_GATT_PERM_WRITE_ENCRYPTED, with this pairing I can read, but if I try to write I am getting insufficient authentication error.
Hi. Thanks for the info. There really seems to be some issues with BLE security.
Athough I can not reproduce your behaviour with the not bonded devices.
Specifying ESP_LE_AUTH_REQ_SC_MITM
in the security setup and ESP_GATT_PERM_READ_ENC_MITM
for the characteristics perfectly works.
Because we have no bonding, the devices need to pair (exchange keys) on every new connection - that's the expected behaviour.
I tested with IDV v3.0.1, with iPhones and Android Phones. Maybe it's different with Cypress pheripherals.
The problem arises as soon as I enable bonding. The keys are stored in NVS and are still available after a reboot - but then I'm getting GATT_INSUF_AUTHENTICATION
.
@linsid An additional important fact:
There is another case, if we set ESP_LE_AUTH_REQ_SC_MITM (no bonding), but we keep characteristic permissions ESP_GATT_PERM_READ_ENC_MITM. It always gives the insufficient authentication error.
Maybe you have to adjust your IOCaps. Because for MITM protection to work, you have your devices to do a secure pairing - e.g. passkey or numeric comparison. So - you have to specify at least ESP_IO_CAP_OUT
. If you for example use ESP_IO_CAP_NONE
your devices end up doing just-works pairing which is not enough for MITM protection.
Thanks for the help, but I am already using:
esp_ble_auth_req_t BLE_SECURITY_AUTH_REQ_MODE = ESP_LE_AUTH_REQ_SC_MITM;
esp_ble_io_cap_t BLE_SECURITY_IO_CAP = ESP_IO_CAP_OUT;
And:
esp_ble_gap_set_security_param(ESP_BLE_SM_AUTHEN_REQ_MODE, &BLE_SECURITY_AUTH_REQ_MODE, sizeof(BLE_SECURITY_AUTH_REQ_MODE));
esp_ble_gap_set_security_param(ESP_BLE_SM_IOCAP_MODE, &BLE_SECURITY_IO_CAP, sizeof(BLE_SECURITY_IO_CAP));
It should be ok for the MITM, but it does not work. Obviously, if it works for you, I am doing something wrong.
I got more details about my problem. After enabling the stack log #define CONFIG_BT_STACK_NO_LOG 0
I got that error:
E (39253) BT: Non bonding: No keys will be exchanged
E (39254) BT: lmp_version_below LMP version 6 < 8
Mr Google says:
LMP | Bluetooth Version |
---|---|
0 | Bluetooth 1.0b |
1 | Bluetooth 1.1 |
2 | Bluetooth 1.2 |
3 | Bluetooth 2.0 + EDR |
4 | Bluetooth 2.1 + EDR |
5 | Bluetooth 3.0 + HS |
6 | Bluetooth 4.0 |
7 | Bluetooth 4.1 |
8 | Bluetooth 4.2 |
9 | Bluetooth 5 |
That means Espressif, which is v4.2, will not talk to v4.0?
I dont know if that is fixed already, but this message was also when device is BLE 5.0
E (39254) BT: lmp_version_below LMP version 6 < 8
You can google about SMP in ble 4.0 devices and here: https://github.com/espressif/esp-idf/issues ive got a lot discussions about security and bonding with android smartphones BLE 4.0. Always answer was: SMP was introduced in BLE 4.2
Hmm. I can find definition of SMP in specification of V4.0 Vol 3 Part H 3. Anyway, I tested with 4.1 and 5 and you are right - it allways gives that message.
This message also does not mean error if you are not using bonding:
E (39253) BT: Non bonding: No keys will be exchanged
and from your post https://github.com/espressif/esp-idf/issues/2036#issuecomment-401697131 we can see you dont use bonding.
After a bit more of investigations I found that:
If I am using ESP_LE_AUTH_REQ_SC_MITM
, after pairing sec_flags=0x1680
.
If I am using ESP_LE_AUTH_REQ_SC_MITM_BOND
, after pairing sec_flags=0x3680
If I am using ESP_LE_AUTH_REQ_SC_MITM_BOND
, after bonding and processor reset sec_flags=0x1680
So the difference is this flag:
#define BTM_SEC_LE_AUTHENTICATED 0x0200 /* LE link is encrypted after pairing with MITM */
#define BTM_SEC_LE_LINK_KEY_AUTHED 0x2000 /* pairing is done with MITM */
Inside BTM_GetSecurityFlagsByTransport
function the flag is shifted (p_dev_rec->sec_flags >> 8)
and later on, when I am trying to read the characteristic gatts_check_attr_readability
function checks the security flag:
if ((perm & GATT_READ_MITM_REQUIRED ) && !(sec_flag & GATT_SEC_FLAG_LKEY_AUTHED)) {
Which means:
#define GATT_SEC_FLAG_LKEY_AUTHED BTM_SEC_FLAG_LKEY_AUTHED
#define BTM_SEC_FLAG_LKEY_AUTHED 0x20
To me, reading the comments, the meaning of BTM_SEC_LE_AUTHENTICATED
and BTM_SEC_LE_LINK_KEY_AUTHED
sounds very similar. But it makes a big difference in operation.
To get BTM_SEC_LE_AUTHENTICATED
the function btm_sec_encrypt_change
is called.
To get BTM_SEC_LE_LINK_KEY_AUTHED
the function btm_sec_save_le_key
is called.
Thanks for this investigation! Actually I think during paring & bonding, the keys should be persisted in the NVS (btm_sec_save_le_key), and later on - when two bonded devices reconnect - the keys should be read form NVS; and in this case, the stack has to recognize, that the pairing has been done with MITM protection.
As everything worked with IDF version 3.0RC, maybe we could try to compare where the 3.0 RC and the current version behave differently.
@Yulong-espressif @blueMoodBHD Could you help here?
Hi all: I am sorry, it is really a bug, I'm fixing the bug in master and v3.1. By the way, it works well in master and v3.1 for Android phones. Now iPhones have a bug, I am fixing it, thanks.
@Weijian-Espressif Thank you!
Thank you!
I dont know if its the same issue, because settings are completely different. I have HID device with ESP_LE_AUTH_REQ_SC_BOND
, ESP_IO_CAP_NONE
and no esp_ble_set_encryption
setup. My esp32 is sending emulated message after 10 seconds when connect with client.
When esp32 or peer device is resetting then i have two different behaviors:
I (211) BTDM_INIT: BT controller compile version [b86be39]
I (213) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE I (281) phy: phy_version: 3910, c0c45a3, May 21 2018, 18:07:06, 0, 0 I (457) BT_BTM: btm_sec_set_security_level : sec: 0x0
I (458) BT_BTM: BTM_SEC_REG[0]: id 50, is_orig 1, psm 0x001f, proto_id 0, chan_id 0
I (461) BT_BTM: : sec: 0x80, service name [] (up to 21 chars saved)
I (469) BT_BTM: btm_sec_set_security_level : sec: 0x0
I (475) BT_BTM: BTM_SEC_REG[0]: id 50, is_orig 0, psm 0x001f, proto_id 0, chan_id 0
I (484) BT_BTM: : sec: 0x80, service name [] (up to 21 chars saved)
I (492) BT_GATT: GATT_Register I (496) BT_GATT: allocated gatt_if=1
I (500) BT_GATT: GATT_StartIf gatt_if=1 I (505) BT_GATT: GATTS_CreateService
I (510) BT_GATT: GATTS_StartService I (514) BT_GATT: GATT_Register I (517) BT_GATT: allocated gatt_if=2
I (521) BT_GATT: GATT_StartIf gatt_if=2 I (526) BT_GATT: GATTS_CreateService
I (531) BT_GATT: GATTS_StartService I (603) BT_BTM: BTM_SetPinType: pin type 0 [variable-0, fixed-1], code , length 0
I (604) BT_BTM: btm_sec_set_security_level : sec: 0x0
I (607) BT_BTM: BTM_SEC_REG[1]: id 42, is_orig 0, psm 0x0003, proto_id 3, chan_id 0
I (615) BT_BTM: : sec: 0x80, service name [RFC_MUX] (up to 21 chars saved)
I (624) BT_BTM: BTM_SetInquiryMode
I (629) BT_BTM: BTM_SetPageScanType
I (633) BT_BTM: BTM_SetInquiryScanType
I (643) BT_BTM: BTM_SecAddDevice, link key type:4
I (645) BT_BTM: BTM_InqDbRead: bd addr [80a58982f5d2]
I (654) BT_BTM: BTM_InqDbRead: bd addr [781fdbc0c7f3]
I (661) BT_BTM: BTM_InqDbRead: bd addr [61c02d673cc9]
I (669) BT_BTM: Write Extended Inquiry Response to controller
I (672) BT_BTM: Write Extended Inquiry Response to controller
I (874) BT_GATT: GATT_Register I (875) BT_GATT: allocated gatt_if=3
I (876) BT_GATT: GATT_StartIf gatt_if=3 I (879) BT_GATT: GATTS_CreateService
I (885) BT_GATT: GATTS_CreateService
I (888) BT_GATT: GATTS_CreateService
I (908) BT_GATT: GATTS_StartService I (927) BT_GATT: GATTS_StartService I (937) BT_GATT: GATTS_StartService I (943) BT_APPL: BTA_DmSetBleAdvParamsAll: 32, 64
I (943) BT_APPL: adv_type = 0, addr_type_own = 0, chnl_map = 7, adv_fil_pol = 0
I (1875) BT_GATT: GATT_GetConnIdIfConnected status=1
I (1877) BT_L2CAP: L2CA_SetDesireRole() new:x1, disallow_switch:0
Im saying its partially reconnection because there is no message sent at all.
Here is normal reconnecting after peer device reset connection:
I (30520) BT_GATT: GATT_GetConnIdIfConnected status=0
I (30522) BT_L2CAP: L2CA_SetDesireRole() new:x1, disallow_switch:0 I (30530) BT_APPL: BTA_DmSetBleAdvParamsAll: 32, 64
I (30531) BT_APPL: adv_type = 0, addr_type_own = 0, chnl_map = 7, adv_fil_pol = 0
I (34239) BT_GATT: GATT_GetConnIdIfConnected status=1
I (34240) BT_L2CAP: L2CA_SetDesireRole() new:x1, disallow_switch:0 W (34244) Task: Task::start - There might be a task already running! I (35893) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (35896) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 1 Status: 0x0000
I (35897) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (36133) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (36136) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 2 Status: 0x0000
I (36136) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (36373) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (36376) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 3 Status: 0x0000
I (36376) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (36613) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (36616) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 4 Status: 0x0000
I (36617) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (36853) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (36856) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 5 Status: 0x0000
I (36857) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (37093) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (37095) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 6 Status: 0x0000
I (37096) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (37333) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (37336) BT_GATT: GATTS_SendRsp: conn_id: 3 trans_id: 7 Status: 0x0000
I (37336) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9 I (44250) BT_GATT: GATT_GetConnectionInfor conn_id=3 I (44250) BT_GATT: GATTS_HandleValueNotification I (44250) BT_L2CAP: L2CA_SendFixedChnlData() CID: 0x0004 BDA: 61c02d673cc9
Hi all: we have fixed the bug, the commit id is 163dbc32 , please update to the latest IDF and retry, thanks.
@Weijian-Espressif: Thank you very much for the fix.
I did a short test with the latest IDF with MITM and bonding and it works. I tested with an iPhone 6, a Huawei P20 and an OSX BLE client.
I will later do some more testing with more devices. I think we can close the issue for now.
Thank you!
Environment
git rev-parse --short HEAD
to get the commit id.): b2ff235bProblem Description
Example Code: examples/bluetooth/gatt_security_server
Client Device: Various iPhones (6,6+,7,8,X) and Android. Same behaviour with ALL of these devices.
If I change the code to use passkey pairing with MITM:
Everything works as expected. After Rebooting the ESP, my client device is still bonded and can connect/read/write without another pairing.
HOWEVER:
If I additionally change the security permissions of the characteristics to 'xxx_ENC_MITM' e.g.
On the first connect, passkey pairing (le secure) is triggered as expected, and the characteristic can afterwards be accessed.
But then, after rebooting the ESP, the client device CAN NOT access those characteristics any more.
The connection works, but somehow, the connection does not have the MITM flag set and therefore there is an auth error.
The lowlevel error ist:
This worked perfectly with release V2 and release V3 RC1.
I checked the connection with OSX' packet-logger and it seems the connection does really use MITM and is encrypted. (This is due to the fact, that security is switched on in the GATT CONNECT handler in the example)
Does anybody know/can confirm if this is a bug?
Thank you very much! Boris