jlusiardi
commented
5 years ago I guess there are a few points unclear to me:
- How does python work with bluetooth LE?
- How to discover Accessories?
- How to communicate with the accessories?
Closed der-punkt closed 5 years ago
jlusiardi
commented
5 years ago I guess there are a few points unclear to me:
der-punkt
commented
5 years ago Hi jlusiardi,
first of all, sorry, it took me a whole week to write back.
- How does python work with bluetooth LE?
I found that there is a Bluetooth LE library for python: https://github.com/IanHarvey/bluepy
- How to discover Accessories?
Here is some general code for discovering BTLE devices:
from bluepy.btle import Scanner, DefaultDelegate
class ScanDelegate(DefaultDelegate):
def __init__(self):
DefaultDelegate.__init__(self)
def handleDiscovery(self, dev, isNewDev, isNewData):
if isNewDev:
print "Discovered device", dev.addr
elif isNewData:
print "Received new data from", dev.addr
scanner = Scanner().withDelegate(ScanDelegate())
devices = scanner.scan(10.0)
for dev in devices:
print "Device %s (%s), RSSI=%d dB" % (dev.addr, dev.addrType, dev.rssi)
for (adtype, desc, value) in dev.getScanData():
print " %s = %s" % (desc, value)
- How to communicate with the accessories?
I had chance to take only a short look at the HAP specification. However, this may be a good starting point:
from bluepy.btle import Peripheral, ADDR_TYPE_RANDOM
addr = "xx:xx:xx:xx:xx:xx"
dev = Peripheral(addr, ADDR_TYPE_RANDOM)
srvc_pairing = "00000055-0000-1000-8000-0026BB765291"
ch_pair_setup = "0000004C-0000-1000-8000-0026BB765291"
ch_pair_verify = "0000004E-0000-1000-8000-0026BB765291"
ch_pairing_features = "0000004F-0000-1000-8000-0026BB765291"
ch_pairing_pairings = "00000050-0000-1000-8000-0026BB765291"
service = dev.getServiceByUUID(srvc_pairing)
print "Service: ", service.uuid.getCommonName()
for characteristic in service.getCharacteristics():
print " Characteristic: ", characteristic.uuid.getCommonName()
print " Properties: ", characteristic.propertiesToString()
if characteristic.supportsRead():
print " Value: ", (characteristic.read(),)All the best!
jlusiardi
commented
5 years ago Hey,
that helps a lot, thanks! I already started tinkering with gatt-python, but i will also consider bluepy. Both seem to have one downside: linux with bluez only? Lets have a look :)
Regards Joachim
jlusiardi
commented
5 years ago Hey again,
https://github.com/markrages/ble states that bluepy uses the hci socket interface to bluez. This works well but it is deprecated and requires root access.
So this doesn't look so good :/
jlusiardi
commented
5 years ago Also interesting: which library allows access to the advertisement packages?
jlusiardi
commented
5 years ago Success so far (with some pretty ugly hacks):
Using subprocess to use hcitool & hcidump
Name: Koogeek-DW1-8ca86c
MAC: e0:44:6a:b8:05:e0
Configuration number (c#): 1
Device ID (id): da9ae07f2e4e
Compatible Version (cv): 2
State Number (s#): 2
Status Flags (sf): unpaired
Category Identifier (ci): (Id: 10)1: >accessory-information<
1.4: () >manufacturer< [pr]
1.14: () >firmware.revision< [pr]
1.3: () >identify< [pw]
1.6: () >serial-number< [pr]
1.2: () >name< [pr]
1.15: () >hardware.revision< [pr]
1.5: () >model< [pr]
None: >Unknown Service: 00001801-0000-1000-8000-00805F9B34FB<
1003: >Unknown Service: 4AAAF961-0DEC-11E5-B939-0800200C9A66<
1003.1005: (Time_zone) >Unknown Characteristic 4AAAF963-0DEC-11E5-B939-0800200C9A66< [pr,pw]
1003.1004: (Time_value) >Unknown Characteristic 4AAAF962-0DEC-11E5-B939-0800200C9A66< [pr,pw]
91: >contact<
91.1002: (One_hundred_data) >Unknown Characteristic 4AAAF966-0DEC-11E5-B939-0800200C9A66< [pr]
91.1001: (Thirday_one_day_data) >Unknown Characteristic 4AAAF965-0DEC-11E5-B939-0800200C9A66< [pr]
91.93: (Contact_sensor_state) >contact-state< [pr,evc,evd]
91.1000: (Today_data) >Unknown Characteristic 4AAAF964-0DEC-11E5-B939-0800200C9A66< [pr]
91.98: (Name) >name< [pr]
7: >service<
7.60000: () >service-signature< [pr]
7.8: () >version< [pr]
9: >pairing<
9.13: () >pairing.pairings< [pr,pw]
9.10: () >pairing.pair-setup< [r,w]
9.12: () >pairing.features< [r]
9.11: () >pairing.pair-verify< [r,w]
900: >Unknown Service: 00001530-1212-EFDE-1523-785FEABCD123<
900.901: (DFU Control Point) >Unknown Characteristic 00001531-1212-EFDE-1523-785FEABCD123< [pw,hd]
900.902: (Service Signature) >service-signature< [r,pr,hd]
57: >battery<
57.60: (Charging_state) >charging-state< [pr,evc,evd]
57.59: (Battery_level) >battery-level< [pr,evc,evd]
57.61: (Status_low_battery) >status-lo-batt< [pr,evc,evd]Looking good! Can I contribute with something?
jlusiardi
commented
5 years ago Hey,
you could test the scripts from branch add_bluetooth from the staging directory:
The first should try to discover all BLE HomeKit devices:
sudo python3 staging/homekit_discovery.pyE.g.:
Name: Koogeek-DW1-8ca86c
MAC: e0:44:6a:b8:05:e0
Configuration number (c#): 1
Device ID (id): da9ae07f2e4e
Compatible Version (cv): 2
State Number (s#): 2
Status Flags (sf): unpaired
Category Identifier (ci): (Id: 10)The second is called with a mac address:
python3 staging/hap_char_sig_read.py e0:44:6a:b8:05:e0Would be cool to get some input data and feedback!
Regards Joachim
der-punkt
commented
5 years ago Hi Joachim,
thank you for the response. I tried them out with the following results:
$ sudo python3 staging/homekit_discovery.pySet scan parameters failed: Input/output error
Name: A60 Ce
MAC: c8:ea:1b:6f:b2:10
Configuration number (c#): 1
Device ID (id): 6f21816f8519
Compatible Version (cv): 2
State Number (s#): 1
Status Flags (sf): unpaired
Category Identifier (ci): (Id: 5)but $ python3 staging/hap_char_sig_read.py c8:ea:1b:6f:b2:10 doesn't produce any output it is just stuck somewhere in manager.run().
Best regards, Vasil
jlusiardi
commented
5 years ago Hey Vasil,
is this some Osram Smart+ product?
It does not even print resolved? That's strange. Some more questions:
bluetoothctl in a second terminal? For me this results in:
[CHG] Device E0:44:6A:B8:05:E0 Connected: yes
[CHG] Device E0:44:6A:B8:05:E0 ServicesResolved: yes
[CHG] Device E0:44:6A:B8:05:E0 ServicesResolved: no
[CHG] Device E0:44:6A:B8:05:E0 Connected: noThanks for the support!
Joachim
der-punkt
commented
5 years ago Hi Joachim,
yes, it is a Osram Smart+ LED light bulb.
On the side: Interestingly, after I paired the device and your script homekit_discovery.py got this status also right, but bluetoothhctl still shows it as unpaired.
I am also running bluez 5.43.
Below is the test run in bluetoothhctl (PS Sorry for the long log):
Thank you for spending time on this!
Vasil
jlusiardi
commented
5 years ago Hi,
my script prints the homekit pairing status not the bluetooth pairing (as in bluetoothctrl) status.
The bluetoothctrl looks ok wondering...
Regards Joachim
jlusiardi
commented
5 years ago Hey Vasil,
I think we had a misunderstanding. Sorry :/
Pushed some changes to hap_char_sig_read.py.
I meant: can you start bluetoothctl in one terminal, do nothing with it and run PYTHONPATH=. python3 staging/hap_char_sig_read.py c8:ea:1b:6f:b2:10 --log=DEBUG in an other terminal and send me this output?
Regards Joachim
der-punkt
commented
5 years ago Hey Joachim,
Sorry I was offline so long :/
I understand now :)
All the best Vasil
jlusiardi
commented
5 years ago Hey Vasil,
can u give it another try with the new code an post back the debug log?
I ordered one of the bulbs myself, too.
Joachim
der-punkt
commented
5 years ago Hi Joachim,
Nice, congrats on the new bulb 😉
Unfortunately, I’m on a business trip. I’ll be able to try some thing not before next week. Sorry. 😐
Best wishes Vasil
Am 19.11.2018 um 22:10 schrieb Joachim Lusiardi notifications@github.com:
Hey Vasil,
can u give it another try with the new code an post back the debug log?
I ordered one of the bulbs myself, too.
Joachim
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
jlusiardi
commented
5 years ago Hey Vasil,
Bulb arrived, and i was able to do some improvements:
10: >service< (000000A2-0000-1000-8000-0026BB765291)
10.11: () >version< [pr]
1: >accessory-information< (0000003E-0000-1000-8000-0026BB765291)
1.3: () >manufacturer< [pr]
1.7: () >firmware.revision< [pr]
1.2: () >identify< [w]
1.5: () >name< [pr]
1.4: () >model< [pr]
1.6: () >serial-number< [pr]
30: >pairing< (00000055-0000-1000-8000-0026BB765291)
30.33: () >pairing.features< [r]
30.31: () >pairing.pair-setup< [r,w]
30.34: () >pairing.pairings< [pr,pw]
30.32: () >pairing.pair-verify< [r,w]
20: >lightbulb< (00000043-0000-1000-8000-0026BB765291)
20.15: () >on< [pr,pw,evc,evd]
20.19: () >service-signature< [r]
20.25: () >saturation< [pr,pw,evc,evd]
20.24: () >hue< [pr,pw,evc,evd]
20.16: () >name< [pr]
20.23: () >brightness< [pr,pw,evc,evd]
1000: >Unknown Service: B0733E83-8434-4C00-A344-25D1C982A0EF< (B0733E83-8434-4C00-A344-25D1C982A0EF)
1000.1001: () >service-signature< [r]
1000.1003: () >Unknown Characteristic B2FD7F2D-EAD3-4F17-B16C-202EC758C697< [pr,pw]
1000.1004: () >Unknown Characteristic B31259A5-9ACC-45C2-838A-956F57825196< [pr]
1000.1002: () >Unknown Characteristic B176BD7F-4148-47BD-A6C6-9D0796E96183< [pr,pw,evc,evd]Try to check when you have the time ;)
Joachim
Jc2k
commented
5 years ago Hey, very interested in this branch. Heres what my eve energy looks like:
sudo ./staging/homekit_discovery.py
Name: Eve Energy XXXX
MAC: c6:87:8d:xx:xx:xx
Configuration number (c#): 2
Device ID (id): xxxxxxxxxxxx
Compatible Version (cv): 2
State Number (s#): 1009
Status Flags (sf): unpaired
Category Identifier (ci): (Id: 7)john@ubuntu:~/homekit_python$ sudo ./staging/hap_char_sig_read.py --log=DEBUG c6:87:8d:xx:xx:xx
2018-11-28 02:44:58,651 hap_char_sig_read.py:0214 DEBUG Running version 20181122
2018-11-28 02:44:58,652 hap_char_sig_read.py:0215 DEBUG using adapter hci0
2018-11-28 02:44:59,604 hap_char_sig_read.py:0131 DEBUG discovered c6:87:8d:xx:xx:xx
2018-11-28 02:44:59,605 hap_char_sig_read.py:0219 DEBUG discovered
2018-11-28 02:44:59,608 hap_char_sig_read.py:0223 DEBUG connecting to device
2018-11-28 02:45:00,788 hap_char_sig_read.py:0225 DEBUG connected to device
2018-11-28 02:45:00,788 hap_char_sig_read.py:0228 DEBUG start managerAnd it never returns
And if i try to pair:
$ sudo ./staging/hap_pair.py --log=DEBUG -p xxxxxxxx c6:87:8d:xx:xx:xx
2018-11-28 02:46:59,593 hap_pair.py:103 DEBUG using adapter hci0
2018-11-28 02:47:00,543 hap_pair.py:23 DEBUG discovered c6:87:8d:xx:xx:xx
2018-11-28 02:47:00,549 hap_pair.py:110 DEBUG connecting to device
2018-11-28 02:47:01,477 hap_pair.py:112 DEBUG connected to deviceAnd it never returns.
Running bluetoothctl in another terminal i see lots of RSSI measurements. And a few seconds after the script has connected to my device, i see a Connected: no fly by.
Jc2k
commented
5 years ago Added a:
def connect_failed(self, error):
super().connect_failed(error)
print("[%s] Connection failed: %s" % (self.mac_address, str(error)))And got:
[c6:87:8d:xx:xx:xx] Connection failed: No such property 'ServicesResolved'Which lead me to this. But I can see the device is discovered so I don't think its that. Looking at the docs, bluez must be >= 5.38. I used xenial vm i already had lying around, so actually only on 5.37. Upgrading.
Jc2k
commented
5 years ago Ok now it works:
-------------------- human readable --------------------
28: >outlet< (00000047-0000-1000-8000-0026BB765291)
28.36: () >service-signature< [pr]
28.35: () >Unknown Characteristic E863F10C-079E-48FF-8F27-9C2605A29F52< [pr]
28.34: () >Unknown Characteristic E863F10D-079E-48FF-8F27-9C2605A29F52< [pr,evc,evd]
28.33: () >Unknown Characteristic E863F126-079E-48FF-8F27-9C2605A29F52< [pr]
28.32: () >Unknown Characteristic E863F10A-079E-48FF-8F27-9C2605A29F52< [pr]
28.31: () >outlet-in-use< [pr,evc,evd]
28.30: () >on< [pr,pw,evc,evd]
28.29: () >name< [pr]
16: >Unknown Service: E863F007-079E-48FF-8F27-9C2605A29F52< (E863F007-079E-48FF-8F27-9C2605A29F52)
16.27: () >service-signature< [pr]
16.26: () >Unknown Characteristic E863F11D-079E-48FF-8F27-9C2605A29F52< [pw]
16.25: () >Unknown Characteristic E863F131-079E-48FF-8F27-9C2605A29F52< [pr]
16.24: () >Unknown Characteristic E863F117-079E-48FF-8F27-9C2605A29F52< [pr]
16.23: () >Unknown Characteristic E863F116-079E-48FF-8F27-9C2605A29F52< [pr]
16.22: () >Unknown Characteristic E863F121-079E-48FF-8F27-9C2605A29F52< [pw]
16.21: () >Unknown Characteristic E863F11C-079E-48FF-8F27-9C2605A29F52< [pw]
16.20: () >Unknown Characteristic E863F112-079E-48FF-8F27-9C2605A29F52< [pr,pw]
16.19: () >Unknown Characteristic E863F11E-079E-48FF-8F27-9C2605A29F52< [pr,pw]
16.17: () >name< [pr]
11: >pairing< (00000055-0000-1000-8000-0026BB765291)
11.15: () >pairing.pairings< [pr,pw]
11.14: () >pairing.features< [r]
11.13: () >pairing.pair-verify< [r,w]
11.12: () >pairing.pair-setup< [r,w]
9: >service< (000000A2-0000-1000-8000-0026BB765291)
9.37: () >service-signature< [pr]
9.10: () >version< [pr]
1: >accessory-information< (0000003E-0000-1000-8000-0026BB765291)
1.8: () >hardware.revision< [pr]
1.7: () >firmware.revision< [pr]
1.6: () >serial-number< [pr]
1.5: () >model< [pr]
1.4: () >manufacturer< [pr]
1.3: () >identify< [pw]
1.2: () >name< [pr]
-------------------- json --------------------
[
{
"aid": 1,
"services": [
{
"type": "00000047-0000-1000-8000-0026BB765291",
"iid": 28,
"characteristics": [
{
"type": "000000a5-0000-1000-8000-0026bb765291",
"description": "",
"aid": 28,
"iid": 36,
"value": "",
"perms": [
"pr"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f10c-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 28,
"iid": 35,
"value": "",
"perms": [
"pr"
],
"format": "float",
"unit": "unitless",
"range": "(0.0, inf)",
"step": "None"
},
{
"type": "e863f10d-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 28,
"iid": 34,
"value": "",
"perms": [
"pr",
"evc",
"evd"
],
"format": "float",
"unit": "unitless",
"range": "(0.0, 5000.0)",
"step": "None"
},
{
"type": "e863f126-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 28,
"iid": 33,
"value": "",
"perms": [
"pr"
],
"format": "float",
"unit": "unitless",
"range": "(0.0, 16.0)",
"step": "None"
},
{
"type": "e863f10a-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 28,
"iid": 32,
"value": "",
"perms": [
"pr"
],
"format": "float",
"unit": "unitless",
"range": "(0.0, 380.0)",
"step": "None"
},
{
"type": "00000026-0000-1000-8000-0026bb765291",
"description": "",
"aid": 28,
"iid": 31,
"value": "",
"perms": [
"pr",
"evc",
"evd"
],
"format": "bool",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000025-0000-1000-8000-0026bb765291",
"description": "",
"aid": 28,
"iid": 30,
"value": "",
"perms": [
"pr",
"pw",
"evc",
"evd"
],
"format": "bool",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000023-0000-1000-8000-0026bb765291",
"description": "",
"aid": 28,
"iid": 29,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
}
]
},
{
"type": "E863F007-079E-48FF-8F27-9C2605A29F52",
"iid": 16,
"characteristics": [
{
"type": "000000a5-0000-1000-8000-0026bb765291",
"description": "",
"aid": 16,
"iid": 27,
"value": "",
"perms": [
"pr"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f11d-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 26,
"value": "",
"perms": [
"pw"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f131-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 25,
"value": "",
"perms": [
"pr"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f117-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 24,
"value": "",
"perms": [
"pr"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f116-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 23,
"value": "",
"perms": [
"pr"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f121-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 22,
"value": "",
"perms": [
"pw"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f11c-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 21,
"value": "",
"perms": [
"pw"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "e863f112-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 20,
"value": "",
"perms": [
"pr",
"pw"
],
"format": "uint32",
"unit": "unitless",
"range": "(None, None)",
"step": "None"
},
{
"type": "e863f11e-079e-48ff-8f27-9c2605a29f52",
"description": "",
"aid": 16,
"iid": 19,
"value": "",
"perms": [
"pr",
"pw"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000023-0000-1000-8000-0026bb765291",
"description": "",
"aid": 16,
"iid": 17,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
}
]
},
{
"type": "00000055-0000-1000-8000-0026BB765291",
"iid": 11,
"characteristics": [
{
"type": "00000050-0000-1000-8000-0026bb765291",
"description": "",
"aid": 11,
"iid": 15,
"value": "",
"perms": [
"pr",
"pw"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "0000004f-0000-1000-8000-0026bb765291",
"description": "",
"aid": 11,
"iid": 14,
"value": "",
"perms": [
"r"
],
"format": "uint8",
"unit": "unitless",
"range": "(0, 1)",
"step": "1"
},
{
"type": "0000004e-0000-1000-8000-0026bb765291",
"description": "",
"aid": 11,
"iid": 13,
"value": "",
"perms": [
"r",
"w"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "0000004c-0000-1000-8000-0026bb765291",
"description": "",
"aid": 11,
"iid": 12,
"value": "",
"perms": [
"r",
"w"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
}
]
},
{
"type": "000000A2-0000-1000-8000-0026BB765291",
"iid": 9,
"characteristics": [
{
"type": "000000a5-0000-1000-8000-0026bb765291",
"description": "",
"aid": 9,
"iid": 37,
"value": "",
"perms": [
"pr"
],
"format": "data",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000037-0000-1000-8000-0026bb765291",
"description": "",
"aid": 9,
"iid": 10,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
}
]
},
{
"type": "0000003E-0000-1000-8000-0026BB765291",
"iid": 1,
"characteristics": [
{
"type": "00000053-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 8,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000052-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 7,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000030-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 6,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000021-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 5,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000020-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 4,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000014-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 3,
"value": "",
"perms": [
"pw"
],
"format": "bool",
"unit": "unitless",
"range": "None",
"step": "None"
},
{
"type": "00000023-0000-1000-8000-0026bb765291",
"description": "",
"aid": 1,
"iid": 2,
"value": "",
"perms": [
"pr"
],
"format": "string",
"unit": "unitless",
"range": "None",
"step": "None"
}
]
}
]
}
]Pairing fails:
2018-11-28 03:38:23,751 hap_pair.py:110 DEBUG connecting to device
2018-11-28 03:38:24,542 hap_pair.py:112 DEBUG connected to device
2018-11-28 03:38:25,378 hap_pair.py:31 DEBUG resolved 6 services
2018-11-28 03:38:25,386 hap_pair.py:119 DEBUG Pairing Service: <gatt.gatt_linux.Service object at 0x7fb73df8a160>
2018-11-28 03:38:25,386 hap_pair.py:124 DEBUG char: e604e95d-a759-4817-87d3-aa005083a0d1 protocol.service-id
2018-11-28 03:38:25,387 hap_pair.py:124 DEBUG char: 00000050-0000-1000-8000-0026bb765291 pairing.pairings
2018-11-28 03:38:25,387 hap_pair.py:124 DEBUG char: 0000004f-0000-1000-8000-0026bb765291 pairing.features
2018-11-28 03:38:25,387 hap_pair.py:124 DEBUG char: 0000004e-0000-1000-8000-0026bb765291 pairing.pair-verify
2018-11-28 03:38:25,387 hap_pair.py:124 DEBUG char: 0000004c-0000-1000-8000-0026bb765291 pairing.pair-setup
2018-11-28 03:38:25,388 hap_pair.py:131 DEBUG setup char: <gatt.gatt_linux.Characteristic object at 0x7fb73b42eb70> <__main__.AnyDevice object at 0x7fb73bce5fd0>
2018-11-28 03:38:25,388 hap_pair.py:132 DEBUG verify char: <gatt.gatt_linux.Characteristic object at 0x7fb73b42edd8>
2018-11-28 03:38:25,388 __init__.py:89 DEBUG #1 ios -> accessory: send SRP start request
2018-11-28 03:38:25,389 hap_pair.py:47 DEBUG entering write function [[6, bytearray(b'\x01')], [0, bytearray(b'\x01')]]
2018-11-28 03:38:25,389 hap_pair.py:57 DEBUG sent 0002d00a000b000901010106060101000101
2018-11-28 03:38:26,391 hap_pair.py:62 DEBUG reading characteristic
2018-11-28 03:38:26,641 hap_pair.py:69 DEBUG control field: 2, tid: d0, status: 4, length: 0
2018-11-28 03:38:26,641 hap_pair.py:77 DEBUG received 02d004
2018-11-28 03:38:26,641 hap_pair.py:78 DEBUG decode
Traceback (most recent call last):
File "./staging/hap_pair.py", line 138, in <module>
pairing = perform_pair_setup(args.pin, str(uuid.uuid4()), write_fun)
File "/home/john/homekit_python/homekit/protocol/__init__.py", line 96, in perform_pair_setup
response_tlv = write_fun(request_tlv, step2_expectations)
File "./staging/hap_pair.py", line 80, in write
result = TLV.decode_bytes(resp_tlv[0][1], expected)
IndexError: list index out of rangeHey @Jc2k ,
cool that this is working at least for the hap_char_sig_read.py!
I am still working on this but I like your support and will try to get back to the pairing problem soon.
Regards Joachim
jlusiardi
commented
5 years ago Hey @Jc2k can you give it another try after pulling? And please always give the full command line and if a script does not work, enable logging like you already did. Thanks! Joachim
Jc2k
commented
5 years ago Looking good!
So first of all, discovery:
Name: Eve Energy XXXX
MAC: C6:87:8D:XX:XX:XX
Configuration number (c#): 2
Device ID (id): XX:XX:XX:XX:XX:XX
Compatible Version (cv): 2
State Number (s#): 1009
Status Flags (sf): unpaired
Category Identifier (ci): Outlet (Id: 7)It's now showing as an Outlet!
At a glance the hap_char_sig_read data looks the same.
Unfortunately hap_pair now fails with an import error. The BlePairing class doesnt seem to exist in homekit.controller.
Jc2k
commented
5 years ago I justed used homekit.controller.Pairing and the script now finishes successfully. Device now shows as paired in homekit discovery:
Status Flags (sf): pairedI have a valid looking JSON file from the pairing process.
jlusiardi
commented
5 years ago yeah I forgot to commit/push properly :/
Jc2k
commented
5 years ago Hey - was thinking of having a go at adding a BLEPairing.get_characteristics but wanted to make sure I wasn’t duplicating effort. And it sounds like you might have a partial BLEPairing 🙃
jlusiardi
commented
5 years ago Hey,
contributions welcome, I am still stuck on how to apply session security on the BLE functions.
What exactly to u mean with partial pairing?
Regards Joachim
Jc2k
commented
5 years ago Oh sorry - replied on my phone and wasn't very clear.
I assumed you had some code already for the BLEPairing class that wasn't committed (because the pairing code references it, but it isnt in git). I said partial because I assumed it wasn't finished yet.
The docs around session security for BLE are a bit hard to follow. It wasn't clear to me whether you encrypted the entire PDU or just the PDU body. I think it means just the body, and that the length is the length of the encrypted body and the auth tag.
If you didn't have any more code to share I was going to start out by adding a BlePairing/BleSession and try to write something generic to do a Request/Response with session security. But if you have something part written that I can play with debugging that would save me some time this evening. New to BLE and HAP so might end in disaster!
jlusiardi
commented
5 years ago No Problem!
I will give a quick look to commit & push the latest code now :) I am still working on some kind of working implementation of a BLE version of pair verify (that part works) and the read out of the device name via the name service of the Koogeek DW1:
...
1: >accessory-information< (0000003E-0000-1000-8000-0026BB765291)
...
1.2: () >name< [pr]
...The code is is hap_pair_verify.py. I hope i did not forget anything this time ;)
Jc2k
commented
5 years ago Didn't have as much time to look at this last night as i'd hope.
I think i'm currently in the same position as you - accessory is rejecting my Read-Request packets. From reading the spec everything looks good, but I just get GATT errors.
I found this: https://github.com/grover/homebridge-ranger. It proxies IP homekit to bluetooth homekit accessories (via homebridge). It just seems to corobarate what you are doing session security wise. The crypto is here. It's invoked around here.
The only thing that pops out at me is that the buildPacket function always writes a payload length. But that doesn't match the spec or what homekit_python does, and doesn't fix my GATT errors.
I'm fighting with node now to see if I can see if it can pair with my Eve Energy, but its node build pain all the way down...
jlusiardi
commented
5 years ago hap_read_pairingfeatures.py allows to read a characteristic without pairing for now. Perhaps it gives clues...
Jc2k
commented
5 years ago If you try to read a characteristic do you get a valid HAP reply with a known BLE status code?
I managed to get a little closer to a working homebridge-ranger. Its not working yet but I have seen it return HAP BLE status codes for the read characteristic op. Even sending the read request with no encryption, my device gives me a valid response packet (with status 5).
But I can't get the same with homekit_python so far.
feature_char, feature_char_id = find_characteristic(
device,
ServicesTypes['public.hap.service.outlet'],
CharacteristicsTypes.ON)
if not feature_char:
print('features characteristic not found')
sys.exit(-1)
transaction_id = random.randrange(0, 255)
transaction_id = 0xff
data = bytearray([0x00, HapBleOpCodes.CHAR_READ, transaction_id])
data.extend(feature_char_id.to_bytes(length=2, byteorder='little'))
# to match homebridge-ranger, probably not needed
data.extend([0, 0])
print(data.hex())
result = feature_char.write_value(value=data)
logging.debug('write resulted in: %s', result)
data = []
while not data or len(data) == 0:
time.sleep(1)
logging.debug('reading characteristic')
data = feature_char.read_value()
resp_data = [b for b in data]
logging.debug('read: %s', bytearray(resp_data).hex())It managed to send a Write and a Read to the dbus API, (i checked with dbus-monitor --system). The Read returns None so the code enters a loop. But now subsequent Reads fail - and the GATT API just continues to return None. Instrumentiont the LoggingDevice class tells me it disconnects after the first Read.
Also I think gatt might have a bug in that characteristic_write_value_succeeded is never called?
jlusiardi
commented
5 years ago I got this result for a read on the pairing features characteristic:
> PYTHONPATH=. python3 staging/hap_read_pairingfeatures.py --log=DEBUG -f ble.json -a DW1
2018-12-05 21:47:34,853 hap_read_pairingfeatures.py:0055 DEBUG running version: 20181204
2018-12-05 21:47:34,853 hap_read_pairingfeatures.py:0056 DEBUG using adapter hci0
2018-12-05 21:47:34,865 hap_read_pairingfeatures.py:0074 DEBUG connecting to device
2018-12-05 21:47:35,099 hap_read_pairingfeatures.py:0076 DEBUG connected to device
2018-12-05 21:47:35,099 tools.py:0033 DEBUG services: [<gatt.gatt_linux.Service object at 0x7fa69ec9f9b0>, <gatt.gatt_linux.Service object at 0x7fa69ecb0320>, <gatt.gatt_linux.Service object at 0x7fa69ecb0400>, <gatt.gatt_linux.Service object at 0x7fa69ec87940>, <gatt.gatt_linux.Service object at 0x7fa69ec78d68>, <gatt.gatt_linux.Service object at 0x7fa69ec87c50>, <gatt.gatt_linux.Service object at 0x7fa69ec8e908>, <gatt.gatt_linux.Service object at 0x7fa69ec8e048>]
2018-12-05 21:47:35,099 tools.py:0038 DEBUG searched service: <gatt.gatt_linux.Service object at 0x7fa69ec87c50>
2018-12-05 21:47:35,099 tools.py:0047 DEBUG char: 00000050-0000-1000-8000-0026bb765291 pairing.pairings
2018-12-05 21:47:35,099 tools.py:0047 DEBUG char: 0000004f-0000-1000-8000-0026bb765291 pairing.features
2018-12-05 21:47:35,160 tools.py:0047 DEBUG char: e604e95d-a759-4817-87d3-aa005083a0d1 protocol.service-id
2018-12-05 21:47:35,160 tools.py:0047 DEBUG char: 0000004c-0000-1000-8000-0026bb765291 pairing.pair-setup
2018-12-05 21:47:35,160 tools.py:0047 DEBUG char: 0000004e-0000-1000-8000-0026bb765291 pairing.pair-verify
2018-12-05 21:47:35,161 tools.py:0061 DEBUG searched char: <gatt.gatt_linux.Characteristic object at 0x7fa69ec8eb00> 12
2018-12-05 21:47:35,161 hap_read_pairingfeatures.py:0091 DEBUG write resulted in: None
2018-12-05 21:47:36,162 hap_read_pairingfeatures.py:0096 DEBUG reading characteristic
2018-12-05 21:47:36,319 hap_read_pairingfeatures.py:0099 DEBUG read: 028c000300010101
2018-12-05 21:47:36,319 hap_read_pairingfeatures.py:0023 DEBUG parse sig read response 028c000300010101
2018-12-05 21:47:36,319 hap_read_pairingfeatures.py:0027 DEBUG control field 2
2018-12-05 21:47:36,320 hap_read_pairingfeatures.py:0029 DEBUG transaction id 140 (expected was 140)
2018-12-05 21:47:36,320 hap_read_pairingfeatures.py:0031 DEBUG status code 0 (The request was successful.)
2018-12-05 21:47:36,320 hap_read_pairingfeatures.py:0038 DEBUG expected body length 3 (got 3)
[
1: (1 bytes) 0x01
]The value of 1 makes sense (Table 4-1 page 58 of spec).
I'll try homebridge-ranger on my Raspberry Pi Zero W as well.
Also I think gatt might have a bug in that characteristic_write_value_succeeded is never called?
Perhaps this is because of the manager is not running?
Jc2k
commented
5 years ago OK - so if I modify hap_read_pairingfeatures.py to try and read my outlets on char then i get an error 5, like homebridge-ranger.
If i copy and paste that same working code to the end of pair_verify (so that i have the crypto keys) then it stops working. The write seems to work. But the first read returns None. If you fix the loop to handle Nones the others fail too, and it seems the connection has been closed.
If i then comment out the call to get_session_keys (and references to the keys) i'm back to the error 5 state.
I guess after doing a pair_verify the connection is considered 'secure' by the other end and it now only expects encrypted messages, and ignores anything it can't verify the auth mac of?
Jc2k
commented
5 years ago I've managed to read a characteristic on my Eve Energy! At the end of hap_pair_verify.py:
write_fun = create_ble_pair_setup_write(pair_verify_char, pair_verify_char_id)
c2a_key, a2c_key = get_session_keys(None, pairing_data, write_fun)
logging.debug('keys: \n\t\tc2a: %s\n\t\ta2c: %s', c2a_key.hex(), a2c_key.hex())
print(c2a_key)
import random
feature_char, feature_char_id = find_characteristic(device,
ServicesTypes['public.hap.service.outlet'],
CharacteristicsTypes.ON)
if not feature_char:
print('features characteristic not found')
sys.exit(-1)
transaction_id = random.randrange(0, 255)
data = bytearray([0x00, HapBleOpCodes.CHAR_READ, transaction_id])
data.extend(feature_char_id.to_bytes(length=2, byteorder='little'))
c2a_counter = 0
cnt_bytes = c2a_counter.to_bytes(8, byteorder='little')
cipher_and_mac = chacha20_aead_encrypt(bytes(), c2a_key, cnt_bytes, bytes([0, 0, 0, 0]), data)
cipher_and_mac[0].extend(cipher_and_mac[1])
data = cipher_and_mac[0]
logging.debug('cipher and mac %s', cipher_and_mac[0].hex())
result = feature_char.write_value(value=data)
logging.debug('write resulted in: %s', result)
data = []
while not data or len(data) == 0:
time.sleep(1)
logging.debug('reading characteristic')
data = feature_char.read_value()
resp_data = bytearray([b for b in data])
logging.debug('read: %s', bytearray(resp_data).hex())
a2c_counter = 0
resp_data = chacha20_aead_decrypt(bytes(), a2c_key, a2c_counter.to_bytes(8, byteorder='little'), bytes([0, 0, 0, 0]), resp_data)
print(resp_data)
parse_read_response(resp_data, transaction_id)The difference between this and your earlier PoC code is just this:
len_bytes = len(data).to_bytes(2, byteorder='little')
cipher_and_mac = chacha20_aead_encrypt(len_bytes, c2a_key, cnt_bytes, bytes([0, 0, 0, 0]), data)vs this
cipher_and_mac = chacha20_aead_encrypt(bytes(), c2a_key, cnt_bytes, bytes([0, 0, 0, 0]), data)When I got rid of len_bytes it all started working.
Jc2k
commented
5 years ago https://github.com/jlusiardi/homekit_python/pull/57 contains staging/test.py. When I run that against my Eve Energy it turns the switch on and off on a loop, checking the result with get_characteristic after every go. It does eventually fail, but the session security seems to be working.
jlusiardi
commented
5 years ago Hey @Jc2k,
that's awesome, works with the DW1 as well, but i'm not sure why. I'll dig into it!
Jc2k
commented
5 years ago I think I mostly get it now. What bit are you not sure about? The aad being bytes() was a surprise but looking at the HAP spec the AAD is documented as set for HTTP messages, but the AAD isn't mentioned at all for BLE secure messages. I can't see that they specify the params for the ChaCha20 call for secure BLE sessions at all, at least in the copy of the spec I have. But I guess if it works it must be right?
There are some bits of my branch i'm not happy with still:
Pretty sure the read code needs to decrypt each read seperately, rather than reading a bunch of times and decrypting all at the end. It is is fine for short payloads, but thats by luck I think. We need to find a large characteristic to test this.
It can keep trying to read when the remote device is disconnected. I guess any looping code should check the device connected status so it doesn't loop forever?
And obviously my branch is messy and needs factoring into a proper API.
I'll leave you to dig at it - let me know how it goes!
jlusiardi
commented
5 years ago hey,
i started to integrate it! works out well so far. But did you also have issues that the bluetooth le accessories just react on the second attempt?
Jc2k
commented
5 years ago Yes I did - we need to find a way for the background service discovery to complete. It forgets all the characteristics every time a session disconnects it seems.
jlusiardi
commented
5 years ago perhaps an own bluetooth lib based on https://docs.python.org/3.3/library/socket.html would help for that.
Jc2k
commented
5 years ago It was actually working in an earlier version of the code - let me dig out the magic
Jc2k
commented
5 years ago So when i was running this version of the code i could work around the 2-attempt issue by uncommenting this line: https://github.com/jlusiardi/homekit_python/commit/ec0b098393e6973c0d1aeee20bdd41f2f5c03e5c#diff-c69531862ba2a133a077f908538ed170R65
Letting the AnyDevice object run until the services were resolved.
Jc2k
commented
5 years ago Hoping to spend some time on this tomorrow. How is it looking on your end?
jlusiardi
commented
5 years ago i did some commits, but the travis ci checks keep on failing. I am looking into that at the moment
jlusiardi
commented
5 years ago Your proposal with the manager will result in a never returning program if the accessory is not available. That is not a good option, i guess. perhaps there is a better option.
Jc2k
commented
5 years ago Looking at the GATT code we could do something like:
def setup_session(self):
mac_address = self.pairing_data['AccessoryMAC']
# TODO specify adapter by config?
manager = staging.gatt.DeviceManager(adapter_name='hci0')
self.device = staging.gatt.Device(manager=manager, mac_address=mac_address)
logging.debug('connecting to device')
self.device.connect()
logging.debug('connected to device')
if not self.device.services:
logging.debug('waiting for services to be resolved')
for i in range(10):
if self.device.is_services_resolved():
break
time.sleep(1)
else:
raise BleSessionError('Unable to resolve device services + characteristics')
# This is called automatically when the mainloop is running, but we
# want to avoid running it and blocking for an indeterminate amount of time.
logging.debug('enumerating resolved services')
self.device.services_resolved()EDIT to add the code i've tested.
Jc2k
commented
5 years ago Didn't get to look at this as much as I wanted today but:
https://github.com/jlusiardi/homekit_python/pull/57
Makes put_char / get_char / list_accessories_and_chars more reliable - at least in terms of service enumeration.
Jc2k
commented
5 years ago (FYI I've raised https://github.com/home-assistant/home-assistant/pull/19549 to switch to using Controller in home assistant, ready for adding BLE support there).
Hey,
what is needed to implement the support for Bluetooth LE Accessories?
I don't have much free time, but I'm motivated to contribute here.
Best regards