search

fg1 / BLEHeartRateLogger

Bluetooth Low-Energy Heart Rate Monitor data logging in Python
Other
129 stars 48 forks source link

Expected gatttool formatting now out of date, breaking BLEHeartRateLogger? #1

Closed gwern closed 8 years ago

gwern commented 8 years ago

So back in November 2014 or so, I got a Zephyr HxM Smart Heart Rate Monitor so I could monitor heart-rate variability and effects of various substances. BLEHRL was the simplest application I could find which would reliably log the data to an SQL DB, so I set up BLEHRL and used it for a while without a problem, and then events caused me to set the belt aside for a while and then my old Debian Dell laptop died. I got a new Acer laptop, and to get GPU/deep learning support, I installed Ubuntu 14.04.3 LTS; unfortunately this did not support the WiFi+BT card and so I bought a SoundBot® SB340 Bluetooth 4.0 USB Adapter to enable BT until such time as I am able to safely upgrade Ubuntu to a kernel that does support BT. This also works fine - for example, I can use a pair of Bluetooth headphones. So the full Bluetooth stack on my install & laptop is working.

When I went back to use my heart-rate belt with BLEHRL HEAD, it failed. I would always get:

$ python BLEHeartRateLogger.py 
2016-02-18 16:56:13,102  Trying to find a BLE device
2016-02-18 16:56:14,907  Establishing connection to 78:A5:04:81:B2:5D
2016-02-18 16:56:45,012  Establishing connection to 78:A5:04:81:B2:5D
...

After double-checking the Bluetooth was working and following the debugging instructions, what I see is that the heart-rate belt is connected and is working and even sending what I think is heart-rate data:

$ hcitool lescan
LE Scan ...
78:A5:04:81:B2:5D Zephyr HXM200008002
78:A5:04:81:B2:5D (unknown)
$ gatttool -b 78:A5:04:81:B2:5D --interactive
[   ][78:A5:04:81:B2:5D][LE]> connect
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f f0 02 fc 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f 00 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f f0 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 d8 02 d8 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 dc 02 
[CON][78:A5:04:81:B2:5D][LE]> characteristics
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 cc 02 d4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 51 dc 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 f8 02 
[CON][78:A5:04:81:B2:5D][LE]> 
handle: 0x0002, char properties: 0x02, char value handle: 0x0003, uuid: 00002a00-0000-1000-8000-00805f9b34fb
handle: 0x0004, char properties: 0x02, char value handle: 0x0005, uuid: 00002a01-0000-1000-8000-00805f9b34fb
handle: 0x0006, char properties: 0x0a, char value handle: 0x0007, uuid: 00002a02-0000-1000-8000-00805f9b34fb
handle: 0x0008, char properties: 0x0a, char value handle: 0x0009, uuid: 00002a03-0000-1000-8000-00805f9b34fb
handle: 0x000a, char properties: 0x02, char value handle: 0x000b, uuid: 00002a04-0000-1000-8000-00805f9b34fb
handle: 0x000d, char properties: 0x20, char value handle: 0x000e, uuid: 00002a05-0000-1000-8000-00805f9b34fb
handle: 0x0011, char properties: 0x10, char value handle: 0x0012, uuid: 00002a37-0000-1000-8000-00805f9b34fb
handle: 0x0014, char properties: 0x02, char value handle: 0x0015, uuid: 00002a38-0000-1000-8000-00805f9b34fb
handle: 0x0016, char properties: 0x08, char value handle: 0x0017, uuid: 00002a39-0000-1000-8000-00805f9b34fb
handle: 0x0019, char properties: 0x02, char value handle: 0x001a, uuid: 00002a23-0000-1000-8000-00805f9b34fb
handle: 0x001b, char properties: 0x02, char value handle: 0x001c, uuid: 00002a24-0000-1000-8000-00805f9b34fb
handle: 0x001d, char properties: 0x02, char value handle: 0x001e, uuid: 00002a25-0000-1000-8000-00805f9b34fb
handle: 0x001f, char properties: 0x02, char value handle: 0x0020, uuid: 00002a26-0000-1000-8000-00805f9b34fb
handle: 0x0021, char properties: 0x02, char value handle: 0x0022, uuid: 00002a27-0000-1000-8000-00805f9b34fb
handle: 0x0023, char properties: 0x02, char value handle: 0x0024, uuid: 00002a28-0000-1000-8000-00805f9b34fb
handle: 0x0025, char properties: 0x02, char value handle: 0x0026, uuid: 00002a29-0000-1000-8000-00805f9b34fb
handle: 0x0027, char properties: 0x02, char value handle: 0x0028, uuid: 00002a2a-0000-1000-8000-00805f9b34fb
handle: 0x0029, char properties: 0x02, char value handle: 0x002a, uuid: 00002a50-0000-1000-8000-00805f9b34fb
handle: 0x002c, char properties: 0x12, char value handle: 0x002d, uuid: 00002a19-0000-1000-8000-00805f9b34fb
handle: 0x0030, char properties: 0x10, char value handle: 0x0031, uuid: befdff11-c979-11e1-9b21-0800200c9a66
handle: 0x0033, char properties: 0x10, char value handle: 0x0034, uuid: befdff12-c979-11e1-9b21-0800200c9a66
handle: 0x0037, char properties: 0x10, char value handle: 0x0038, uuid: befdffb1-c979-11e1-9b21-0800200c9a66
handle: 0x003a, char properties: 0x08, char value handle: 0x003b, uuid: befdffb2-c979-11e1-9b21-0800200c9a66
[CON][78:A5:04:81:B2:5D][LE]> 
[CON][78:A5:04:81:B2:5D][LE]> exit
    ][78:A5:04:81:B2:5D][LE]>

But BLEHRL itself is refusing to work. I noticed that this gatttool output doesn't look exactly the same as I remembered it; I don't think it had the [CON] or [LE] parts, for example. I looked a bit through BLEHeartRateLogger.py and while I'm not a Python programmer, it looks like in the 'expect'-related calls, the program is hardwiring the formatting it expects/parses the gatttool data into. Could BLEHRL now be broken by upstream changes to how gatttool prints stuff?

fg1 commented 8 years ago

As you pointed in your issue, the format of your version of gatttool isn't handled properly by BLEHRL. It seems like the version of bluez (which contains gatttool) shipped with Ubuntu 14.04 is actually an old one (bluez 4.101) compared to the one I'm using (bluez 5.36).

I pushed a change which should address your issue and handle both formats. Let me know if it is working for you now.

gwern commented 8 years ago

Thanks but looks like it's not quite there:

$ python BLEHeartRateLogger.py 
2016-02-19 12:22:37,413  Trying to find a BLE device
2016-02-19 12:22:38,840  Establishing connection to 78:A5:04:81:B2:5D
2016-02-19 12:22:38,966  Connected to 78:A5:04:81:B2:5D
2016-02-19 12:23:09,067  Couldn't find the heart rate measurement handle?!
$
fg1 commented 8 years ago

I pushed a new change. @gwern: Can you tell me if is it working on your side now?

gwern commented 8 years ago

Looks like no:

$ python BLEHeartRateLogger.py -d
2016-02-28 12:35:16,350  Trying to find a BLE device
2016-02-28 12:35:18,123  Establishing connection to 78:A5:04:81:B2:5D
[   ][78:A5:04:81:B2:5D][LE]> connect
connect
[CON][78:A5:04:81:B2:5D][LE]> 2016-02-28 12:35:18,745  Connected to 78:A5:04:81:B2:5D
char-desc
char-desc
[   ][78:A5:04:81:B2:5D][LE]> 2016-02-28 12:35:23,851  Couldn't find the heart rate measurement handle?!

$ gatttool -b 78:A5:04:81:B2:5D --interactive
[   ][78:A5:04:81:B2:5D][LE]> connect
[CON][78:A5:04:81:B2:5D][LE]> char-desc
[CON][78:A5:04:81:B2:5D][LE]> 
handle: 0x0001, uuid: 2800
handle: 0x0002, uuid: 2803
handle: 0x0003, uuid: 2a00
handle: 0x0004, uuid: 2803
handle: 0x0005, uuid: 2a01
[CON][78:A5:04:81:B2:5D][LE]> exit

I notice the 'char-desc' and also 'connect' and 'characteristics' seem to take a few seconds to return results, while BLEHeartRateLogger.py exits quickly. Could BLEHRL being impatient and not waiting long enough for results?

fg1 commented 8 years ago

BLEHRL might indeed be impatient. Could you try modifying the value of timeout on line 248 in BLEHeartRateLogger.py?

If this is successful, I'll add a new parameter to the script in order to be able to set other values.

gwern commented 8 years ago

I tried changing it to 20/30/60, didn't help. Output still looks like this:

$ python BLEHeartRateLogger.py -d
2016-02-29 12:43:54,585  Trying to find a BLE device
2016-02-29 12:43:56,847  Establishing connection to 78:A5:04:81:B2:5D
[   ][78:A5:04:81:B2:5D][LE]> connect
connect
[CON][78:A5:04:81:B2:5D][LE]> 2016-02-29 12:43:57,474  Connected to 78:A5:04:81:B2:5D
char-desc
char-desc
[CON][78:A5:04:81:B2:5D][LE]> 
handle: 0x0001, uuid: 2800
handle: 0x0002, uuid: 2803
handle: 0x0003, uuid: 2a00
handle: 0x0004, uuid: 2803
handle: 0x0005, uuid: 2a01
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 8c 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 55 7c 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 55 88 02 a4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 56 bc 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 55 e0 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 54 00 03 18 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 ec 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 f8 02 e4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 d0 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 c0 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 bc 02 b8 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 54 b0 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 54 e0 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 53 20 03 8c 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 51 4c 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 58 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f 40 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f f4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f ec 02 f4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f e4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 c8 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 e0 02 00 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f 0c 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f f8 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f 14 03 2c 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4e 04 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4e 04 03 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 4f ec 02 c4 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 c8 02 
[CON][78:A5:04:81:B2:5D][LE]> 
Notification handle = 0x0012 value: 16 50 cc 02 
[CON][78:A5:04:81:B2:5D][LE]> 2016-02-29 12:44:27,575  Couldn't find the heart rate measurement handle?!
fg1 commented 8 years ago

I pushed a new change which should enable you to manually set the HR handle. Please try again with the same command as you did earlier as it now sends characteristics in place of char-desc.

Otherwise, you can also force the script to skip this discovery phase: $ python BLEHeartRateLogger.py -H 0x0012

gwern commented 8 years ago

Yes, that seems to be working!

$ BLEHeartRateLogger.py -v -o /home/gwern/selfexperiment/hrv.sql 
2016-02-29 14:50:48,070  Trying to find a BLE device
2016-02-29 14:50:49,964  Establishing connection to 78:A5:04:81:B2:5D
2016-02-29 14:50:50,579  Connected to 78:A5:04:81:B2:5D
2016-02-29 14:51:05,682  {'rr': [776], 'rr_interval': True, 'hr': 81, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,684  {'rr': [740, 760], 'rr_interval': True, 'hr': 81, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,684  {'rr': [740], 'rr_interval': True, 'hr': 81, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,685  {'rr': [704, 688], 'rr_interval': True, 'hr': 82, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,685  {'rr': [700], 'rr_interval': True, 'hr': 82, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,685  {'rr': [728], 'rr_interval': True, 'hr': 82, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}
2016-02-29 14:51:05,697  sqlite commit time: 0.0125398635864
2016-02-29 14:51:05,698  {'rr': [688, 716], 'rr_interval': True, 'hr': 83, 'sensor_contact': 'Contact detected', 'hrv_uint8': True, 'ee_status': False}

...

fg1 commented 8 years ago

Thank you for your help figuring out this bug.