Use pygatt instead of bluepy

[ACrazyConcept]

Making a new post even though it is related to my "latency" issue https://github.com/PatrickE94/pycalima/issues/11

I reinstalled my Pi and then I simply could not get bluepy to work reliably again. I then found pygatt https://github.com/peplin/pygatt and decided to try it out. And the response time for retrieving the fan state is so much better, and so far rock solid:

Script used for the graph:

import pygatt
from struct import unpack, pack
import time
start_time = time.time()

adapter = pygatt.GATTToolBackend()

try:
    adapter.start()
    device = adapter.connect('58:2B:DB:01:D5:33', timeout=15)
    value = device.char_read("528b80e8-c47a-4c0a-bdf1-916a7748f412")
finally:
    adapter.stop()

v = unpack('<4HBHB', value)
print(v)
print(" %s seconds" % round((time.time() - start_time),3))

Edit: Took me a while to get the data types wrangled for the write function. If you know a shorter route from the human language inputs to the bytearray gatttool uses feel free to enlighten me.

import pygatt
from struct import pack
import time
import binascii

start_time = time.time()

adapter = pygatt.GATTToolBackend()

try:
    adapter.start()
    device = adapter.connect('58:2B:DB:01:D5:33', timeout=30)
    # write pin
    hex_string = "73C04000"
    device.char_write_handle(0x18, bytearray.fromhex(hex_string), wait_for_response=True)
    # and read pin confirmation uuid before control is possible.
    device.char_read('d1ae6b70-ee12-4f6d-b166-d2063dcaffe1')
    # set boost mode
    hex_string = binascii.b2a_hex(pack('<BHH', 1, 2000, 30)).decode('utf-8')
    device.char_write_handle(0x32, bytearray.fromhex(hex_string), wait_for_response=True)

finally:
    adapter.stop()

print(" %s seconds" % round((time.time() - start_time),3))

[PatrickE94]

I get the latency issues here, it does matter. However, I'm a bit on the fence regarding pygatt. It's based on the deprecated gattool and is already a hassle under Arch (my OS of choice). I'll give it a whirl and see what i find.

Some initial thoughts though. I'd guess by your script that you never really disconnect from the device. If so, the led on the fan would continuously glow solid. That might explain your low latency as the service discovery part is especially slow (done when connecting). Since pygatt wraps a CLI tool, the Bluez stack is likely still connected after script has terminated if not told to disconnect. I'm guessing here. EDIT: My bad, adapter.stops() does disconnect.

I tried staying connected once and got issues with the fan, it stopped responding to sensors (light, humidity etc) and was likely stuck in "app preview"-mode. I haven't exhausted this fully. Do check if your experience is otherwise!

[PatrickE94]

Alright, quick research results. I converted it to fully use handles (not UUID's). I also wrote the corresponding script for bluepy. And I don't really see the gains you find. Rather the opposite, bluepy performs much better. I even failed to benchmark pygatt since my bluetooth service got angry with start limits and enter a failed state.

import bluepy.btle as ble
from struct import pack
import time
import binascii

start_time = time.time()

try:
    device = ble.Peripheral(deviceAddr='58:2B:DB:02:83:6A')
    # write pin
    pin = "--"
    device.writeCharacteristic(0x18, pack('<I', int(pin)), withResponse=True)
    # and read pin confirmation uuid before control is possible.
    device.readCharacteristic(0x1a)
    # set boost mode
    device.writeCharacteristic(0x32, pack('<BHH', 1, 2000, 30), withResponse=True)

finally:
    device.disconnect()

print(" %s seconds" % round((time.time() - start_time),3))

 $ hyperfine --show-output "python test_bluepy.py"
Benchmark #1: python test_bluepy.py
 3.352 seconds
 0.326 seconds
 0.289 seconds
 0.281 seconds
 0.279 seconds
 0.284 seconds
 0.278 seconds
 0.286 seconds
 0.285 seconds
 0.278 seconds
  Time (mean ± σ):     659.0 ms ± 971.2 ms    [User: 64.6 ms, System: 12.3 ms]
  Range (min … max):   342.2 ms … 3422.7 ms    10 runs

import pygatt
from struct import pack
import time
import binascii

start_time = time.time()

adapter = pygatt.GATTToolBackend()

try:
    adapter.start()
    device = adapter.connect('58:2B:DB:02:83:6A', timeout=30)
    # write pin
    pin = "69288170"
    device.char_write_handle(0x18, pack('<I', int(pin)), wait_for_response=True)
    # and read pin confirmation uuid before control is possible.
    device.char_read_handle(0x1a) #'d1ae6b70-ee12-4f6d-b166-d2063dcaffe1')
    # set boost mode
    device.char_write_handle(0x32, pack('<BHH', 1, 2000, 30), wait_for_response=True)

finally:
    adapter.stop()

print(" %s seconds" % round((time.time() - start_time),3))

Benchmark #1: python test2.py
 1.357 seconds
 1.697 seconds
 1.474 seconds
 1.473 seconds
 1.353 seconds
Job for bluetooth.service failed.

---

Aug 08 20:50:01 atheon systemd[1]: bluetooth.service: Start request repeated too quickly.
Aug 08 20:50:01 atheon systemd[1]: bluetooth.service: Failed with result 'start-limit-hit'.

[PatrickE94]

I should note that I'm not really a fan of the bluez stack, not really bluetooth as a whole. With this in mind, I'm more for writing a lighter firmware for ESP32 or some NRF circuit which can "retain" a characteristics scan and bypass the stiff init procedures of bluez.

[ACrazyConcept]

Wow that's impressive! Thank you for that investigation. Which python version and hardware is that on ? I am by no means a fan of either, I just wanted something that worked better and I will definitely try your example out.

[PatrickE94]

Python 3.8.5 and using my laptop (on Arch). I speculate wildly about bluetooth generally, I'm by no means an expert on the protocol. But I get the feeling that the bluez stack is trying to adhere both to stable and "safe" use of the protocol. I'm talking about things like always discovering services before requesting read/writes etc. If we could get a lower level BLE stack where we could bypass initial checks (we can map UUID's to handles one time and persist the settings between runs) I think we could speed this up.

If you want to go on a google quest, find out if you can (by any means) write a characteristic or connect without performing an initial scan using bluez. I'm not sure yet if it's fully possible on the protocol level. But if this is possible, we can speed it up!

[ACrazyConcept]

Hah well I'm way less of an expert on this that's for damn sure. But do you know aioblescan? I have no idea wether it is comparable at all. But it works really good in this repo for picking up sensor beacons. It just seems like all bluetooth devices should work like that as long as you only need to recieve data and not control anything. I donno...

But I tried your script now and it is just super fast, so thank you so much for that!