The bluepy package has been super helpful in developing a Raspberry application!

I am working with a custom bluetooth device that requires the following to get indications from a characteristic: ---CCCD set to 0x02 0x00 ---I get an error when I read the characteristic directly (don't understand why) ---Therefore, I NEED the delegate function to read any indications from the device

However, I cannot consistently rely on waitForNotifications to return True when a new indication is received. There are these two scenarios:

1.) The device at certain times sends an indication ---waitForNotifications works (i.e. returns True) ---the indication is received by the delegate function after, AND ONLY IF, waitForNotifications is called

2.) I send a message to the device by writing the characteristic ---The device immediately sends an indication in response (by design) ---the response is received by the delegate function ---waitForNotifications DOES NOT return True

This behavior is very consistent and reproducible. It would be desirable if either ---waitForNotifications ALWAYS returns true if a new indication is received ---and/or the callback function is ALWAYS called when a new indication is received (whether or not waitForNotifications is called)

Any insights into what might be going on; whether this behavior is by design or a bug; and and tips for dealing with it would be appreciated!

Thanks, Michael

Michael, some of what you're saying sounds familiar and I think that what you're seeing is by design. The waitForNotifications function basically unblocks the IO, so you have to keep calling it when you're expecting notifications.

Here's a pattern that mostly works for me (I'm seeing a bug with delayed notifications which is fixed by PR #355, but other than that it works)-

I start a thread that runs a loop like this:

while True:
    while not tx_queue.empty():
        command = tx_queue.get_nowait()
        peripheral.writeCharacteristic(...<command>...)

    peripheral.waitForNotifications(...)

Whenever I want to write a characteristic value, I drop it into the tx_queue. Any time that is not spent "sending messages" is spent waiting for notifications. (There's a lot of room for improvement in the code above, but it gives you the basic idea)

It would be my preference if bluepy internally ran a separate rx thread for receiving notifications so that we didn't have to call waitForNotifications at all, but there's probably a decent reason why it isn't that way. (but someone say something if I should take a shot at it and submit a PR...)

As far as the other issue of getting an error when you try to read the characteristic value, it may be that the characteristic properties (check Characteristic.properties) are set to Notify only. If the Read property bit is not set, then you will get an error trying to read it.

Carey

Hi Carey, Thanks for your response! It gives me some clues.

I am having a hard time understanding how the queue works. Maybe there is some additional code that you could share?

Again, I observe this:

bgtc.tx_characteristic.write(self.tx_frame, True) print (bgtc.peripheral.waitForNotifications(1))

False

Yet, the notify callback has received a new indication about 300ms after the write, and thus while waitForNotification is presumably monitoring?! So I can’t rely on waitForNotifications.

Regarding the characteristic properties being set to Notify only, is that done by the server (the remote BLE device), and can my client do anything about it?

Thanks again, Michael

On May 10, 2019, at 1:11 PM, csonsino notifications@github.com<mailto:notifications@github.com> wrote:

Michael, some of what you're saying sounds familiar and I think that what you're seeing is by design. The waitForNotifications function basically unblocks the IO, so you have to keep calling it when you're expecting notifications.

Here's a pattern that mostly works for me (I'm seeing a bug with delayed notifications which is fixed by PR #355https://github.com/IanHarvey/bluepy/pull/355, but other than that it works)-

I start a thread that runs a loop like this:

while True: while not tx_queue.empty(): command = tx_queue.get_nowait() peripheral.writeCharacteristic(......)

peripheral.waitForNotifications(...)

Whenever I want to write a characteristic value, I drop it into the tx_queue. Any time that is not spent "sending messages" is spent waiting for notifications. (There's a lot of room for improvement in the code above, but it gives you the basic idea)

It would be my preference if bluepy internally ran a separate rx thread for receiving notifications so that we didn't have to call waitForNotifications at all, but there's probably a decent reason why it isn't that way. (but someone say something if I should take a shot at it and submit a PR...)

As far as the other issue of getting an error when you try to read the characteristic value, it may be that the characteristic properties (check Characteristic.properties) are set to Notify only. If the Read property bit is not set, then you will get an error trying to read it.

Carey

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/IanHarvey/bluepy/issues/360#issuecomment-491415802, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACAKGIMYYB2S36JSTGWGG23PUXJHBANCNFSM4HLNX4NA.

Here's some sample code. It uses the Nordic UART BLE GATT service, and works with the Adafruit Bluefruit LE Connect iOS app. It is very much example code and not quite how I would do things, and it lacks a lot of cleanup and error handling, but it gives you a more concrete example.

Run the phone app in "Peripheral Mode", and tap the UART button to open the UART page. Set the phone's MAC address in the __main__ function at the bottom, and make sure that you have the correct addr_type line uncommented (my iPhone uses random addresses, which is kind of a pain).

from queue import Queue
from threading import Thread

import bluepy
from bluepy.btle import Peripheral, DefaultDelegate, BTLEException

# NOTE - This class is directly inheriting bluepy.btle.DefaultDelegate for notifications,
#        but the delegate could be a separate class
class BluepyExample(DefaultDelegate):

    def __init__(self, address, type=bluepy.btle.ADDR_TYPE_PUBLIC):
        super().__init__()

        self._peripheral_address = address
        self._peripheral_address_type = type
        self._peripheral = None

        # create the TX queue
        self._tx_queue = Queue()

        # start the bluepy IO thread
        self._bluepy_thread = Thread(target=self._bluepy_handler)
        self._bluepy_thread.name = "bluepy_handler"
        self._bluepy_thread.daemon = True
        self._bluepy_thread.start()

    def handleNotification(self, cHandle, data):
        """This is the notification delegate function from DefaultDelegate
        """
        print("\nReceived Notification: " + str(data))

    def _bluepy_handler(self):
        """This is the bluepy IO thread
        :return:
        """
        try:
            # Connect to the peripheral
            self._peripheral = Peripheral(self._peripheral_address, self._peripheral_address_type)
            # Set the notification delegate
            self._peripheral.setDelegate(self)

            # get the list of services
            services = self._peripheral.getServices()

            write_handle = None
            subscribe_handle = None

            # magic stuff for the Nordic UART GATT service
            uart_uuid = "6e400001-b5a3-f393-e0a9-e50e24dcca9e"
            uart_write_uuid_prefix = "6e400002"

            # this is general magic GATT stuff
            # notify handles will have a UUID that begins with this
            uart_notify_uuid_prefix = "00002902"
            # these are the byte values that we need to write to subscribe/unsubscribe for notifications
            subscribe_bytes = b'\x01\x00'
            # unsubscribe_bytes = b'\x00\x00'

            # dump out some info for the services that we found
            for service in services:
                print("Found service: " + str(service))
                if str(service.uuid).lower() == uart_uuid:
                    # this is the Nordic UART service that we're looking for
                    chars = service.getCharacteristics()
                    for char in chars:
                        print("  char: " + str(char) + ", handle: " + str(char.handle) +
                              ", props: " + str(char.properties))
                    descs = service.getDescriptors()
                    # this is the important part-
                    # find the handles that we will write to and subscribe for notifications
                    for desc in descs:
                        print("  desc: " + str(desc))
                        str_uuid = str(desc.uuid).lower()
                        if str_uuid.startswith(uart_write_uuid_prefix):
                            write_handle = desc.handle
                            print("*** Found write handle: " + str(write_handle))
                        elif str_uuid.startswith(uart_notify_uuid_prefix):
                            subscribe_handle = desc.handle
                            print("*** Found subscribe handle: " + str(subscribe_handle))

            if write_handle is not None and subscribe_handle is not None:
                # we found the handles that we need

                # this call performs the subscribe for notifications
                response = self._peripheral.writeCharacteristic(subscribe_handle, subscribe_bytes, withResponse=True)

                # now that we're subscribed for notifications, waiting for TX/RX...
                while True:
                    while not self._tx_queue.empty():
                        msg = self._tx_queue.get_nowait()
                        msg_bytes = bytes(msg, encoding="utf-8")
                        self._peripheral.writeCharacteristic(108, msg_bytes)

                    self._peripheral.waitForNotifications(1.0)

        except BTLEException as e:
            print(e)

    def send(self, message):
        """Call this function to send a BLE message over the UART service
        :param message: Message to send
        :return:
        """

        # put the message in the TX queue
        self._tx_queue.put_nowait(message)

if __name__ == "__main__":

    mac = None
    # mac = "AA:BB:CC:DD:EE:FF"

    # NOTE - MUST set this appropriately, depending on the type of address that the peripheral is advertising
    # addr_type = bluepy.btle.ADDR_TYPE_PUBLIC
    addr_type = bluepy.btle.ADDR_TYPE_RANDOM

    if mac is None:
        print("Need to set the MAC address...")
        exit(1)

    example = BluepyExample(mac, addr_type)
    while True:
        msg = input()
        if msg.upper() == "Q":
            break
        # else:
        example.send(msg + "\n")

Oh, and the characteristic properties are controlled 100% by the side that's advertising the service (peripheral). Nothing you can do about it from your client (central) side.

Hello Carey,

Thanks for the sample code, and for the clarification regarding the characteristic permissions controlled by the server (peripheral). I confirmed that it is, indeed, set to notify and write, so it doesn’t allow a read on the client side.

The sample code is very instructive, thanks for providing it. I did one thing a little different: I used the following code to get the handle for the client configuration (2902) once, and I am using the handle to enable indications and notifications. Not as elegant but it works.

for development: get the details of the device

def details(self):
    services = self.peripheral.getServices()
    for service in services:
        print("Service:", service.uuid)
        characteristics = service.getCharacteristics()
        for characteristic in characteristics:
            print("Characteristic:", characteristic.uuid, characteristic.propertiesToString())
    descriptors = self.peripheral.getDescriptors()
    for descriptor in descriptors:
        print (descriptor.handle, descriptor, descriptor.uuid)

Now regarding my issue with waitForNotification: Apparently, the characteristic for exchanging data is somehow configured to require a response from the server when I write to it, and I have to use the following command for writing (my peripheral is called bgtc):

bgtc.tx_characteristic.write(self.tx_frame, withResponse=True)

When the withResponse flag is set to False, there is no response from the client, ever.
When the withResponse flag is set to True, there is a response (in my case after about 300ms), and the handleNotification delegate will receive it, but waitForNotification does not return True in this case.

I don’t understand why, but this behavior is what happens. Are you aware of something in the BLE spec that explains this?

Last not least, my device is disconnected from time to time (I don’t understand why but that’s a different issue). There is a function to check if the connection is up:

peripheral.getState()

If the device is connected, this function returns ‘conn’. If the device is NOT connected, calling this function results in an exception. Of course it’s possible to handle the exception in my code, but this behavior was unexpected. I can’t imaging that’s by design?

Best, Michael

On May 11, 2019, at 2:36 PM, csonsino notifications@github.com<mailto:notifications@github.com> wrote:

Here's some sample code. It uses the Nordic UART BLE GATT service, and works with the Adafruit Bluefruit LE Connect iOS app. It is very much example code and not quite how I would do things, and it lacks a lot of cleanup and error handling, but it gives you a more concrete example.

Run the phone app in "Peripheral Mode", and tap the UART button to open the UART page. Set the phone's MAC address in the main function at the bottom, and make sure that you have the correct addr_type line uncommented (my iPhone uses random addresses, which is kind of a pain).

from queue import Queue from threading import Thread

import bluepy from bluepy.btle import Peripheral, DefaultDelegate, BTLEException

NOTE - This class is directly inheriting bluepy.btle.DefaultDelegate for notifications,

but the delegate could be a separate class

class BluepyExample(DefaultDelegate):

def __init__(self, address, type=bluepy.btle.ADDR_TYPE_PUBLIC):
    super().__init__()

    self._peripheral_address = address
    self._peripheral_address_type = type
    self._peripheral = None

    # create the TX queue
    self._tx_queue = Queue()

    # start the bluepy IO thread
    self._bluepy_thread = Thread(target=self._bluepy_handler)
    self._bluepy_thread.name = "bluepy_handler"
    self._bluepy_thread.daemon = True
    self._bluepy_thread.start()

def handleNotification(self, cHandle, data):
    """This is the notification delegate function from DefaultDelegate
    """
    print("\nReceived Notification: " + str(data))

def _bluepy_handler(self):
    """This is the bluepy IO thread
    :return:
    """
    try:
        # Connect to the peripheral
        self._peripheral = Peripheral(self._peripheral_address, self._peripheral_address_type)
        # Set the notification delegate
        self._peripheral.setDelegate(self)

        # get the list of services
        services = self._peripheral.getServices()

        write_handle = None
        subscribe_handle = None

        # magic stuff for the Nordic UART GATT service
        uart_uuid = "6e400001-b5a3-f393-e0a9-e50e24dcca9e"
        uart_write_uuid_prefix = "6e400002"

        # this is general magic GATT stuff
        # notify handles will have a UUID that begins with this
        uart_notify_uuid_prefix = "00002902"
        # these are the byte values that we need to write to subscribe/unsubscribe for notifications
        subscribe_bytes = b'\x01\x00'
        # unsubscribe_bytes = b'\x00\x00'

        # dump out some info for the services that we found
        for service in services:
            print("Found service: " + str(service))
            if str(service.uuid).lower() == uart_uuid:
                # this is the Nordic UART service that we're looking for
                chars = service.getCharacteristics()
                for char in chars:
                    print("  char: " + str(char) + ", handle: " + str(char.handle) +
                          ", props: " + str(char.properties))
                descs = service.getDescriptors()
                # this is the important part-
                # find the handles that we will write to and subscribe for notifications
                for desc in descs:
                    print("  desc: " + str(desc))
                    str_uuid = str(desc.uuid).lower()
                    if str_uuid.startswith(uart_write_uuid_prefix):
                        write_handle = desc.handle
                        print("*** Found write handle: " + str(write_handle))
                    elif str_uuid.startswith(uart_notify_uuid_prefix):
                        subscribe_handle = desc.handle
                        print("*** Found subscribe handle: " + str(subscribe_handle))

        if write_handle is not None and subscribe_handle is not None:
            # we found the handles that we need

            # this call performs the subscribe for notifications
            response = self._peripheral.writeCharacteristic(subscribe_handle, subscribe_bytes, withResponse=True)

            # now that we're subscribed for notifications, waiting for TX/RX...
            while True:
                while not self._tx_queue.empty():
                    msg = self._tx_queue.get_nowait()
                    msg_bytes = bytes(msg, encoding="utf-8")
                    self._peripheral.writeCharacteristic(108, msg_bytes)

                self._peripheral.waitForNotifications(1.0)

    except BTLEException as e:
        print(e)

def send(self, message):
    """Call this function to send a BLE message over the UART service
    :param message: Message to send
    :return:
    """

    # put the message in the TX queue
    self._tx_queue.put_nowait(message)

if name == "main":

mac = None
# mac = "AA:BB:CC:DD:EE:FF"

# NOTE - MUST set this appropriately, depending on the type of address that the peripheral is advertising
# addr_type = bluepy.btle.ADDR_TYPE_PUBLIC
addr_type = bluepy.btle.ADDR_TYPE_RANDOM

if mac is None:
    print("Need to set the MAC address...")
    exit(1)

example = BluepyExample(mac, addr_type)
while True:
    msg = input()
    if msg.upper() == "Q":
        break
    # else:
    example.send(msg + "\n")

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/IanHarvey/bluepy/issues/360#issuecomment-491545641, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACAKGIPYJCJJN6VXKSNVNF3PU434TANCNFSM4HLNX4NA.

If I recall correctly, the withResponse=True argument means that the characteristic.write call will immediately return the value...

response = bgtc.tx_characteristic.write(self.tx_frame, withResponse=True)

I don't think that the response for that comes through the notification handler.

The disconnect may be caused by the peripheral. One way to get an idea of what's going on is to run btmon (probably needs sudo) while you're running the bluepy application. That will dump out the BLE traffic.

I think that the peripheral.getState() function may be relatively new and I haven't really used it. I think I've run into cases where the peripheral itself gets into a bad/invalid state after a disconnect, and attempting to call any function on it results in an exception, but I figured that it was my code :) I typically track the connected state myself, and catching that exception means that I need to reconnect. I'm not sure if BLE connection state is difficult by nature. I don't believe that there are heartbeat/keepalive messages built into the base protocol, so one side could disconnect / power off / leave the area at any time, and the other side thinks it's still connected.

The withResponse=True argument appears to be necessary for the server(device)-specified configuration “WRITE” for a characteristic. For this, a response is expected from the server. The response does come through the notification handler (the callback), however waitForNotification is not applicable in this mode.

Currently only READ is used in supportsRead function,

# the rest is included to facilitate supportsXXXX functions if required
props = {"BROADCAST":    0b00000001,
         "READ":         0b00000010,
         "WRITE_NO_RESP":0b00000100,
         "WRITE":        0b00001000,
         "NOTIFY":       0b00010000,
         "INDICATE":     0b00100000,
         "WRITE_SIGNED": 0b01000000,
         "EXTENDED":     0b10000000,
}

I also have code in which I check for a connection and, if necessary, reconnect. That seems to be working. Recently, as my code and the timings and order of things in it have become more stable, I have not seen disconnects. Also, I had one peripheral board that was behaving strangely, that might have been a contributing factor: it decided that it’s address was no longer PUBLIC but became RANDOM. So I check for the addrType before connecting (there is not always a device name). That seems critical to prevent the connect from hanging:

def handleDiscovery(self, dev, isNewDev, isNewData): if isNewDev: device_name = '' # try to get a device name try: device_name = dev.scanData[9].decode() except: pass logger.infohttp://logger.info('Device ' + dev.addr + ' ' + dev.addrType + ' RSSI: ' + ' ' + str(dev.rssi) + ' ' + device_name) elif isNewData: logger.infohttp://logger.info("Received new data from " + str(dev.addr))

After discovery, I check the device_names to find the one I need to connect to.

def discover(self): scanner = Scanner().withDelegate(BGTC_Delegate()) devices = scanner.scan(2) # scan for two seconds

# look for a BGTrap Controller
for device in devices:
    name = device.getValueText(9)           # get the device name
    if name == "BGTrap Controller":         # a bg trap controller was found!
        logger.info<http://logger.info>('BG Trap Controller Found!')
        self.present = True
        self.device = device
        self.addr_type = self.addr_types[device.addrType]
        return
# no controller was found
logger.info<http://logger.info>('No BG Trap Controller found!')
self.present = False

Then, I check the addrType and connect:

def connect(self): if not self.present: return # no device if self.isConnected(): return # no action, already connected

self.peripheral.connect(self.device.addr, self.addr_type)   # connect to the device
self.peripheral.withDelegate(BGTC_Delegate())
self.tx_characteristic = self.peripheral.getCharacteristics(
        uuid='e7add780-b042-4876-aae1-112855353cc1'
        )[0]
# set client characteristic configuration
self.peripheral.writeCharacteristic(9, b'\x02\x00') # enable indications

def isConnected(self): try: if self.peripheral.getState() == 'conn': return True except: return False

IanHarvey / bluepy

waitForNotifications not consistently working for indications #360

for development: get the details of the device

NOTE - This class is directly inheriting bluepy.btle.DefaultDelegate for notifications,

but the delegate could be a separate class

Currently only READ is used in supportsRead function,