Reading from CharacteristicBuffer returns zero-padded buffer when a timeout occurs

[anthonytw]

In the event of a connection timeout the read method will return a buffer with some data up front, zeros padding the end, and no indication of how many bytes were actually read. There's no way to distinguish between zeros padding the data and zeros occurring immediately before the timeout.

The issue happens because the read function looks like this:

buffer = bytearray(
    min(nbytes, self._buffer_size) if nbytes else self._buffer_size
)
if self.readinto(buffer) == 0:
    return None
return buffer

buffer is a zero-filled array, then readinto does this:

length = len(buf)
idx = 0
end = time.time() + self._timeout
while idx < length and time.time() < end:
    try:
        buf[idx] = self._queue.get_nowait()
        idx += 1
    except queue.Empty:
        # Let the BLE code run for a bit, and try again.
        adapter.await_bleak(asyncio.sleep(0.1))

return idx

Notice if a transaction timeout occurs, the bytes read is returned, but the read function neither stores this information nor resizes the buffer. This leads to zero-padded data any time a transmission takes too long, which is impossible to differentiate from "real" zeros. Besides adding junk to the data stream, it also breaks up and invalidates messages. This is considerable in higher throughput applications. In my application, data is transmitted at 8KB/s. The zero padding added an additional 30% on top of this, and invalidated a lot of messages!