Note on wider applicability and compatibility

[Wind4Greg]

Thanks for the great library! Just a couple of points that could be in the documentation if desired.

For those looking to use this library with other boards and other serial interfaces here is the approach I used. I applied this to a Feather M0 board which has an additional Serial1 interface defined.

Outside of any functions:

PacketSerial serial1; // Create a specific wrapper, you can call it anything

Inside of the Arduino setup() method:

    Serial1.begin(9600); // Starting AdaFruit's additional serial interface
    serial1.begin(&Serial1); // Wrap it with PacketSerial instance
    serial1.setPacketHandler(&onPacket); // Use it for receiving...

Note 1: I used PacketSerial on Serial1 while using still using Serial for debugging output and such. So unless I read your code wrong, it seems that one can apply PacketSerial to some serial interfaces and not others based on how one wants to use the serial interface.

Note 2: I experienced full compatibility of the COBS implementation with a nice Python COBS implementation receiving data from PySerial. These folks have a good reference to the COBS Transactions on Networking (TON) paper.

Once again thanks for the library! Cheers Greg

[igor47]

Ditto! This library rocks. I used it on a Maple Mini clone, where it builds just fine with platformio. The other end was a Raspberry Pi running the python COBS implementation. My code is at igor47/spaceboard specifically in https://github.com/igor47/spaceboard/blob/master/src/main.cpp (for the C++ code) and https://github.com/igor47/spaceboard/blob/master/spaceteam/microcontroller.py (for the python) and is probably a nice example of two-way comms between the two languages.

my application was to drive a ws281x led strip (hard to do on a raspberry pi) and using cobs i was able to efficiently transfer big chunks of data for setting large number of led colors at once.

[bakercp]

I just added a note in the README pointing back here. It's a nice idea!

https://github.com/bakercp/PacketSerial/commit/8be082ea13a81be26972418e4c121baa90bef0c9

Thanks for the feedback.

[comperem]

This library, along with the python COBS library is outstanding. The COBS implementation provides encoding and decoding to for fast, robust binary data transfer between an Arduino and python on a PC or Raspberry Pi.

I'm getting 14 floating point numbers at 100Hz from an Arduino using 115200 baud.

The most difficult part of sending binary data over serial is figuring out the beginning and end of each message, or frame. Sending COBS-encoded messages makes that very straightforward when reading in python using serial.read_until(). You read the serial stream until the zero-byte is detected, b'\0x00. So, open a serial port and serial.read_until( b'\x00' ).

The example Arduino sketch below gathers an array of floats, encodes with COBS and sends the binary result over serial:

#include <PacketSerial.h>
PacketSerial myPacketSerial;

#define N_FLOATS 3

char  myChars[4*N_FLOATS]; // floats are 4-bytes each in Ardiuno
float myFloats[N_FLOATS];
unsigned int cnt=0;

void setup() {
  myPacketSerial.begin(115200);
}

void loop() {

  myFloats[0] = (float) micros();
  myFloats[1] = PI;
  myFloats[2] = cnt++;

  memcpy( myChars , myFloats, sizeof(myFloats) );

  myPacketSerial.send( myChars, sizeof(myChars) );

  delay(1000);
}

The python script below reads the incoming serial stream searching for the zero-byte, which COBS ensures is not in the data portion of the packets. Locating the zero-byte indicates the end of the packet. It then (1) decodes the COBS-encoded message and (2) converts the binary array of characters back into floating point numbers with struct.unpack(). The relevant python clips are below. The complete file is attached.

ser = serial.Serial('/dev/ttyACM0', baudrate=115200, bytesize=8, parity='N', stopbits=1, timeout=3)

zeroByte = b'\x00' # COBS 1-byte delimiter is hex zero as a (binary) bytes character

while True:
    try:
        str = ser.read_until( zeroByte ) # read until the COBS packet ending delimiter is found
        n=len(str)
        if n>0:
            decodeStr = str[0:(n-1)] # take everything except the trailing zero byte, b'\x00'
            res = cobs.decode( decodeStr ) # recover binary data encoded on Arduino
            n_binary = len(res)

            if (n_binary==expectedBytes):
                num = struct.unpack('fff',res) # voila - floats in python from the Arduino

Both sender and receiver files are attached. mySerialSenderToPython_simple.ino.txt

simpleReceiver.py.txt

[comperem]

Here's a round-trip example over serial from a microcontroller like Arduino or ESP32:

• from python to microcontroller

and:

• from microcontroller to Python over the same serial port.

The python example from @igor47 above helped clarify how to handle the terminating character. Without that terminating character the packetSerial() handler is never triggered.

It works like a champ using the python COBS library.

The demo sends and received 5 floating-point numbers.

Arduino or ESP32 receiver-sender code:

#include <PacketSerial.h>

PacketSerial myPacketSerial;

void setup()
{
  myPacketSerial.begin(115200);
  myPacketSerial.setPacketHandler(&onPacketReceived);
}

void loop()
{
  myPacketSerial.update();
}

void onPacketReceived(const uint8_t* buffer, size_t size)
{  
  myPacketSerial.send(buffer, size); // send buffer just received (unchanged) back over same serial port 
}

Python sender-receiver code. Upload Aduino or ESP32 sketch then let it reboot and listen on the serial port. Then run this to send 5 floats and listen for the return message with the same 5 floats:

import sys
import serial
import struct
import time
from cobs import cobs # smart binary serial encoding and decoding

serPort='/dev/ttyUSB1'
ser = serial.Serial(serPort, baudrate=115200, bytesize=8, parity='N', stopbits=1, timeout=3)
print('opened serial port: {0}'.format(serPort))

zeroByte = b'\x00' # COBS 1-byte delimiter is hex zero as a (binary) bytes character
cnt=0
tStart=time.time() #time.perf_counter()

while True:
    try:
        dataNums    = [1.23, 4.56, 0.0, float(cnt), time.time()-tStart]
        dataPacked  = struct.pack("fffff",*dataNums) # convert python list of 5 floats into packed binary data
        dataEncoded = cobs.encode( dataPacked ) # encode packed data as COBS message for serial send
        nBytes = ser.write( dataEncoded + b'\x00' ) # write data with COBS-terminating packet
        print( "python wrote {0} bytes to   {1}: dataNums={2}, dataEncoded={3}".format(nBytes,serPort,dataNums,dataEncoded) )
        while ser.in_waiting:
            # COBS ensures the zero-byte is *only* used as the packet-end delimiter
            strRead = ser.read_until( zeroByte )
            nBytesRead=len(strRead)
            if nBytesRead>0:
                decodeStr   = strRead[0:(nBytesRead-1)] # take everything except the trailing zero byte
                dataDecoded = cobs.decode( decodeStr ) # recover binary data sent from Arduino
                s = struct.Struct('fffff') # look at this binary data as 5 floats
                dataUnpacked = s.unpack(dataDecoded)
                print("python read  {0} bytes from {1}: {3}, {2}".format(nBytesRead,serPort,strRead,dataUnpacked))

        print('\n')
        time.sleep(0.1)

    except KeyboardInterrupt as err:
        print("caught keyboard ctrl-c:".format(err))
        print("exiting.")
        exit(0)
    except:
        print("Unexpected error:", sys.exc_info()[0])

[manchoz]

Hi all, At Arduino, we have included an example using PacketSerial in the Arduino_BHY2 library.

DataHarvester

The example comes with a companion Python app that works as a remote control and data collector. We distribute it as an example showing high-speed data capture for the Arduino Nicla Sense ME over UART/USB.

[dwijtsma]

Hello,

I'm doing something similar between a Pi Pico and a python application on my PC. What I do not understand is why python COBS library doesn't add the zero byte at the end. In your example you also add this manually.

I really love the simple interface of your library. It works like a charm. 👍