Saving List onto Arduino

[jmdelahanty]

Hello PowerBroker!

My name is Jeremy and I just discovered this package for reading/writing to an Arduino using Python. I'm trying to send a list of numbers that indicate a trial type for an experiment I'm working on. Generating the list externally through Python and sending it for use in the experiment would be so helpful for my team!

I've successfully gotten your package to communicate back and forth to the Arduino, but I'm not sure how to make sure the data I sent is actually saved and usable in the Arduino's memory. I'm not sure if this is the appropriate place to ask about this kind of testing since the package seems to work, but I didn't see a different way of trying to contact you.

Any advice would be greatly appreciated! It would be cool to contribute to the project as I learn and provide examples relevant for my team's use that others could use.

Sincerely,

Jeremy Delahanty Salk Institute for Biological Studies, La Jolla CA

[jmdelahanty]

Here's my attempt to accomplish something like this (I'm pretty new to Arduino, hopefully it's not too bad!):

#include "SerialTransfer.h"

SerialTransfer myTransfer;

int anArray[20];
byte arrayIndex = 0;

void setup() {
  // put your setup code here, to run once:

  Serial.begin(115200);
  myTransfer.begin(Serial);
}

void loop() {
  // put your main code here, to run repeatedly:
  if(myTransfer.available())
  {
    for (uint16_t i=0; i < myTransfer.bytesRead; i++)
      myTransfer.packet.txBuff[i] = myTransfer.packet.rxBuff[i];
      for (int i=0; i < 21; i++)
        anArray[arrayIndex] = myTransfer.packet.txBuff[i];

    myTransfer.sendData(myTransfer.bytesRead);
  }

}

[PowerBroker2]

Happy to help and I'm glad you at least have Python and your Arduino talking nice. In fact, if Python is telling you it's received data while the Arduino is running the exact code you posted, the data sent from Python is in fact saved in the Arduino's memory and is available for usage.

Also note that you can use rxObj() on the Arduino side to parse multi-byte objects out of the received packet. This prevents you from having to bit shift, bit mask, and direct memory manipulation for things like floats, structs, etc. See the rx_data examples in the Arduino library examples for more guidance.

[jmdelahanty]

Thank you for the advice and so much for your help! I'm really grateful for your time. Before finding your package I was feeling pretty intimidated trying to get data sent to the Arduino. Your package seems like the ideal way for people to do this! I will check out the rxObj() documentation to learn more about how to parse packets.

What I noticed when using the python library was that my terminal was continuously printing the list communicated to and from the Arduino which makes sense because the example script waits for a Keyboard Interrupt. Since I only need to confirm the data was received properly once, is there a good way of introducing a break after I know it's been transferred?

[jmdelahanty]

It looks like doing something like this does the trick! But it would be great to know if this isn't a good way:

# After printing SENT/RCVD information:
if list_ == rec_list_:
    print("Confirmed!")
    try:
        link.close()
    except:
        pass

[PowerBroker2]

Here's the documentation for rxObj(), but it makes more sense in an example:

     uint16_t Packet::rxObj(const T &val, const uint16_t &index=0, const uint16_t &len=sizeof(T))
     Description:
     ------------
      * Reads "len" number of bytes from the receive buffer (rxBuff)
      starting at the index as specified by the argument "index"
      into an arbitrary object (byte, int, float, double, struct, etc...)
     Inputs:
     -------
      * const T &val - Reference to the object to be copied into from the
      receive buffer (rxBuff)
      * const uint16_t &index - Starting index of the object within the
      receive buffer (rxBuff)
      * const uint16_t &len - Number of bytes in the object "val" received
     Return:
     -------
      * uint16_t maxIndex - Index of the receive buffer (rxBuff) that directly follows the bytes processed
      by the calling of this member function

Also note that txObj() can do the same thing - just for transmitting.

Since I only need to confirm the data was received properly once, is there a good way of introducing a break after I know it's been transferred?

I think you have a really good approach so far, but don't forget to resend the packet if it wasn't echoed correctly (also might want to include a timeout function, and maybe autobaud, maybe auto port detection...lots of things to think about lol)

[jmdelahanty]

These are all great ideas! I'm hoping to implement best practices for our set up so it's easy to troubleshoot problems if anything goes wrong for people who use the setup. If there's additional things you recommend I implement I'd be happy to learn about them!

[jmdelahanty]

Hey again PowerBroker! I've encountered a new problem.

I'm having trouble referencing what I sent to the Arduino inside of its code.

I've written (read: copy/pasted from your example) a writing test file in Python that simply creates a random list of numbers and transmits them to the Arduino. Initially, the program was working as it sent and confirmed the list had successfully made it. Now, however, after trying to use the rxObj() function, it appears that the data gets sent but I don't get anything back.

Here's what my .ino file looks like:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

struct STRUCT {
  int x;
} trialStruct; // structure for containing received trial type array

int trialArray[20]; // create array that is 20 integers long

void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);
}

void loop()
{
  if(myTransfer.available())
  {   
  // used to track how many bytes we've processed from the receive buffer
   uint16_t recSize = 0;
   recSize = myTransfer.rxObj(trialStruct, recSize);
   Serial.print(trialStruct.x);
   recSize = myTransfer.rxObj(trialArray, recSize);
  }
}

The python test script using your example is below. I added a couple print statements to see where it could be getting stuck and it gets up to the "Data Sent" message, but hangs after that. It looks like this:

import time
import random
from pySerialTransfer import pySerialTransfer as txfer

trial_list = []
random.seed(10)

for i in range(0,20):
    n = random.randint(0,2)
    trial_list.append(n)

print("Trial List Created")

if __name__ == '__main__':
    print("Starting tx")
    try:
        link = txfer.SerialTransfer("COM13", 115200)

        link.open()
        print("Comms Opened")
        time.sleep(3) # give time for Arudino to reset
        print("Arduino Reset")

        while True:
            send_size = 0

            ###################################################################
            # Send a list
            ###################################################################
            list_ = trial_list
            list_size = link.tx_obj(list_)
            send_size += list_size
            link.send(send_size)
            print("Data Sent")

            while not link.available():
                if link.status <0:
                    if link.status == txfer.CRC_ERROR:
                        print("ERROR: CRC_ERROR")
                    elif link.status == txfer.PAYLOAD_ERROR:
                        print("ERROR: PAYLOAD_ERROR")
                    elif link.status == txfer.STOP_BYTE_ERROR:
                        print("ERROR: STOP_BYTE_ERROR")
                    else:
                        print("ERROR: {}".format(link.status))
            ###################################################################
            # Parse response list
            ###################################################################
            print("Receiving Response")
            rec_list_  = link.rx_obj(obj_type=type(list_),
                                     obj_byte_size=list_size,
                                     list_format='i')
            ###################################################################
            # Display the received data
            ###################################################################
            print('SENT: {}'.format(list_))
            print('RCVD: {}'.format(rec_list_))
            print(' ')

            if list_ == rec_list_:
                print("Confirmed!")
                try:
                    link.close()
                except:
                    print("Error!")
                    break
            # TODO: Add packet checking and retry if there's an error in comms

    except KeyboardInterrupt:
        try:
            link.close()
        except:
            pass

    except:
        import traceback
        traceback.print_exc()

        try:
            link.close()
        except:
            pass

Any clue what might be going wrong?

One last thing is that I've uploaded the .ino file to the board first and then I run the Python script. Does the order in which I run things matter?

Thanks again!

[PowerBroker2]

You have to first understand what each line in each program is doing before trying to mix and match them. The file transfer examples weren't made to work "copy and paste" with the Python example. More specifically, there's 2 things to keep in mind when using this library:

1. The data you send should match the type/length of the data you expect to receive. For example, you can't transmit a list and then try to parse a struct, you should be trying to parse an array.
2. You can't use the serial monitor and Python unless you're using 2 UART ports on the Arduino (one for Python and one for Serial Monitor debugging). What's happening in your case is you're sending debug strings to Python and since Python is seeing the strings as malformed packets, it's simply dropping them and saying it didn't receive any valid data.

[jmdelahanty]

I see what you mean, I definitely need to spend much more time learning what each line accomplishes before I try moving forward. I'll be sure to spend much more time practicing with the pieces before trying to put them together like I did here.

Your points make a lot of sense. I really wasn't sure how the .ino file would properly save what I sent and I definitely need practice with the data types used in C++. I also hadn't known that the print statements would be interpreted as packets themselves so that's great to learn too.

Thanks for you help again, I'll be more careful. I hope I'm not a bother.

[PowerBroker2]

No bother, it's a huge compliment whenever someone wants to use one of my libraries

[jmdelahanty]

Good evening! So I think I'm starting to understand things a little better, but I'm somewhat confused about the rxObj() function I'll try and say what I think it does and what I need to do:

rxObj

1. The Python library puts my list into one packet of a specific size and sends it to the Arduino.
2. On the Arduino, I initialize an array of the correct size
3. The Arduino creates a receive/transmit buffer using myTransfer.begin()
4. We initialize a received size of 0 for the function
5. recSize is updated using myTransfer.rxObj(array, recSize) because rxObj() returns the size of the transferred object as obtained by the last index's position. Since I initialized the array as the same size as what Python is delivering, the rxObj() function can parse it correctly.

I hope that I have that part correct.

What I'm not sure about is this. Is the first parameter of rxObj() the array I sent via Python? Or is it the empty array I initialized at the start? If it's the former, I'm not sure how I access it and save it. I tried simply saying return trialArray but I haven't seemed to successfully get that to work yet. Here's what the .ino file looks like:

// Use package SerialTransfer.h from PowerBroker2 https://github.com/PowerBroker2/SerialTransfer
#include "SerialTransfer.h"

// Rename SerialTransfer to myTransfer
SerialTransfer myTransfer;

// Create array that is 20 integers long
int trialArray[20];

// Create setup
void setup()
{
  Serial.begin(115200); 
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

// Try rxObj()
void rxObjTransfer()
{
  if(myTransfer.available())
  {
    uint16_t recSize = 0;
    recSize = myTransfer.rxObj(trialArray, recSize);
  }
  return trialArray;
}

// Try looping it
void loop() 
{
  rxObjTransfer();
  Serial.print(trialArray[1]);
}

I'm also getting an error when I try to compile this because I'm trying to return a value in a function declared as void, but trying to state declare the function as int tells me that I'm doing an invalid conversion from int* to int. This means that I'm trying to convert a pointer to an integer correct? I've been trying to use C++ beginner tutorials online so I can do it properly, but am still confused by how to use pointers.

[PowerBroker2]

Here's a more detailed explanation on what the function does (you had it pretty close):

    template <typename T> // Use a template so user can parse an object of ANY type (except for a class)
    uint16_t rxObj(const T& val, const uint16_t& index = 0, const uint16_t& len = sizeof(T))
    {
        uint8_t* ptr = (uint8_t*)&val; // Create a pointer to the object passed in the first argument (the data parsed will be saved at the location this pointer "points to"
        uint16_t maxIndex; // This is the last index within the rxBuff array to be included when parsing

        if ((len + index) > MAX_PACKET_SIZE) // Don't access memory outside of the rxBuff array
            maxIndex = MAX_PACKET_SIZE;
        else
            maxIndex = len + index;

        for (uint16_t i = index; i < maxIndex; i++) // Process all required bytes
        {
            *ptr = rxBuff[i]; // Translation: Set the value of the byte in memory address "ptr" to what byte is in rxBuff[i] - This is basically a single-byte memory copy
            ptr++; // Increment the pointer to the next byte in memory
        }

        return maxIndex; // Let the user know how many bytes were processed
    }

Basically, rxObj() is a specialized memcpy implementation that copies the required bytes from the rxBuff to directly overwrite the bytes in memory comprising val. Even simpler: it just overwrites the passed variable with the corresponding bytes in the receive buffer.

Does any of that make sense?

What I'm not sure about is this. Is the first parameter of rxObj() the array I sent via Python?

Yes, IF the array you're trying to send is less than 255 bytes in total. Otherwise, you'll have to break up the array into multiple pieces and it's a total pain.

I'm not sure how I access it and save it

rxObj() saves the data for you. Also, since trialArray is a global, you can simply call it and use it where and how you like.

I've been trying to use C++ beginner tutorials online so I can do it properly, but am still confused by how to use pointers.

Pointers are one of the hardest things to learn, but this tutorial is one of the best I've ever seen. I don't think you should need to worry about pointers too much for what you're trying to do

[jmdelahanty]

Hey again PowerBroker,

Sorry for not responding for a few days. I feel like I'm so close to getting this working, but there must be something simple in my way. I see you've closed this, but I'll try to describe where I'm at now.

When I'm trying to communicate to the Arduino Mega, I currently upload the .ino to the board over a USB cable. In my searches online to use Serial1, it appears that I need an additional cable that can run into the Arduino using a USB to TTL connection! I'm guessing that's why I can't get any responses from the Arduino in my code using Serial1!

That's what I've seen online at least and I wanted to ask if this was the cause of my troubles. Do I need to purchase a USB to TTL cable to transfer the information when using Serial1?

Hoping to not have egg on my face,

Jeremy

[PowerBroker2]

No problem! Yeah, if you want to communicate with Python over USB and have debug prints on your serial monitor, you'll need a USB to UART converter like you mentioned (these are good for prototyping and these are pretty good for PCB work)

[jmdelahanty]

Okay, awesome! To make sure I got this right, I can just use a normal USB cable connected to the computer and connect the USB-UART converter. I'll get the prototyping one first to try it out.

Here's what I understand for each of the pins it has afterwards: VCC: Power input from the Arduino GND: Ground connection to Arduino TXD: Transmission pin that goes to TX1/2/3 connection on Arduino RXD: Receiving pin that goes to RX1/2/3 connection on Arduino RST: Request to send pin, not sure where this should go... just an output channel from Arduino? CTS: Clear to send, not sure where this should go... another output channel from Arduino?

I've ordered one and it'll be here in a couple days!

[PowerBroker2]

GND, TXD, and RXD are the only ones you need to wire up for your purposes

[jmdelahanty]

Great to know! When it gets here I'll let you know how it goes! I think actually talking to the Arduino correctly will be pretty helpful haha

[jmdelahanty]

Good evening! So I'm trying out the board and, so far, am not having much luck. Here's the problem I'm encountering.

First, when trying to communicate over the port it's connected to on Serial1, I keep getting an error telling me that permission is denied. I'm unsure if that's because the port itself is unavailable from Windows or if the Arduino isn't listening to the Serial1 pins. I have a function in my main .ino file that looks like this:

void trials_rx() {
  if (acquireTrials) {
    Serial.println("STARTING TRANSFER");
    if (myTransfer.available())
    { Serial.println("TRANSFERING...");
      uint16_t recSize = 0;
      recSize = myTransfer.rxObj(trialArray, recSize);
      myTransfer.sendData(myTransfer.bytesRead);
    }
    acquireTrials = false;
    newTrial = false;
  }
}

I have those print statements in there because I'm trying to make sure that the board is indeed communicating with my Python script.

Through this debugging, I only get as far as "STARTING TRANSFER". It appears that myTransfer's status is unavailable then, right? Do I need to set a pinMode for the Tx/Dx pins?

Here's my set up code:

void setup() {
  Serial.begin(9600);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
  ... initialize pins, etc ...
}

And here's my loop:

void loop() {
  trials_rx();
  if (currentTrial < totalNumberOfTrials) {
  ... rest of experiment ...
}

My goal is for pySerialTransfer to receive the trials, send back to Python the confirmation that the trials have indeed been transferred, and then move through those trials one by one for the experiment. I've been going at this for a bit today and seem to be stuck getting trials sent within the main program.

Also, one thing that I've thought of trying to do was have Python listen for a "Ready to receive signal" from the Arduino and, when received, send the trials, confirm they're stored on the Arduino, and then close the connection. Is that kind of additional control what's necessary for this to work?

[PowerBroker2]

First, when trying to communicate over the port it's connected to on Serial1, I keep getting an error telling me that permission is denied

That means that another program is using that port. Try closing all other apps that might be using it (i.e. serial monitors, putty, etc) and unplug/replug it back into your computer.

Do I need to set a pinMode for the Tx/Dx pins?

Nope

Overall it looks like your program is too complex to easily debug. Start small and work your way up. Try the following:

Python:

import time
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM#') ########## replace COM port number ##########
        link.open()

        while True:
            list_ = [1.5, 3.7]
            link.send(link.tx_obj(list_))
            time.sleep(1)

    except KeyboardInterrupt:
        try:
            link.close()
        except:
            pass

    except:
        import traceback
        traceback.print_exc()

        try:
            link.close()
        except:
            pass

Arduino:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

float list[2];

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);

  myTransfer.begin(Serial1);
}

void loop()
{
  if(myTransfer.available())
  {
    myTransfer.rxObj(list);

    Serial.println("Received:");
    Serial.print(list[0]);
    Serial.println(list[1]);
    Serial.println();
  }
}

NOTE: Above code is uncompiled and untested, but should work

ALSO: Remember to keep straight which COM port is for debugging and which one is for Python coms

[jmdelahanty]

That means that another program is using that port. Try closing all other apps that might be using it (i.e. serial monitors, putty, etc) and unplug/replug it back into your computer.

That was the issue! I had thought I closed all my apps that were using it but missed one. Thankfully that was simple!

So I've tried running your code and, while I see the TX signal on the board blinking which I think means it's communicating, the Arduino doesn't seem to be responding to Serial1. The Arduino loop doesn't seem to get past if(myTransfer.available()). Any thoughts on why? I've been looking into the code for the function but can't see why your code wouldn't work. I've tried running Python before uploading to the Arduino and vice versa with the same result, if that helps at all. Maybe I'm using the board incorrectly?

[PowerBroker2]

I should've warned you, but forgot: I do all serial comms at 115200 baud if possible. 9600 is painfully slow. A baud mismatch in the serial monitor might be the issue here. Are both TX and RX LEDs lit?

[PowerBroker2]

Ok, so I think using Serial for data transfer doesn't play nice with the debug mode or something like that. I successfully tested the following code using Serial1 for Python data and Serial for debug messages:

Python:

import time
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM4') ########## replace COM port number ##########
        link.open()

        while True:
            list_ = [1.5, 3.7]
            link.send(link.tx_obj(list_))
            print("Sent: {}".format(list_))
            time.sleep(1)

    except KeyboardInterrupt:
        try:
            link.close()
        except:
            pass

    except:
        import traceback
        traceback.print_exc()

        try:
            link.close()
        except:
            pass

Arduino:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

float list[2];

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);

  myTransfer.begin(Serial1, true);
}

void loop()
{
  if(myTransfer.available())
  {
    myTransfer.rxObj(list);

    Serial.println("Received:");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();
  }
}

[jmdelahanty]

I should've warned you, but forgot: I do all serial comms at 115200 baud if possible. 9600 is painfully slow. A baud mismatch in the serial monitor might be the issue here.

I was wondering why you used 115200, that's good to learn it's a bit faster.

Your code works! Nice job!

Are both TX and RX LEDs lit?

Only the TX LED is lit which makes sense because the Arduino isn't sending anything back yet right? I've tried simply adding a myTransfer.txObj(list); line to Arduino and the following from your example to the Python code:

send_size = 0

            list_ = [1.5, 3.7]
            list_size = link.tx_obj(list_)
            send_size += list_size
            link.send(send_size)
            print("Sent: {}".format(list_))
            time.sleep(1)

            rec_list_  = link.rx_obj(obj_type=type(list_),
                                     obj_byte_size=list_size,
                                     list_format='i')
            print("Received: {}".format(rec_list_))
            time.sleep(1)

It seems like Python is receiving something at least from the Arduino because I get an error from the rxBuff function that looks like this:

Sent: [1.5, 3.7]
Receiving
Traceback (most recent call last):
  File "pyserial_test.py", line 21, in <module>
    rec_list_  = link.rx_obj(obj_type=type(list_),
  File "C:\Users\jdelahanty\.conda\envs\arduino_python\pySerialTransfer\pySerialTransfer.py", line 331, in rx_obj
    buff = bytes(self.rxBuff[start_pos:(start_pos + obj_byte_size)])
TypeError: 'str' object cannot be interpreted as an integer

Is this because I'm using an incorrect list_format declaration maybe? Or does the Arduino send back a string for some reason? I've tried finding out what type list is but can't seem to get that to work on Arduino. The internet tells me that it's because the Arduino compiler just isn't too heavy duty. I thought it was still a float since that's what we declare it as at the start.

[PowerBroker2]

list_format should be 'f' for float, since the contents of the list are floats

[jmdelahanty]

It's still giving me that same error even with the correct list_format parameter. I've tried changing the list to different types (i.e. int) on both Python and Arduino and get the same error complaining about a string. Could I have accidentally changed the type somewhere in my usage of the code on accident?

[PowerBroker2]

ohhh, hold up - you're not testing in the Python script if you've received a packet via link.available(). You need to call that function until it returns true. Then you can call link.rxObj(). You still need 'f' as the list_format, though

[PowerBroker2]

If you post both scripts in full, I might be able to id any other bugs.

Also, sorry for things being this difficult to get working. I probably should have better documentation...

[jmdelahanty]

It's no problem at all! I'm so grateful for your help. This is going to make running and organizing our work much easier.

Currently, people in the lab just hand code in the trials they'd like to do. When we get this going and with a little GUI that I'll try to make, people can have a configuration file generated that we can access instead of having to go into the Arduino's .ino every time we want to see what happened for a given trial set. Hopefully, there will just be a set .ino file that we can use that we don't need to make copies of!

You need to call that function until it returns true. Then you can call link.rxObj().

So we can use something like while link.available != True after the sending piece of code?

Here's the Arduino script:

// Use package SerialTransfer.h from PowerBroker2 https://github.com/PowerBroker2/SerialTransfer
#include "SerialTransfer.h"

// Rename SerialTransfer to myTransfer
SerialTransfer myTransfer;

float list[2];

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);

  myTransfer.begin(Serial1, true);
}

void loop()
{
  if(myTransfer.available())
  {
    myTransfer.rxObj(list);

    Serial.println("Received:");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();

    myTransfer.txObj(list);
    Serial.println("Sent");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();
  }
}

And here's Python:

import time
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM12', 115200) ########## replace COM port number ##########
        link.open()

        while True:
            send_size = 0

            list_ = [1.2, 3.7]
            print(type(list_))
            list_size = link.tx_obj(list_)
            send_size += list_size
            link.send(send_size)
            print("Sent: {}".format(list_))
            time.sleep(1)

            print("Receiving")
            print(type(list_))
            rec_list_  = link.rx_obj(obj_type=type(list_),
                                     obj_byte_size=list_size,
                                     list_format='f')
            print("Received: {}".format(rec_list_))
            time.sleep(1)

    except KeyboardInterrupt:
        try:
            link.close()
        except:
            pass

    except:
        import traceback
        traceback.print_exc()

        try:
            link.close()
        except:
            pass

[PowerBroker2]

Turns out there was an issue with the Arduino sketch. Note that txObj() and rxObj() functions (both in Arduino and Python) do nothing but encode and parse (respectively) the data. In order to actually transmit data, you have to use send() (Python) and sendData() (Arduino). On the Arduino side, you can also use sendDatum() to transmit a single object. In order to receive data before parsing, you need to call available() (both Arduino and Python). This function is what actually parses packets.

Here is the confirmed working code:

Python:

import time
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM4', 115200, debug=True) ########## replace COM port number ##########
        link.open()

        while True:
            list_ = [1.2, 3.7]

            list_size = link.tx_obj(list_)
            link.send(list_size)
            print("Sent:\t\t{}".format(list_))

            while not link.available():
                pass

            rec_list_ = link.rx_obj(obj_type=type(list_),
                                    obj_byte_size=list_size,
                                    list_format='f')
            print("Received:\t{}".format(rec_list_))
            print("")
            time.sleep(1)

    except KeyboardInterrupt:
        try:
            link.close()
        except:
            pass

    except:
        import traceback
        traceback.print_exc()

        try:
            link.close()
        except:
            pass

Arduino:

// Use package SerialTransfer.h from PowerBroker2 https://github.com/PowerBroker2/SerialTransfer
#include "SerialTransfer.h"

// Rename SerialTransfer to myTransfer
SerialTransfer myTransfer;

float list[2];

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);

  myTransfer.begin(Serial1, true);
}

void loop()
{
  if(myTransfer.available())
  {
    myTransfer.rxObj(list);

    Serial.println("Received:");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();

    list[0] = 4.5;
    list[1] = 9.2;

    myTransfer.sendDatum(list);
    Serial.println("Sent");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();
  }
}

[jmdelahanty]

Awesome! It's working great! One interesting thing it's doing is giving me a few additional decimal values upon receiving them on the Python side.

Sent:           [1.2, 3.7]
Received:       [1.2000000476837158, 3.700000047683716]

The Arduino says it's receiving 1.20 and 3.70. Could that additional zero be interpreted differently when Python is parsing the information? The list stays the same when using an integer value.

[PowerBroker2]

This is an issue with how floating point values work and their limitations in precision. I don't believe there is any corruption or loss in the data. If there was, you'd probably get a CRC error and the packet would be dropped.

[jmdelahanty]

Got it, thanks for pointing out that resource for me to learn.

It looks like there's a new issue when I explicitly use integers as the list. For example, substituting [1, 3] as the list_ parameter yields the Arduino saying it's sending and receiving [1,0]. I can send whatever integer I'd like in the first position, but I always get 0 in the next. Python also crashes when receiving the list with the same error as before. I'm using list_format='i' in the rx_obj function in Python so I must be doing something incorrectly on accident. Any ideas?

Here's the code just in case: Arduino

// Use package SerialTransfer.h from PowerBroker2 https://github.com/PowerBroker2/SerialTransfer
#include "SerialTransfer.h"

// Rename SerialTransfer to myTransfer
SerialTransfer myTransfer;

int list[2];

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);

  myTransfer.begin(Serial1, true);
}

void loop()
{
  if(myTransfer.available())
  {
    myTransfer.rxObj(list);
    Serial.println("Received:");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();

    myTransfer.sendDatum(list);
    Serial.println("Sent");
    Serial.print("[");
    Serial.print(list[0]);
    Serial.print(", ");
    Serial.print(list[1]);
    Serial.println("]");
    Serial.println();

    myTransfer.sendDatum(list);
  }
}

Python

import time
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM12', 115200, debug=True) ########## replace COM port number ##########
        link.open()

        while True:
            list_ = [1, 3]

            list_size = link.tx_obj(list_)
            link.send(list_size)
            print("Sent:\t\t{}".format(list_))

            while not link.available():
                pass

            rec_list_ = link.rx_obj(obj_type=type(list_),
                                    obj_byte_size=list_size,
                                    list_format='i')
            print("Received:\t{}".format(rec_list_))
            print("")
            time.sleep(1)

    except KeyboardInterrupt:
        try:
            link.close()
        except:
            pass

    except:
        import traceback
        traceback.print_exc()

        try:
            link.close()
        except:
            pass

[PowerBroker2]

Python uses 32-bit integers by default and I'm guessing your Arduino defaults to 16-bit integers. Try int32_t list[2];

[jmdelahanty]

That was it! Dang, I'm learning tons. Thanks so much! That works perfect.

Now that I can get lists to and from the Arduino with Python, getting this into my main function is the next task. First, I tried to just add this function alone using what you've helped me create so far. That works great. When I try to add it into the Arduino's loop, however, it seems like the Arduino can't listen any longer. I'm not getting a "Received" print statement in the debugger.

Function for Arduino:

void trials_rx() {
  if (acquireTrials) {
    if (myTransfer.available())
    { 
      myTransfer.rxObj(list);
      Serial.println("Received");

      myTransfer.sendDatum(list);
      Serial.println("Sent");
    }
  }
  acquireTrials = false;
}

void loop() {
  trials_rx();
}

I'm feeling a little lost about why it might be failing. Any advice?

[PowerBroker2]

Try this:

void trials_rx() {
  if (acquireTrials) {
    if (myTransfer.available())
    { 
      myTransfer.rxObj(list);
      Serial.println("Received");

      myTransfer.sendDatum(list);
      Serial.println("Sent");

      acquireTrials = false;
    }
  }
}

[jmdelahanty]

That worked! Cool! So my flow control was wrong with where I had the acquireTrials flag? I'm sure it matters where you place it, but I'm not sure how to know if where I put it is wrong without just running the program over and over again. Is trial and error the best route or is there a way to know where certain flags should go?

[PowerBroker2]

Yeah, available() isn't a blocking function. If you don't have a full packet to parse in the Serial input buffer during the first iteration of the loop, it returns false and, in your code, the acquireTrials flag is cleared - preventing available() from being called again. I moved the flag clearing to inside the if-statement that is only true when a new packet is "available"

[jmdelahanty]

That makes sense! Awesome work the Broker of Power.

Using it in this way allows me to control our experiment through Python now! With some additional tinkering, I think having a configuration created in Python sent to the Arduino is something I can do nicely now. How do I thank you!?

[PowerBroker2]

Glad to help! From your bio I can see you work in a neuroscience lab and from your replies it seems you intend to use the library for lab work. I would love to know what sort of impact my code/help has been in a more detailed way (if possible) to help boost my resume. I can send you a linkedin request email if you want to discuss in a more private forum

[jmdelahanty]

I definitely will be using it for our work! I'm happy to talk with you more over email or LinkedIn, whichever you'd prefer. Eventually, when our work is out, I'll also make sure to cite your code!