Previously we had IMU parse fails due to sunlink unable to keep up with messages coming in. We confirmed this by checking the raw UART stream coming to my comuter's port and noticed that all messages were indeed sent and sunlink could not get all the messages because it was too slow.
To fix this, I implemented a simple chunking method where we read 512 bytes from the stream at a time (variable and can be adjusted. See next steps). Once the chunk is received, it is extracted for the messages inside it which are then sent to the parser.
To verify this algorithm I ran it with a load of 1926 bytes per second (78 imu messages and 3 GPS messages ran in their respect tasks instead of the normal 13 and 1) and it got all the data except the last 13 or so messages which could have still been in the 512 byte chunk (cannot access this until the full amount is read). Quantitatively, 614/623 lines were received where all lines before line 616 matched the SD log.
Downside:
With this method you will have to parse messages in bursts, however, it guarantees all messages are parsed. The lag is about 0.5 seconds but will be reduced if the chunk is filled faster.
Specific Changes:
removed unused imports
added CHUNK_SIZE constant
added process_message function which takes in the 512 byte chunk and returns a list of all the valid messages
added the sendToParser function to abstract the functionally of creating a payload, sending to the parser, and expecting a callback.
added appropriate changes in main of link_telemetry to use these functions.
Next Steps:
Perhaps using a multiprocess approach (different from multithreading so that we can actually achieve parallelization to boost throughput in case of parse fails later on).
We can optimize the algorithm with a better choice of CHUNK_SIZE in the case that parse fails are received.
Previously we had IMU parse fails due to sunlink unable to keep up with messages coming in. We confirmed this by checking the raw UART stream coming to my comuter's port and noticed that all messages were indeed sent and sunlink could not get all the messages because it was too slow.
To fix this, I implemented a simple chunking method where we read 512 bytes from the stream at a time (variable and can be adjusted. See next steps). Once the chunk is received, it is extracted for the messages inside it which are then sent to the parser.
To verify this algorithm I ran it with a load of 1926 bytes per second (78 imu messages and 3 GPS messages ran in their respect tasks instead of the normal 13 and 1) and it got all the data except the last 13 or so messages which could have still been in the 512 byte chunk (cannot access this until the full amount is read). Quantitatively, 614/623 lines were received where all lines before line 616 matched the SD log.
Downside:
Specific Changes:
process_messagefunction which takes in the 512 byte chunk and returns a list of all the valid messagessendToParserfunction to abstract the functionally of creating a payload, sending to the parser, and expecting a callback.Next Steps: