search

farrellf / TelemetryViewer

Data Visualization Tool
166 stars 69 forks source link

Checksum, sync word not needed for binary messages over TCP or UDP #25

Closed MiloMindbender closed 3 years ago

MiloMindbender commented 5 years ago

When using TCP or UDP for binary telemetry, neither the sync word or checksum is needed since both TCP and UDP will only deliver correct, complete packets. These really only make sense for serial connections, for TCP/UDP it's just excess characters.

Also, it would be nice if UDP supported having multiple COMPLETE messages in the same UDP packet. ie: if you have a 24 byte telemetry packet you could put a lot of these in a 1440 UDP packet. This helps avoid network congestion.

This tool has been really useful, thanks for releasing it!

farrellf commented 5 years ago

TCP and UDP are treated like streams. You can put several telemetry packets into a single TCP or UDP packet. Or even do the opposite, and split one telemetry packet across multiple TCP or UDP packets.

The checksum is optional.

A sync word is always a good idea and adds very little overhead to send or receive. With a tool like TelemetryViewer, you don't want to second guess if a problem is caused by the tool or by the device sending telemetry. TelemetryViewer is far from perfect, but it seems like a good idea to keep the sync word requirement.

MiloMindbender commented 5 years ago

For some reason I didn't realize the checksum was optional...

About the sync word, I use several different telemetry programs, yours, https://cosmosrb.com/, and some programs written here. The other programs don't use a sync word for TCP and UDP so it would be nice if I could turn that off in yours so all the programs could use the same message format.

If I could remove the sync word, then I could send the data to any of the programs without having to switch message formats.

Also, it's a bad idea to split a message across multiple UDP packets, if a UDP packet was lost you would get a partial message, go out-of-sync and it might take awhile to recover correctly even with the sync word. It's better to have the UDP sender NOT split messages across packets, then if a UDP packet is lost you will still be in sync.

farrellf commented 5 years ago

Alright, I'm open to making the sync word optional. It's a low priority for me, and I have little time available these days, but I'll keep it on my to do list.

The reason why I allow telemetry to be split into multiple packets is because it's possible for telemetry to be too big to fit into a single packet. It's not likely to be a bottleneck right now, but I have plans to add support for visualizing image sensor data, so each telemetry packet could be a megabyte or more -- too big to fit into a single network packet. Of course this should be done with TCP, since TCP will ensure the ordering of packets, but I allow it with UDP as well.

A big part of why I added TCP and UDP support was so the user could easily adapt TelemetryViewer to handle nearly any protocol or device. For example, I have started to test support for FTDI's FIFO245 mode by writing a little helper program that talks to the FTDI driver, receives telemetry, then "transmits" it to localhost via UDP packets. In cases like this, where the sender and receiver are both localhost, I'm not sure if packets could be received out of order.

farrellf commented 3 years ago

The latest release made sync words optional. If a sync word is used, you can now specify it's value instead of having to use 0xAA:

http://www.farrellf.com/projects/software/2021-07-24_Telemetry_Viewer_v0.8/ https://www.youtube.com/watch?v=NUBm9b8YQMQ&t=1226s