Find a new Telemetry Method (Away from JSON)

[rcastle1]

What do you want to change? This might have just been my perspective, but it seemed like it took longer to export data to excel from the JSON file than the actual vacuum test took. Consider a python script or different utility to make this exporting process quicker.

Why make that change? More data -> more faster -> more better

Describe alternatives you've considered

Additional context With an actual HITL setup, I could see us running tests overnight (click valves at like 0.1Hz and check for failures). It would be suck to have to wait an additional 12 hours for data to review the data

[vkottler]

I noticed another JSON-related issue in pod and wanted to re-iterate my "please please please do not use JSON" position on this topic.

You all made so many great engineering evolutions for Badgerloop with this pod iteration, JSON as a means to encapsulate telemetry data was an oversight and I think now is as good of a time as any to re-implement the telemetry protocol / schema now that the performance issues have manifested in practice.

Even if you don't attempt a fancy dynamic protocol with metadata to parse the body, just write raw structs to a file or over a network stream where the data elements are packed big-endian (network byte order) and parse it that way on the other end. If you want import/export-able data write a Python binary -> CSV converter as Ryan is suggesting.

[vkottler]

+ @ezraboley

[ezraboley]

Yeah, I'd agree with this. I think it makes sense to investigate more scalable ways to manage our data and telemetry. Currently even with the telemetry we have it is becoming a bit of a coding mess, and there is never a better time to make a switch than at the beginning of a new comp.

Plus I realllly do not want to be debugging some JSON library.

Also would like to bring in @rdaruwala

[ezraboley]

It would also remove a large dependency from the embedded repo, and you know how we feel about dependencies

[rdaruwala]

Yup! Strong agree that moving away from JSON is the way to go. The irritating errors in dealing with 3rd-party libraries is just more reason to do so