Determine likelihood of log file corruption.

[harrisonCassar]

Description/Motivation

Currently, we don't have a good sense of how likely/not-likely it is for our log files to corrupt. when writing to them during the DAQ operation. Therefore, we might run into an issue where we corrupt our log file early on in our test session, extracting data following that corruption will prove to be difficult (and perhaps impossible depending on the type of corruption). This problem is perhaps even a bigger problem until #8 is completed (depending on the implementation done there).

Possible corruption sources include:

• Upon write of new data.
• Upon opening file after RPi powered on.
• Upon closing file after performing some sort of file rotation (might be implemented as a part of #8).

Work to Be Done

This could involve understanding how we are currently saving the data, as well as perhaps research into Debian OS and/or RPi, and performing functional testing. Additionally, identifying a more complete list of possible corruption sources would be critical before completion.

Once this likelihood is better understood, we should consider ways of either: 1) Minimizing chances of corruption 2) Preventing any chance of corruption through proper safe filesystem choices 3) Minimizing consequences of file corruption (an example metric could be "number of data points lost", etc.). This work could be considered as follow-up, either done as a part of this ticket or another (TBD).

[harrisonCassar]

Labeling as low-priority due to extensive number of hours of operation of the Ground Systems without any apparent knowledge of file corruption.

We still want to definitely invest resources into doing this for the long-term, but in the meantime, let's table this in favor of some other higher-priority issues.

[coshio]

The corruption of the log file can be primarily be caused by power loss and overclocking (but I don't think we doing this). The main way to prevent log file corruption is being careful with the cutting power, shutting down, turning on, etc. Although, we can get a sense that this corruption is unlikely, and with the careful procedures listed (although definitely not guaranteed), it can help give us the best chance of the log file not corrupting. Especially with the extensive number of hours that Ground Systems is doing, this should be something that we should not be too worried about in the long term

Here are some links for avoiding this: http://www.ko4bb.com/getsimple/index.php?id=raspberry-pi-flash-corruption-avoidance https://forums.raspberrypi.com/viewtopic.php?t=36533

[harrisonCassar]

For future reference, before closing, be sure to request a review/ping a manager (Philip or I) just to double-check/review to see if the closure is valid!

But otherwise, looks good for now! I agree with your point on the number of hours we've been running the Ground Systems.

Thanks for doing this!