Saving to CSV and saving to DB is slow

[bastianilso]

Currently our data logging workflow is like this: 1) Data is collected into a dictionary structure and stored in RAM for highest performance during data collection. 2) When "Save data" is pressed, the dictionary structure is looped through and saved as CSV. This is a synchronous process which blocks the UI.

When collecting several minutes of data, saving data takes a very long time. How can we adjust this?

Here are some ideas:

• Investigate whether we can concatenate strings more efficiently using e.g. functions from the String.Builder class like Join() or concat() instead of the + sign. See https://stackoverflow.com/a/21093 We should try to move as much of the operation into lower level programming language as possible.
• Investigate whether we can build each data as a string row as one string on the fly (but still just keep in RAM). - this would save us a for loop later during the save process.
• We could move the saving process into an asynchronous thread and show a progress bar while saving. This at least gives people some indication how much time is left.
• Finally, we can asynchronously send/save data every 10 seconds to either database or to disk temporary file on disk (appending). However, the important part is to not create too much extra performance overhead during gameplay, e.g. fx when we have CPU-heavy tasks like eye tracking in Whack-A-Mole VR.

[bastianilso]

The logging code and database saving code both come from the LoggingManager package. So @Vincent-SD if you find issues with logging, please verify that the issue exist in LoggingManager repository too, so all Unity projects can benefit.

https://github.com/med-material/LoggingManager

As a first step, please update the Logging Manager in the ArduinoLogger to use the latest version provided by the repository above.

[Vincent-SD]

Maybe this issue can be closed, what do you think @bastianilso ? See #46