FLASC often works with large amounts of data. This becomes particularly apparent if the user is working with 1Hz data rather than 10-minute averages. To reduce file size and memory usage, dataframes can be formatted to an appropriate precision.
Namely:
Almost all floats can be saved as 32-bit floats rather than 64-bit floats, since the additional precision will drown in measurement noise anyway.
According to @paulf81 strings are better saved as categories or chars, not just objects
True/False columns should be saved as booleans, not integers
This can generally reduce the size of the dataframes by typically 50%, sometimes more. A simple function can be written and added to dataframe_manipulations to reduce the precision of any dataframe.
Description
FLASC often works with large amounts of data. This becomes particularly apparent if the user is working with 1Hz data rather than 10-minute averages. To reduce file size and memory usage, dataframes can be formatted to an appropriate precision.
Namely:
This can generally reduce the size of the dataframes by typically 50%, sometimes more. A simple function can be written and added to
dataframe_manipulationsto reduce the precision of any dataframe.Related URLs
No response