Data compression in logging

mrbdahlem / SugarCookie

A home repo for SugarCookie development and associated projects

https://mrbdahlem.github.io/SugarCookie/

1 stars 0 forks source link

Open mrbdahlem opened 3 years ago

mrbdahlem commented 3 years ago

Here's a simple compression idea.

In .csv, replace "," string, with an END_OF_FIELD character. The last character in the field (the last number or letter) has its MSB turned on. This should save > 10%
also replace "\n" END_OF_RECORD or END_OF_FIELD\n sequence. will save two bytes per record

mrbdahlem commented 3 years ago

save as ".ccs" file instead of .csv to indicate it needs to be expanded