Problem:
Time series buckets are written in static size buffer chunks, this is done as standard memory allocation technique. Therefore, currently a single time bucket can get defragmented over time for a given bucket. Reading across different slices for the same bucket may not be sequential when it comes to actual locality of data in-memory. Additionally, the buffers may be not be compact enough therefore wasting substantial disk space.
Compaction:
Compaction in case of Sidewinder is the process of merging these fragmented buffers using a better compression algorithm (optional). To reduce the amount of disk space used and improve linear read performance.
ambud
commented
6 years ago
Problem: Time series buckets are written in static size buffer chunks, this is done as standard memory allocation technique. Therefore, currently a single time bucket can get defragmented over time for a given bucket. Reading across different slices for the same bucket may not be sequential when it comes to actual locality of data in-memory. Additionally, the buffers may be not be compact enough therefore wasting substantial disk space.
Compaction: Compaction in case of Sidewinder is the process of merging these fragmented buffers using a better compression algorithm (optional). To reduce the amount of disk space used and improve linear read performance.