Working on #47, I realized that there is much to gain from trace compression. Since storage is the bottleneck, we may also achieve better performance.
Now there are some steps to optimize all for compressed traces.
[x] Optimizations in the trace format. For uncompressed traces, bit-tracing was optimal. However, it seems to be that the text trace could be better compress. This trace has one byte for each if/else element of just one bit. It seems that compression algorithms better handle this byte-wise structure than bit-wise structure. So the idea is that this new trace format combines text and compact traces.
[ ] Compression and storing in parallel. Not every platform might have compressed filesystems. Furthermore, we might get a performance penalty from the filesystem. So the idea is to do the compression as a separate thread directly in our application (the same as the RR debugger does).
[ ] Select optimal compression algorithm. Again, we can look into the RR debugger for inspiration. They use Brotli Level 5 but we can also use zstd.
Working on #47, I realized that there is much to gain from trace compression. Since storage is the bottleneck, we may also achieve better performance.
Now there are some steps to optimize all for compressed traces.