search

lks9 / src-tracer

Other
0 stars 0 forks source link

Trace Compression! #47

Closed lks9 closed 11 months ago

lks9 commented 1 year ago

I ran a short experiment to record the busybox test suit.

Recording User mode Kernel mode
Baseline 0.0 55s 15s
(a) inactive recording 0.0 1m17s 16s
Our approach (b) no function calls 9.2 GiB 2m29s 23s
(c) full 36.1 GiB 3m23s 48s
IntelPT with perf 11.0 GiB 1m17s 55s
RR debugger (rr v5.6.0) 1.0 GiB 20m25s 29m14s

It seems that the RR debugger outperformed our approach in size.

But than I realized that the rr debugger just uses compression. 👀

See here [1]. So I compressed the 36.1 GiB with brotli level 5 (same as rr) and I got only 222 MiB.

Task: Use compression in the source tracer.

[1] https://robert.ocallahan.org/2017/07/selecting-compression-algorithm-for-rr.html

lks9 commented 1 year ago

I see two implementation approaches:

  1. Use a compression method when writing the trace. This probably needs to a separate thread.
  2. Don't change anything and just use a compressed filesystem. BTRFS seems to be a good option [1].

I think I will go with 2 because it is trivial to implement and has probably less overhead.

[1] https://btrfs.readthedocs.io/en/latest/Compression.html

lks9 commented 1 year ago

Btw. here is a list of compressed filesystems:

https://en.wikipedia.org/wiki/Category:Compression_file_systems

I am not really a fan of btrfs. Perhaps it would be better to use F2FS or ZFS or, when supported by linux, Bcachefs.

lks9 commented 11 months ago

Ok, I think this one is finished, it is possible with a compressed file system. I will open a new issue for direct compression.