Benchmarking

[Sparrow0hawk]

An issue to keep track of benchmarking of sagitta

See earlier stuff - https://github.com/Sparrow0hawk/sagitta/issues/4

[Sparrow0hawk]

For v0.3.0

At the top of a file:

hyperfine 'sagitta -f -j 1001 accounting' 'sagitta -j 1001 accounting' 'qacct -j 1001'
Benchmark 1: sagitta -f -j 1001 accounting
  Time (mean ± σ):      69.3 ms ± 189.3 ms    [User: 2.8 ms, System: 64.7 ms]
  Range (min … max):     7.4 ms … 607.9 ms    10 runs

  Warning: The first benchmarking run for this command was significantly slower than the rest (607.9 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.

Benchmark 2: sagitta -j 1001 accounting
  Time (mean ± σ):     65.626 s ±  7.251 s    [User: 18.086 s, System: 7.028 s]
  Range (min … max):   45.951 s … 71.002 s    10 runs

Benchmark 3: qacct -j 1001
  Time (mean ± σ):      6.400 s ±  0.874 s    [User: 4.444 s, System: 1.936 s]
  Range (min … max):    5.655 s …  7.667 s    10 runs

Summary
  'sagitta -f -j 1001 accounting' ran
   92.42 ± 252.88 times faster than 'qacct -j 1001'
  947.63 ± 2591.92 times faster than 'sagitta -j 1001 accounting'

At the bottom of a file:

hyperfine 'sagitta -f -j 4808634 accounting' 'sagitta -j 4808634 accounting' 'qacct -j 4808634'
Benchmark 1: sagitta -f -j 4808634 accounting
  Time (mean ± σ):      5.617 s ±  0.826 s    [User: 3.866 s, System: 1.730 s]
  Range (min … max):    4.848 s …  7.042 s    10 runs

Benchmark 2: sagitta -j 4808634 accounting
  Time (mean ± σ):       3.7 ms ±   2.5 ms    [User: 0.9 ms, System: 2.4 ms]
  Range (min … max):     0.1 ms …   8.9 ms    305 runs

  Warning: Command took less than 5 ms to complete. Note that the results might be inaccurate because hyperfine can not calibrate the shell startup time much more precise than this limit. You can try to use the `-N`/`--shell=none` option to disable the shell completely.

Benchmark 3: qacct -j 4808634
  Time (mean ± σ):      5.693 s ±  0.706 s    [User: 4.325 s, System: 1.344 s]
  Range (min … max):    5.263 s …  7.069 s    10 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Summary
  'sagitta -j 4808634 accounting' ran
 1505.98 ± 1033.69 times faster than 'sagitta -f -j 4808634 accounting'
 1526.36 ± 1040.74 times faster than 'qacct -j 4808634'

[Sparrow0hawk]

Flamegraph for sagitta -j 4541659 path/to/accounting

Flamegraph for sagitta -f -j 4541659 path/to/accounting