Accept an argument as input to prevent constant folding everything away
Print the sum of all outputs, rather than each individual output
Running times normalized to 250ms on my M2 Mac for "llvm-peep", should graphs a bit easier to look at and nightlies should run a little faster
Exceptions:
bril/core/catalan.bril, bril/core/up-arrow.bril, bril/long/function_call.bril: Irregular running time because the algorithm runs in exponential time or more, so changing the input by 1 can have big effects on the running time. I've selected the smallest inputs that have running time at least that of everything else.
bril/mem/bubblesort.bril: Because a sorting an array doesn't change its sum, this benchmark was changed to output the the sum of array elements times their index.
bril/core/sum-check.bril, bril/core/sum-sq-diff.bril: LLVM optimizes the inner loop from linear to constant run time, so there is no input that will result in reasonable run times across both LLVM on and LLVM off benchmark tests. The default input is set to a value reasonable for LLVM off so that nightlies won't get hung up on these. If comparing with LLVM on, set the input to something between 1e8 and 1e9. Neither has a significant speedup.
Also:
New bench_one.py script to run one benchmark locally (4 modes with llvm and eggcc on/off)
To-do in next PR(s):
Update polybench benchmarks
It is now revealed that 8 benchmarks have eggcc faster with non-overlapping error bars. (Most other benchmarks are flat, a few are slower.)
All passing bril benchmarks have been updated to:
Exceptions:
bril/core/catalan.bril,bril/core/up-arrow.bril,bril/long/function_call.bril: Irregular running time because the algorithm runs in exponential time or more, so changing the input by 1 can have big effects on the running time. I've selected the smallest inputs that have running time at least that of everything else.bril/mem/bubblesort.bril: Because a sorting an array doesn't change its sum, this benchmark was changed to output the the sum of array elements times their index.bril/core/sum-check.bril,bril/core/sum-sq-diff.bril: LLVM optimizes the inner loop from linear to constant run time, so there is no input that will result in reasonable run times across both LLVM on and LLVM off benchmark tests. The default input is set to a value reasonable for LLVM off so that nightlies won't get hung up on these. If comparing with LLVM on, set the input to something between 1e8 and 1e9. Neither has a significant speedup.Also:
To-do in next PR(s):
It is now revealed that 8 benchmarks have eggcc faster with non-overlapping error bars. (Most other benchmarks are flat, a few are slower.)