What would make good benchmarks for the Haskell binding?

Hi there, I am still in the process of writing the Haskell binding, it seems to be working now and I have bound every function in include/ffts.h. At this point in time it all seems to work but that is only half the battle; I want to know how fast it is as well so it is time for Benchmarking.

I have started writing the benchmarking and you can view the code that I have written or you can just review the interesting results I have so far:

Running 1 benchmarks...
Benchmark bench-ffts: RUNNING...
warming up
estimating clock resolution...
mean is 1.441820 us (640001 iterations)
found 3913 outliers among 639999 samples (0.6%)
  3507 (0.5%) high severe
estimating cost of a clock call...
mean is 38.08749 ns (12 iterations)

benchmarking 1d Real 16 Values
mean: 9.497489 us, lb 9.452236 us, ub 9.553633 us, ci 0.950
std dev: 256.6657 ns, lb 215.7990 ns, ub 298.1516 ns, ci 0.950
found 17 outliers among 100 samples (17.0%)
  11 (11.0%) high mild
  6 (6.0%) high severe
variance introduced by outliers: 20.965%
variance is moderately inflated by outliers

benchmarking 1d Real 512 Values
mean: 267.5601 us, lb 264.9554 us, ub 274.5509 us, ci 0.950
std dev: 19.89528 us, lb 4.578779 us, ub 40.04049 us, ci 0.950
found 4 outliers among 100 samples (4.0%)
  2 (2.0%) high mild
  2 (2.0%) high severe
variance introduced by outliers: 67.655%
variance is severely inflated by outliers

benchmarking 1d Real 1024 Values
mean: 546.5007 us, lb 540.0555 us, ub 562.0257 us, ci 0.950
std dev: 47.99116 us, lb 21.33818 us, ub 87.39147 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
  3 (3.0%) high mild
  3 (3.0%) high severe
variance introduced by outliers: 74.830%
variance is severely inflated by outliers

benchmarking 1d Real 2048 Values
mean: 1.215410 ms, lb 1.196909 ms, ub 1.241009 ms, ci 0.950
std dev: 110.3181 us, lb 84.76925 us, ub 143.6377 us, ci 0.950
found 8 outliers among 100 samples (8.0%)
  3 (3.0%) high mild
  5 (5.0%) high severe
variance introduced by outliers: 75.876%
variance is severely inflated by outliers

benchmarking 1d Complex 16 Values
mean: 18.60266 us, lb 18.52926 us, ub 18.69820 us, ci 0.950
std dev: 428.8377 ns, lb 348.4019 ns, ub 515.2131 ns, ci 0.950
found 14 outliers among 100 samples (14.0%)
  10 (10.0%) high mild
  4 (4.0%) high severe
variance introduced by outliers: 16.150%
variance is moderately inflated by outliers

benchmarking 1d Complex 512 Values
mean: 611.0597 us, lb 607.6002 us, ub 614.6598 us, ci 0.950
std dev: 18.08215 us, lb 15.99014 us, ub 21.20516 us, ci 0.950
found 1 outliers among 100 samples (1.0%)
variance introduced by outliers: 24.794%
variance is moderately inflated by outliers

benchmarking 1d Complex 1024 Values
mean: 1.363070 ms, lb 1.344092 ms, ub 1.387594 ms, ci 0.950
std dev: 110.0677 us, lb 89.10057 us, ub 130.3623 us, ci 0.950
found 14 outliers among 100 samples (14.0%)
  2 (2.0%) high mild
  12 (12.0%) high severe
variance introduced by outliers: 70.758%
variance is severely inflated by outliers

benchmarking 1d Complex 2048 Values
mean: 3.202281 ms, lb 3.193803 ms, ub 3.213675 ms, ci 0.950
std dev: 50.09133 us, lb 37.46054 us, ub 72.28393 us, ci 0.950
found 4 outliers among 100 samples (4.0%)
  2 (2.0%) high severe
variance introduced by outliers: 8.483%
variance is slightly inflated by outliers

benchmarking 1d Reverse Real 16 Values
mean: 8.949454 us, lb 8.928603 us, ub 8.975940 us, ci 0.950
std dev: 120.6416 ns, lb 99.41294 ns, ub 155.4416 ns, ci 0.950
found 4 outliers among 100 samples (4.0%)
  3 (3.0%) high mild
  1 (1.0%) high severe
variance introduced by outliers: 6.579%
variance is slightly inflated by outliers

benchmarking 1d Reverse Real 512 Values
mean: 248.8768 us, lb 247.9895 us, ub 250.3459 us, ci 0.950
std dev: 5.715797 us, lb 3.902308 us, ub 9.094875 us, ci 0.950
found 7 outliers among 100 samples (7.0%)
  4 (4.0%) high mild
  3 (3.0%) high severe
variance introduced by outliers: 16.144%
variance is moderately inflated by outliers

benchmarking 1d Reverse Real 1024 Values
mean: 507.0067 us, lb 505.2487 us, ub 508.9387 us, ci 0.950
std dev: 9.465558 us, lb 8.227033 us, ub 11.19390 us, ci 0.950
found 2 outliers among 100 samples (2.0%)
  2 (2.0%) high mild
variance introduced by outliers: 11.354%
variance is moderately inflated by outliers

benchmarking 1d Reverse Real 2048 Values
mean: 1.142064 ms, lb 1.128539 ms, ub 1.159102 ms, ci 0.950
std dev: 77.62682 us, lb 65.57185 us, ub 92.82481 us, ci 0.950
found 22 outliers among 100 samples (22.0%)
  11 (11.0%) high mild
  11 (11.0%) high severe
variance introduced by outliers: 63.571%
variance is severely inflated by outliers
Benchmark bench-ffts: FINISH

That seems pretty good but I don't know how to compare it with the benchmarks that you already seem to have written. So my questions are:

How can I compare my benchmark results with the benchmarks that you have already written?
How does my first version of the Haskell binding compare to the raw C version?
Are there any benchmarks that you can suggest that I try and write so that I can really flex the library?

Thankyou for your time.

anthonix / ffts

What would make good benchmarks for the Haskell binding? #18