Cabal bench

[mboes]

@qnikst what's the difference between *Wrapper and *Run? Their rationale should be explained somewhere.

[mboes]

Also, why do you say forkOS might introduce additional overhead, so long as the benchmark time is taken to start after the forking?

[qnikst]

there is the following difference:

• Wrapper - creates a one executable that will for 2 workers with default options, and run a benchmark. It's used in cabal-bench
• Run - creates an executable that you can run with different options on a different machines and thus it impossible to use it in cabal-bench.

I can add description but what will be an appropriate place?

[qnikst]

forkOS uses one RTS so global GC will block both threads, while in case if we will use real fork or runProcess we will have 2 separate runtimes. I not sure that it will have a big impact on a result, but I have to address it at least in commit message :)

[mboes]

I don't see a need for 3 separate files for each benchmark. What with the difference between 2 of those files not being immediately obvious to the reader, and not readily documentable, the following scheme would be better I think:

• one file per benchmark.
• fold the main function that is found in each *Run.hs file into the core benchmark file.
• Make the main function do something sane by default, i.e. when no command line arguments are provided. When calling cabal bench, that is what will be used.
• Make the main function allow command line arguments. These are useful for manual testing on remote machines etc.

[qnikst]

Updated.

[mboes]

Not that I can see. Pushed to right branch?

[qnikst]

No, pushed to qnikst repo. Fixed.

[mboes]

Pushed a fix to the flag name. I noticed in doing so that most benchmarks are actually not disabled from being installed when the flag is off.

That needs to be fixed before this is merged.