iu-parfunc / HSBencher

General benchmarking framework. Especially good at parameter studies.
13 stars 8 forks source link

Allow tags to override ALL builtin fields #89

Closed rrnewton closed 9 years ago

rrnewton commented 9 years ago

This is related to #75 and #73.

The new multi-benchmark-per-process framework introduced in 0abf2424e86518ed63bd1850918e0332081400a5 will form the basis for both a passive upload mode (#75), and better integration of Criterion benchmark suites (which typically also run multiple benchmarks per criterion run).

But for this START_BENCHMARK/END_BENCHMARK protocol to be useful, the lines of output in between those delimiters had better be able to set/override all relevant fields of BenchmarkResult. For example PROGNAME:, VARIANT: and so on.

This will probably take the form of a whole pile of LineHarvesters that correspond to the builtin fields. It would be nice to derive that but not too painful to just write it out either.