The custom stopping criterion introduced in #151 stores all parameters for all criterion implementations in a single map. This requires that all parameters must be uniquely named across implementations along with degraded error handling capabilities while parsing CLI arguments.
Each instance of stopping_criterion should hold its own params map, and the CLI parser should check that any arguments relating to these criterion are valid for the selected implementation.
The existing stopping criterion parameters are all documented as follows: Applies to the most recent --benchmark, or all benchmarks if specified before any --benchmark arguments. The new stopping criterion framework should support this pattern as well to allow individual benchmarks to be run with different tolerances.
The custom stopping criterion introduced in #151 stores all parameters for all criterion implementations in a single map. This requires that all parameters must be uniquely named across implementations along with degraded error handling capabilities while parsing CLI arguments.
Each instance of stopping_criterion should hold its own params map, and the CLI parser should check that any arguments relating to these criterion are valid for the selected implementation.
The existing stopping criterion parameters are all documented as follows:
Applies to the most recent --benchmark, or all benchmarks if specified before any --benchmark arguments.The new stopping criterion framework should support this pattern as well to allow individual benchmarks to be run with different tolerances.