pytest_benchmark_group_stats not working as expected

[swethmandava]

@pytest.mark.hookwrapper
def pytest_benchmark_group_stats(config, benchmarks, group_by):
    result = defaultdict(list)
    for bench in benchmarks:
        result[bench.group].append(bench)
    return result.items()

@pytest.mark.benchmark(group='search')
@pytest.mark.parametrize("query", test_queries)
def test_benchmark_search(benchmark, query):
    benchmark(some_func, query)

when called with pytest test_benchmark.py --benchmark-json=benchmark.json returns a list of timings for each parameterized version. I'd want it to group all parameterized calls into a single output called "search". Is that possible?

[ionelmc]

You'd like to average all the benchmark from the 'search' group together?

It could be possible if you do the averaging yourself and mess with the stats a bit...

I would like to understand a bit what's the purpose here.

Ref #166.

[swethmandava]

I can think of some_func as a function which takes time T dependent on the parameter. I'd want to benchmark this function to get stats for a variety of parameters that are relevant to the user. So if i have 10 relevant parameters - I want to collect stats (mean/median) for these 10 parameters with the benchmark function. instead of mean/mean for each parameter.