Let's say i want to compare three data structures that, in their way, probe a vector of size M (as you can see the probing logic is performed by using the index trait) by accepting a parameter that goes from 0 up to M-1. I would like to test my structures' behavior with every possibile input (let's call it 'slot') i.e. every possibile value from 0 up to M-1. Thus, i have something like that:
let slots_to_probe = VEC_WITH_EVERY_POSSIBILE_SLOT
let mut group = c.benchmark_group("probing");
for slot_to_probe in slots_to_probe.iter() {
group.bench_with_input([...] b.iter(|| first_struct[black_box(slot_to_probe)]));
group.bench_with_input([...] b.iter(|| second_struct[black_box(slot_to_probe)]));
group.bench_with_input([...] b.iter(|| third_struct[black_box(slot_to_probe)]));
}
group.finish();
My goal is to get the report/lines.svg chart showing the comparison. The problem arises when the vector is quite big (like 2^20 elements) since criterion creates a directory in target/ for each slot plus another directory for every function where it stores another dir for every slot. This implies consuming a lot of memory very soon.
I tried to use --noplot when running the bench but it avoids creating even the chart that i need. Even looking into the criterion's documentation, i have not been able to find a suitable solution for my case.
Ps: when trying to reduce the number of slots to bench with, i've been able to get correctly the chart only when going down up to something like 1000 slots since, with bigger sets of values, i was getting called Result::unwrap() on an Err value: Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" }.
Let's say i want to compare three data structures that, in their way, probe a vector of size M (as you can see the probing logic is performed by using the index trait) by accepting a parameter that goes from 0 up to M-1. I would like to test my structures' behavior with every possibile input (let's call it 'slot') i.e. every possibile value from 0 up to M-1. Thus, i have something like that:
My goal is to get the report/lines.svg chart showing the comparison. The problem arises when the vector is quite big (like 2^20 elements) since criterion creates a directory in target/ for each slot plus another directory for every function where it stores another dir for every slot. This implies consuming a lot of memory very soon.
I tried to use --noplot when running the bench but it avoids creating even the chart that i need. Even looking into the criterion's documentation, i have not been able to find a suitable solution for my case.
Ps: when trying to reduce the number of slots to bench with, i've been able to get correctly the chart only when going down up to something like 1000 slots since, with bigger sets of values, i was getting
called Result::unwrap() on an Err value: Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" }.