pytest-bench: determine number of rounds

[awf]

We need pseudocode like this at https://github.com/microsoft/knossos-ksc/blob/edb05342ad5f664fc2b406c456c345e6b327fab6/src/bench/test_run_bench.py#L34

for N in 2^range(0,10):
  result = benchmark.pedantic(torch.autograd.grad, setup=create_fresh_args, rounds=N)
  if that took more than 1 sec:  # ideally without wrapping with our own timer?
     break
 else discard result (ensure it didn't go into json)

By "wrapping with our own timer" I mean this code, which is maybe perfectly fine:

for N in 2^range(0,10):
  start = timer.
  result = benchmark.pedantic(torch.autograd.grad, setup=create_fresh_args, rounds=N)
  if timer-start > 1:
     break
 else discard result (ensure it didn't go into json)

[cgravill]

Just a pytest-benchmark note: I think multiple calls to benchmark.* are not permitted.

We can do the same time probing separately.

I had a discussion with @dcrc2 separately and we think a solution to this might also solve our issue of not wanting to run the nice/slow PyTorch style against larger examples but we need to for the fast/GPU/Knossos optimised ones.