Can process run in parallel?

[ylxxwx]

A question. Can the process run in parallel? That means add more processes, we can create bigger qps for the performance test. If the processes can't run in parallel, what is the purpose we support -p [num]?

[vstinner]

When I designed pyperf, I didn't want to run tests in more than 1 process for different reasons:

• A single CPU may benefit from Turbo Boost (even if pyperf suggests to disable it)
• If you only isolate a single physical CPU core, multiple processes will race to grab CPU time
• Intel CPU now have complex rules to select the CPU frequency: C-state, P-state, Turbo Boost, etc.

If you are confident that multiple processes will have similar or also the same performance, you can run a single process with computes tons of values in the same process: -p 1 -n 50, for example. Linux ASLR makes performance harder to measure.

Code placement became more critical for L1-instruction cache performance. See the BOLT compiler to see how it matters on nowadays Intel CPUs.

Well, said differently: you have to make a compromise between your "time budget" (how long all benchmarks will take on the wall clock) and the accuracy of results (get "reproducible" and "stable" benchmarks results). pyperf default configuration is designed for best accuracy. There is the --fast option which is a small trade-off towards shorter time budget: https://pyperf.readthedocs.io/en/latest/runner.html#loop-iterations

I also suggest you to have a look at my old articles: https://vstinner.github.io/category/benchmark.html

These days, I am away from pyperf and I let others like @corona10 considers changing pyperf ;-)

[corona10]

Well, I am on the same side with @vstinner We can provide such an option as optional until we know the use case of such an option. I am conservative in providing such options. Our biggest user is pyperformance, but I am sure that they want accuracy more than fast execution.

[ylxxwx]

Thanks for the reply. I am thinking if we can make pyperf to run load test as well with ability to generate request with certain qps. Another thought about that, or another project for that?

[corona10]

if we can make pyperf to run load test as well with ability to generate request with certain qps.

If you are considering load tests for specific tests, why are you considering using pyperf instead of load testing tools? I think that pyperf measures the performance of python code rather than measuring stress-test for external services.