mdboom / pyperf-codspeed

Pyperf plugin to create CodSpeed benchmarks
https://codspeed.io
MIT License
0 stars 0 forks source link

Creation of the benchmarking venvs is slow #5

Open mdboom opened 1 month ago

mdboom commented 1 month ago

pyperformance run first creates venvs for each of the benchmarks first in one pass, and then actually runs the benchmarks.

By running the venv creation step inside of valgrind, it's pretty slow, even though none of that code needs to be instrumented.

We should probably modify pyperformance so the venv creation step can be done independently before doing pyperformance run.

mdboom commented 1 month ago

An additional complexity is that the creation of venvs isn't thread safe. pyperformance tries to carefully create as few venvs as possible (it only creates a new one when there is a dependency conflict), which means it needs to create one at a time. We could remove those semantics at the expense of more disk usage and I/O.