mdboom / pyperf-codspeed

Pyperf plugin to create CodSpeed benchmarks
https://codspeed.io
MIT License
0 stars 0 forks source link

Running long enough for specialization / JIT to matter #9

Open mdboom opened 3 months ago

mdboom commented 3 months ago

In order to get the total runtime down below Github's limit of 6 hours, I used --debug-single-run which runs each benchmark exactly once. This is likely not enough time for specialization or the JIT to warmup in most cases. This will need to be adjusted -- maybe not all the way back up to current levels, however.

I think the prerequisite here is to probably add more parallelism, see #7.