search

psf / pyperf

Toolkit to run Python benchmarks
http://pyperf.readthedocs.io/
MIT License
799 stars 78 forks source link

Implement 'faster on x% of runs' #118

Open sweeneyde opened 3 years ago

sweeneyde commented 3 years ago

Based on https://github.com/psf/pyperf/issues/22:

Example:

py -m pyperf compare_to ..\cpython\floatbench.json ..\multiply\floatbench.json --table --win_percentage
+----------------+----------------------------+-----------------------------------------------+
| Benchmark      | ..\cpython\floatbench.json | ..\multiply\floatbench.json                   |
+================+============================+===============================================+
| int: x*x       | 28.5 ns                    | 27.7 ns: 1.03x faster (faster on 88% of runs) |
+----------------+----------------------------+-----------------------------------------------+
| float: x*x     | 28.4 ns                    | 27.9 ns: 1.02x faster (faster on 80% of runs) |
+----------------+----------------------------+-----------------------------------------------+
| int: x*...*x   | 262 ms                     | 220 ms: 1.19x faster (faster on 100% of runs) |
+----------------+----------------------------+-----------------------------------------------+
| float: x*...*x | 260 ms                     | 253 ms: 1.03x faster (faster on 90% of runs)  |
+----------------+----------------------------+-----------------------------------------------+
| Geometric mean | (ref)                      | 1.06x faster                                  |
+----------------+----------------------------+-----------------------------------------------+

I've not worked with this code before, so let me know what needs fixing.