search

airspeed-velocity / asv

Airspeed Velocity: A simple Python benchmarking tool with web-based reporting
https://asv.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
870 stars 180 forks source link

ENH: asv continuous. Disable result table after round 2 per commit. #848

Open BeforeFlight opened 5 years ago

BeforeFlight commented 5 years ago

I'm using class with params. And total params combinations count is pretty big. But while using asv continuous to compare performance of 2 commits after "[ 75.00%] · For ... commit ... (round 2/2):" there is shown a table with results of this step like so:

====== ================== =================== ============= ============ =============
prm_1         prm_2                prm_3           prm_4         prm_5         
 ------ -------------  ----------------------- ------------- ------------ -------------
   1            1                   1               bool        (0, 0)      30.0±0.9μs 
   1            1                   1               bool        (0, 1)      30.1±0.8μs 
   1            1                   1               bool        (0, 2)      30.6±0.3μs 
   1            1                   1               bool        (0, 3)      29.8±0.4μs

And same after second commit round 2 tests. Due to huge params combination count this table just cluttering stdout and takes much time to be built.

Besides one will use the result comparing table (with changes only) foremost and such tables may be redundant at all. So proposal is to add ability to disable these (haven't find corresponding flag in docs).

pv commented 5 years ago

Note that it's not building the table that takes time: all benchmarks are run also on the second round.