search

iterative / dvc-bench

Benchmarks for DVC
http://bench.dvc.org/
Apache License 2.0
20 stars 10 forks source link

plots: pull everything before benchmarking #427

Closed skshetry closed 1 year ago

efiop commented 1 year ago

@skshetry Looks like the test broke though

github-actions[bot] commented 1 year ago 

----------------------------------------------------------------------------- benchmark 'test_plots-plots-diff': 5 tests -----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-diff[2.39.0] 906.0368 (1.25) 931.6205 (1.25) 919.8757 (1.26) 11.5522 (6.16) 919.9461 (1.26) 21.4822 (7.74) 2;0 1.0871 (0.79) 5 3
test_plots-plots-diff[2.40.0] 868.8997 (1.20) 883.1225 (1.19) 876.2923 (1.20) 5.3302 (2.84) 875.4840 (1.20) 7.0330 (2.53) 2;0 1.1412 (0.83) 5 3
test_plots-plots-diff[2.41.1] 864.8074 (1.19) 870.0821 (1.17) 867.6506 (1.19) 2.3448 (1.25) 868.6678 (1.19) 4.0800 (1.47) 2;0 1.1525 (0.84) 5 3
test_plots-plots-diff[2.45.0] 831.0323 (1.15) 835.9050 (1.12) 833.3708 (1.14) 1.8766 (1.0) 833.4368 (1.14) 2.7759 (1.0) 2;0 1.1999 (0.88) 5 3
test_plots-plots-diff[main] 725.2115 (1.0) 743.9835 (1.0) 731.2770 (1.0) 7.8552 (4.19) 728.1575 (1.0) 10.7528 (3.87) 1;0 1.3675 (1.0) 5 3
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_plots-plots-diff-json': 5 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-diff-json[2.39.0] 1.3779 (1.52) 1.4138 (1.53) 1.3984 (1.53) 0.0146 (6.34) 1.3991 (1.53) 0.0235 (6.36) 2;0 0.7151 (0.65) 5 3
test_plots-plots-diff-json[2.40.0] 1.3667 (1.51) 1.3858 (1.50) 1.3757 (1.50) 0.0073 (3.15) 1.3742 (1.50) 0.0101 (2.73) 2;0 0.7269 (0.66) 5 3
test_plots-plots-diff-json[2.41.1] 1.3420 (1.48) 1.3522 (1.47) 1.3484 (1.47) 0.0041 (1.77) 1.3502 (1.47) 0.0055 (1.49) 1;0 0.7416 (0.68) 5 3
test_plots-plots-diff-json[2.45.0] 1.2592 (1.39) 1.2646 (1.37) 1.2625 (1.38) 0.0023 (1.0) 1.2636 (1.38) 0.0037 (1.0) 1;0 0.7921 (0.72) 5 3
test_plots-plots-diff-json[main] 0.9076 (1.0) 0.9220 (1.0) 0.9145 (1.0) 0.0063 (2.71) 0.9170 (1.0) 0.0105 (2.84) 2;0 1.0935 (1.0) 5 3
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'test_plots-plots-show': 5 tests -----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-show[2.39.0] 795.8715 (1.19) 820.0858 (1.22) 811.8922 (1.21) 10.3458 (6.61) 816.3647 (1.22) 15.4640 (6.67) 1;0 1.2317 (0.83) 5 3
test_plots-plots-show[2.40.0] 737.1245 (1.10) 748.4113 (1.11) 741.8256 (1.11) 4.7718 (3.05) 741.9975 (1.11) 7.9594 (3.43) 1;0 1.3480 (0.90) 5 3
test_plots-plots-show[2.41.1] 738.1481 (1.10) 743.0671 (1.10) 741.0489 (1.10) 2.4459 (1.56) 742.4454 (1.11) 4.4749 (1.93) 1;0 1.3494 (0.91) 5 3
test_plots-plots-show[2.45.0] 701.7950 (1.05) 706.1715 (1.05) 703.9909 (1.05) 1.6975 (1.08) 703.9531 (1.05) 2.5737 (1.11) 2;0 1.4205 (0.95) 5 3
test_plots-plots-show[main] 669.4256 (1.0) 673.3731 (1.0) 671.0944 (1.0) 1.5657 (1.0) 671.1815 (1.0) 2.3191 (1.0) 2;0 1.4901 (1.0) 5 3
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_plots-plots-show-json': 5 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-show-json[2.39.0] 1.2651 (1.48) 1.3391 (1.54) 1.2979 (1.50) 0.0281 (12.11) 1.2930 (1.50) 0.0386 (14.34) 2;0 0.7705 (0.66) 5 3
test_plots-plots-show-json[2.40.0] 1.2373 (1.45) 1.2469 (1.43) 1.2418 (1.44) 0.0039 (1.68) 1.2416 (1.44) 0.0065 (2.41) 2;0 0.8053 (0.69) 5 3
test_plots-plots-show-json[2.41.1] 1.2398 (1.45) 1.2453 (1.43) 1.2421 (1.44) 0.0023 (1.0) 1.2409 (1.44) 0.0037 (1.38) 1;0 0.8051 (0.69) 5 3
test_plots-plots-show-json[2.45.0] 1.1542 (1.35) 1.1622 (1.34) 1.1579 (1.34) 0.0029 (1.23) 1.1579 (1.34) 0.0027 (1.0) 2;0 0.8636 (0.74) 5 3
test_plots-plots-show-json[main] 0.8558 (1.0) 0.8700 (1.0) 0.8625 (1.0) 0.0053 (2.30) 0.8621 (1.0) 0.0074 (2.75) 2;0 1.1594 (1.0) 5 3
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```