search

iterative / dvc-bench

Benchmarks for DVC
http://bench.dvc.org/
Apache License 2.0
20 stars 10 forks source link

add benchmark for data status #370

Closed skshetry closed 2 years ago

github-actions[bot] commented 2 years ago 

Fri Jul 22 12:56:13 UTC 2022
dataset: small
project: example-get-started

--------------------------------------------------------------------------------- benchmark 'test_add-add': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.10.2] (0001_test_ad) 11.4189 (1.15) 11.4189 (1.15) 11.4189 (1.15) 0.0000 (1.0) 11.4189 (1.15) 0.0000 (1.0) 0;0 0.0876 (0.87) 1 1
test_add-add[2.11.0] (0001_test_ad) 10.7654 (1.08) 10.7654 (1.08) 10.7654 (1.08) 0.0000 (1.0) 10.7654 (1.08) 0.0000 (1.0) 0;0 0.0929 (0.92) 1 1
test_add-add[2.13.0] (0001_test_ad) 10.2087 (1.03) 10.2087 (1.03) 10.2087 (1.03) 0.0000 (1.0) 10.2087 (1.03) 0.0000 (1.0) 0;0 0.0980 (0.97) 1 1
test_add-add[2.6.3] (0001_test_ad) 13.0020 (1.31) 13.0020 (1.31) 13.0020 (1.31) 0.0000 (1.0) 13.0020 (1.31) 0.0000 (1.0) 0;0 0.0769 (0.76) 1 1
test_add-add[main] (0001_test_ad) 9.9438 (1.0) 9.9438 (1.0) 9.9438 (1.0) 0.0000 (1.0) 9.9438 (1.0) 0.0000 (1.0) 0;0 0.1006 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout[2.10.2] (0001_test_ch) 3.4230 (1.31) 3.4230 (1.31) 3.4230 (1.31) 0.0000 (1.0) 3.4230 (1.31) 0.0000 (1.0) 0;0 0.2921 (0.77) 1 1
test_checkout-checkout[2.11.0] (0001_test_ch) 3.3438 (1.28) 3.3438 (1.28) 3.3438 (1.28) 0.0000 (1.0) 3.3438 (1.28) 0.0000 (1.0) 0;0 0.2991 (0.78) 1 1
test_checkout-checkout[2.13.0] (0001_test_ch) 3.7526 (1.43) 3.7526 (1.43) 3.7526 (1.43) 0.0000 (1.0) 3.7526 (1.43) 0.0000 (1.0) 0;0 0.2665 (0.70) 1 1
test_checkout-checkout[2.6.3] (0001_test_ch) 2.6225 (1.0) 2.6225 (1.0) 2.6225 (1.0) 0.0000 (1.0) 2.6225 (1.0) 0.0000 (1.0) 0;0 0.3813 (1.0) 1 1
test_checkout-checkout[main] (0001_test_ch) 3.3037 (1.26) 3.3037 (1.26) 3.3037 (1.26) 0.0000 (1.0) 3.3037 (1.26) 0.0000 (1.0) 0;0 0.3027 (0.79) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_checkout-checkout-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop[2.10.2] (0001_test_ch) 507.1239 (1.25) 507.1239 (1.25) 507.1239 (1.25) 0.0000 (1.0) 507.1239 (1.25) 0.0000 (1.0) 0;0 1.9719 (0.80) 1 1
test_checkout-checkout-noop[2.11.0] (0001_test_ch) 502.1078 (1.24) 502.1078 (1.24) 502.1078 (1.24) 0.0000 (1.0) 502.1078 (1.24) 0.0000 (1.0) 0;0 1.9916 (0.81) 1 1
test_checkout-checkout-noop[2.13.0] (0001_test_ch) 582.0467 (1.43) 582.0467 (1.43) 582.0467 (1.43) 0.0000 (1.0) 582.0467 (1.43) 0.0000 (1.0) 0;0 1.7181 (0.70) 1 1
test_checkout-checkout-noop[2.6.3] (0001_test_ch) 406.4233 (1.0) 406.4233 (1.0) 406.4233 (1.0) 0.0000 (1.0) 406.4233 (1.0) 0.0000 (1.0) 0;0 2.4605 (1.0) 1 1
test_checkout-checkout-noop[main] (0001_test_ch) 581.8315 (1.43) 581.8315 (1.43) 581.8315 (1.43) 0.0000 (1.0) 581.8315 (1.43) 0.0000 (1.0) 0;0 1.7187 (0.70) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-update': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update[2.10.2] (0001_test_ch) 1.3753 (2.13) 1.3753 (2.13) 1.3753 (2.13) 0.0000 (1.0) 1.3753 (2.13) 0.0000 (1.0) 0;0 0.7271 (0.47) 1 1
test_checkout-checkout-update[2.11.0] (0001_test_ch) 1.3656 (2.12) 1.3656 (2.12) 1.3656 (2.12) 0.0000 (1.0) 1.3656 (2.12) 0.0000 (1.0) 0;0 0.7323 (0.47) 1 1
test_checkout-checkout-update[2.13.0] (0001_test_ch) 0.6446 (1.0) 0.6446 (1.0) 0.6446 (1.0) 0.0000 (1.0) 0.6446 (1.0) 0.0000 (1.0) 0;0 1.5514 (1.0) 1 1
test_checkout-checkout-update[2.6.3] (0001_test_ch) 3.1951 (4.96) 3.1951 (4.96) 3.1951 (4.96) 0.0000 (1.0) 3.1951 (4.96) 0.0000 (1.0) 0;0 0.3130 (0.20) 1 1
test_checkout-checkout-update[main] (0001_test_ch) 0.6568 (1.02) 0.6568 (1.02) 0.6568 (1.02) 0.0000 (1.0) 0.6568 (1.02) 0.0000 (1.0) 0;0 1.5226 (0.98) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_diff-diff': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.10.2] (0001_test_di) 2.5342 (1.62) 2.5342 (1.62) 2.5342 (1.62) 0.0000 (1.0) 2.5342 (1.62) 0.0000 (1.0) 0;0 0.3946 (0.62) 1 1
test_diff-diff[2.11.0] (0001_test_di) 1.9346 (1.23) 1.9346 (1.23) 1.9346 (1.23) 0.0000 (1.0) 1.9346 (1.23) 0.0000 (1.0) 0;0 0.5169 (0.81) 1 1
test_diff-diff[2.13.0] (0001_test_di) 2.6589 (1.70) 2.6589 (1.70) 2.6589 (1.70) 0.0000 (1.0) 2.6589 (1.70) 0.0000 (1.0) 0;0 0.3761 (0.59) 1 1
test_diff-diff[2.6.3] (0001_test_di) 4.8408 (3.09) 4.8408 (3.09) 4.8408 (3.09) 0.0000 (1.0) 4.8408 (3.09) 0.0000 (1.0) 0;0 0.2066 (0.32) 1 1
test_diff-diff[main] (0001_test_di) 1.5669 (1.0) 1.5669 (1.0) 1.5669 (1.0) 0.0000 (1.0) 1.5669 (1.0) 0.0000 (1.0) 0;0 0.6382 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.10.2] (0001_test_di) 1.2043 (2.10) 1.2043 (2.10) 1.2043 (2.10) 0.0000 (1.0) 1.2043 (2.10) 0.0000 (1.0) 0;0 0.8304 (0.48) 1 1
test_diff-diff-changed[2.11.0] (0001_test_di) 1.1890 (2.07) 1.1890 (2.07) 1.1890 (2.07) 0.0000 (1.0) 1.1890 (2.07) 0.0000 (1.0) 0;0 0.8411 (0.48) 1 1
test_diff-diff-changed[2.13.0] (0001_test_di) 0.5735 (1.0) 0.5735 (1.0) 0.5735 (1.0) 0.0000 (1.0) 0.5735 (1.0) 0.0000 (1.0) 0;0 1.7436 (1.0) 1 1
test_diff-diff-changed[2.6.3] (0001_test_di) 2.4822 (4.33) 2.4822 (4.33) 2.4822 (4.33) 0.0000 (1.0) 2.4822 (4.33) 0.0000 (1.0) 0;0 0.4029 (0.23) 1 1
test_diff-diff-changed[main] (0001_test_di) 0.6998 (1.22) 0.6998 (1.22) 0.6998 (1.22) 0.0000 (1.0) 0.6998 (1.22) 0.0000 (1.0) 0;0 1.4289 (0.82) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.10.2] (0001_test_di) 1.2090 (2.37) 1.2090 (2.37) 1.2090 (2.37) 0.0000 (1.0) 1.2090 (2.37) 0.0000 (1.0) 0;0 0.8271 (0.42) 1 1
test_diff-diff-changed-noop[2.11.0] (0001_test_di) 0.5093 (1.0) 0.5093 (1.0) 0.5093 (1.0) 0.0000 (1.0) 0.5093 (1.0) 0.0000 (1.0) 0;0 1.9635 (1.0) 1 1
test_diff-diff-changed-noop[2.13.0] (0001_test_di) 0.5712 (1.12) 0.5712 (1.12) 0.5712 (1.12) 0.0000 (1.0) 0.5712 (1.12) 0.0000 (1.0) 0;0 1.7508 (0.89) 1 1
test_diff-diff-changed-noop[2.6.3] (0001_test_di) 2.4852 (4.88) 2.4852 (4.88) 2.4852 (4.88) 0.0000 (1.0) 2.4852 (4.88) 0.0000 (1.0) 0;0 0.4024 (0.20) 1 1
test_diff-diff-changed-noop[main] (0001_test_di) 0.6588 (1.29) 0.6588 (1.29) 0.6588 (1.29) 0.0000 (1.0) 0.6588 (1.29) 0.0000 (1.0) 0;0 1.5180 (0.77) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.10.2] (0001_test_di) 472.3108 (1.01) 472.3108 (1.01) 472.3108 (1.01) 0.0000 (1.0) 472.3108 (1.01) 0.0000 (1.0) 0;0 2.1172 (0.99) 1 1
test_diff-diff-noop[2.11.0] (0001_test_di) 466.3963 (1.0) 466.3963 (1.0) 466.3963 (1.0) 0.0000 (1.0) 466.3963 (1.0) 0.0000 (1.0) 0;0 2.1441 (1.0) 1 1
test_diff-diff-noop[2.13.0] (0001_test_di) 519.3671 (1.11) 519.3671 (1.11) 519.3671 (1.11) 0.0000 (1.0) 519.3671 (1.11) 0.0000 (1.0) 0;0 1.9254 (0.90) 1 1
test_diff-diff-noop[2.6.3] (0001_test_di) 1,838.1097 (3.94) 1,838.1097 (3.94) 1,838.1097 (3.94) 0.0000 (1.0) 1,838.1097 (3.94) 0.0000 (1.0) 0;0 0.5440 (0.25) 1 1
test_diff-diff-noop[main] (0001_test_di) 597.5220 (1.28) 597.5220 (1.28) 597.5220 (1.28) 0.0000 (1.0) 597.5220 (1.28) 0.0000 (1.0) 0;0 1.6736 (0.78) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 4 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.10.2] (0001_test_ex) 4.1901 (1.03) 4.1901 (1.03) 4.1901 (1.03) 0.0000 (1.0) 4.1901 (1.03) 0.0000 (1.0) 0;0 0.2387 (0.97) 1 1
test_exp_show-exp[2.11.0] (0001_test_ex) 4.2090 (1.04) 4.2090 (1.04) 4.2090 (1.04) 0.0000 (1.0) 4.2090 (1.04) 0.0000 (1.0) 0;0 0.2376 (0.97) 1 1
test_exp_show-exp[2.13.0] (0001_test_ex) 4.0637 (1.0) 4.0637 (1.0) 4.0637 (1.0) 0.0000 (1.0) 4.0637 (1.0) 0.0000 (1.0) 0;0 0.2461 (1.0) 1 1
test_exp_show-exp[main] (0001_test_ex) 4.1319 (1.02) 4.1319 (1.02) 4.1319 (1.02) 0.0000 (1.0) 4.1319 (1.02) 0.0000 (1.0) 0;0 0.2420 (0.98) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_gc-gc': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.10.2] (0001_test_gc) 564.3758 (1.00) 564.3758 (1.00) 564.3758 (1.00) 0.0000 (1.0) 564.3758 (1.00) 0.0000 (1.0) 0;0 1.7719 (1.00) 1 1
test_gc-gc[2.11.0] (0001_test_gc) 572.2641 (1.02) 572.2641 (1.02) 572.2641 (1.02) 0.0000 (1.0) 572.2641 (1.02) 0.0000 (1.0) 0;0 1.7474 (0.98) 1 1
test_gc-gc[2.13.0] (0001_test_gc) 563.5410 (1.0) 563.5410 (1.0) 563.5410 (1.0) 0.0000 (1.0) 563.5410 (1.0) 0.0000 (1.0) 0;0 1.7745 (1.0) 1 1
test_gc-gc[2.6.3] (0001_test_gc) 621.0100 (1.10) 621.0100 (1.10) 621.0100 (1.10) 0.0000 (1.0) 621.0100 (1.10) 0.0000 (1.0) 0;0 1.6103 (0.91) 1 1
test_gc-gc[main] (0001_test_gc) 603.9009 (1.07) 603.9009 (1.07) 603.9009 (1.07) 0.0000 (1.0) 603.9009 (1.07) 0.0000 (1.0) 0;0 1.6559 (0.93) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_get-get': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.10.2] (0001_test_ge) 6.5315 (1.63) 6.5315 (1.63) 6.5315 (1.63) 0.0000 (1.0) 6.5315 (1.63) 0.0000 (1.0) 0;0 0.1531 (0.61) 1 1
test_get-get[2.11.0] (0001_test_ge) 5.3728 (1.34) 5.3728 (1.34) 5.3728 (1.34) 0.0000 (1.0) 5.3728 (1.34) 0.0000 (1.0) 0;0 0.1861 (0.75) 1 1
test_get-get[2.13.0] (0001_test_ge) 5.3684 (1.34) 5.3684 (1.34) 5.3684 (1.34) 0.0000 (1.0) 5.3684 (1.34) 0.0000 (1.0) 0;0 0.1863 (0.75) 1 1
test_get-get[2.6.3] (0001_test_ge) 4.0155 (1.0) 4.0155 (1.0) 4.0155 (1.0) 0.0000 (1.0) 4.0155 (1.0) 0.0000 (1.0) 0;0 0.2490 (1.0) 1 1
test_get-get[main] (0001_test_ge) 5.3904 (1.34) 5.3904 (1.34) 5.3904 (1.34) 0.0000 (1.0) 5.3904 (1.34) 0.0000 (1.0) 0;0 0.1855 (0.74) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_help---help': 5 tests -------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.10.2] (0001_test_he) 145.2300 (1.08) 216.8936 (1.0) 171.3628 (1.01) 16.4556 (1.08) 171.2399 (1.0) 22.2345 (1.25) 37;2 5.8356 (1.00) 100 1
test_help---help[2.11.0] (0001_test_he) 135.0319 (1.0) 233.6061 (1.08) 170.5093 (1.0) 15.2732 (1.0) 171.2930 (1.00) 17.8524 (1.0) 35;3 5.8648 (1.0) 100 1
test_help---help[2.13.0] (0001_test_he) 152.0610 (1.13) 227.9009 (1.05) 180.9365 (1.06) 16.0358 (1.05) 174.1012 (1.02) 20.5131 (1.15) 27;4 5.5268 (0.94) 100 1
test_help---help[2.6.3] (0001_test_he) 179.4065 (1.33) 280.2822 (1.29) 228.8091 (1.34) 20.4866 (1.34) 219.6506 (1.28) 27.4308 (1.54) 22;0 4.3705 (0.75) 100 1
test_help---help[main] (0001_test_he) 154.0551 (1.14) 268.8101 (1.24) 181.2160 (1.06) 17.1147 (1.12) 175.5298 (1.03) 17.8817 (1.00) 25;4 5.5183 (0.94) 100 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_import-import': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.10.2] (0001_test_im) 17.1719 (1.39) 17.1719 (1.39) 17.1719 (1.39) 0.0000 (1.0) 17.1719 (1.39) 0.0000 (1.0) 0;0 0.0582 (0.72) 1 1
test_import-import[2.11.0] (0001_test_im) 16.3840 (1.32) 16.3840 (1.32) 16.3840 (1.32) 0.0000 (1.0) 16.3840 (1.32) 0.0000 (1.0) 0;0 0.0610 (0.76) 1 1
test_import-import[2.13.0] (0001_test_im) 15.3610 (1.24) 15.3610 (1.24) 15.3610 (1.24) 0.0000 (1.0) 15.3610 (1.24) 0.0000 (1.0) 0;0 0.0651 (0.81) 1 1
test_import-import[2.6.3] (0001_test_im) 12.3816 (1.0) 12.3816 (1.0) 12.3816 (1.0) 0.0000 (1.0) 12.3816 (1.0) 0.0000 (1.0) 0;0 0.0808 (1.0) 1 1
test_import-import[main] (0001_test_im) 15.0222 (1.21) 15.0222 (1.21) 15.0222 (1.21) 0.0000 (1.0) 15.0222 (1.21) 0.0000 (1.0) 0;0 0.0666 (0.82) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_init-init': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.10.2] (0001_test_in) 326.7202 (1.07) 408.3471 (1.27) 332.7018 (1.07) 8.3454 (2.70) 331.5076 (1.06) 4.3580 (1.20) 3;3 3.0057 (0.94) 100 1
test_init-init[2.11.0] (0001_test_in) 325.1990 (1.07) 372.8537 (1.16) 335.5742 (1.08) 5.5880 (1.80) 335.4546 (1.08) 6.3823 (1.75) 20;2 2.9800 (0.93) 100 1
test_init-init[2.13.0] (0001_test_in) 332.3667 (1.09) 401.7911 (1.25) 340.3103 (1.09) 7.3349 (2.37) 339.2733 (1.09) 3.8633 (1.06) 6;4 2.9385 (0.92) 100 1
test_init-init[2.6.3] (0001_test_in) 305.1139 (1.0) 321.2208 (1.0) 311.4088 (1.0) 3.0960 (1.0) 311.4011 (1.0) 3.6399 (1.0) 32;2 3.2112 (1.0) 100 1
test_init-init[main] (0001_test_in) 340.2922 (1.12) 401.7658 (1.25) 350.7420 (1.13) 7.7603 (2.51) 349.7331 (1.12) 8.4248 (2.31) 18;2 2.8511 (0.89) 100 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_list-list': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.10.2] (0001_test_li) 493.0630 (1.03) 493.0630 (1.03) 493.0630 (1.03) 0.0000 (1.0) 493.0630 (1.03) 0.0000 (1.0) 0;0 2.0281 (0.97) 1 1
test_list-list[2.11.0] (0001_test_li) 480.4494 (1.00) 480.4494 (1.00) 480.4494 (1.00) 0.0000 (1.0) 480.4494 (1.00) 0.0000 (1.0) 0;0 2.0814 (1.00) 1 1
test_list-list[2.13.0] (0001_test_li) 478.3261 (1.0) 478.3261 (1.0) 478.3261 (1.0) 0.0000 (1.0) 478.3261 (1.0) 0.0000 (1.0) 0;0 2.0906 (1.0) 1 1
test_list-list[2.6.3] (0001_test_li) 523.1244 (1.09) 523.1244 (1.09) 523.1244 (1.09) 0.0000 (1.0) 523.1244 (1.09) 0.0000 (1.0) 0;0 1.9116 (0.91) 1 1
test_list-list[main] (0001_test_li) 492.0053 (1.03) 492.0053 (1.03) 492.0053 (1.03) 0.0000 (1.0) 492.0053 (1.03) 0.0000 (1.0) 0;0 2.0325 (0.97) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_list-list-dvc-only': 5 tests -----------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.10.2] (0001_test_li) 477.8651 (1.02) 477.8651 (1.02) 477.8651 (1.02) 0.0000 (1.0) 477.8651 (1.02) 0.0000 (1.0) 0;0 2.0926 (0.98) 1 1
test_list-list-dvc-only[2.11.0] (0001_test_li) 478.0757 (1.02) 478.0757 (1.02) 478.0757 (1.02) 0.0000 (1.0) 478.0757 (1.02) 0.0000 (1.0) 0;0 2.0917 (0.98) 1 1
test_list-list-dvc-only[2.13.0] (0001_test_li) 468.0114 (1.0) 468.0114 (1.0) 468.0114 (1.0) 0.0000 (1.0) 468.0114 (1.0) 0.0000 (1.0) 0;0 2.1367 (1.0) 1 1
test_list-list-dvc-only[2.6.3] (0001_test_li) 492.0211 (1.05) 492.0211 (1.05) 492.0211 (1.05) 0.0000 (1.0) 492.0211 (1.05) 0.0000 (1.0) 0;0 2.0324 (0.95) 1 1
test_list-list-dvc-only[main] (0001_test_li) 492.3433 (1.05) 492.3433 (1.05) 492.3433 (1.05) 0.0000 (1.0) 492.3433 (1.05) 0.0000 (1.0) 0;0 2.0311 (0.95) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_plots_show-plots': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots_show-plots[2.10.2] (0001_test_pl) 930.2219 (1.54) 930.2219 (1.54) 930.2219 (1.54) 0.0000 (1.0) 930.2219 (1.54) 0.0000 (1.0) 0;0 1.0750 (0.65) 1 1
test_plots_show-plots[2.11.0] (0001_test_pl) 1,572.9585 (2.60) 1,572.9585 (2.60) 1,572.9585 (2.60) 0.0000 (1.0) 1,572.9585 (2.60) 0.0000 (1.0) 0;0 0.6357 (0.38) 1 1
test_plots_show-plots[2.13.0] (0001_test_pl) 1,620.1287 (2.68) 1,620.1287 (2.68) 1,620.1287 (2.68) 0.0000 (1.0) 1,620.1287 (2.68) 0.0000 (1.0) 0;0 0.6172 (0.37) 1 1
test_plots_show-plots[2.6.3] (0001_test_pl) 604.0583 (1.0) 604.0583 (1.0) 604.0583 (1.0) 0.0000 (1.0) 604.0583 (1.0) 0.0000 (1.0) 0;0 1.6555 (1.0) 1 1
test_plots_show-plots[main] (0001_test_pl) 1,749.3299 (2.90) 1,749.3299 (2.90) 1,749.3299 (2.90) 0.0000 (1.0) 1,749.3299 (2.90) 0.0000 (1.0) 0;0 0.5716 (0.35) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_pull-pull': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.10.2] (0001_test_pu) 10.2390 (1.27) 10.2390 (1.27) 10.2390 (1.27) 0.0000 (1.0) 10.2390 (1.27) 0.0000 (1.0) 0;0 0.0977 (0.79) 1 1
test_pull-pull[2.11.0] (0001_test_pu) 10.2314 (1.27) 10.2314 (1.27) 10.2314 (1.27) 0.0000 (1.0) 10.2314 (1.27) 0.0000 (1.0) 0;0 0.0977 (0.79) 1 1
test_pull-pull[2.13.0] (0001_test_pu) 9.7005 (1.20) 9.7005 (1.20) 9.7005 (1.20) 0.0000 (1.0) 9.7005 (1.20) 0.0000 (1.0) 0;0 0.1031 (0.83) 1 1
test_pull-pull[2.6.3] (0001_test_pu) 8.0867 (1.0) 8.0867 (1.0) 8.0867 (1.0) 0.0000 (1.0) 8.0867 (1.0) 0.0000 (1.0) 0;0 0.1237 (1.0) 1 1
test_pull-pull[main] (0001_test_pu) 9.6528 (1.19) 9.6528 (1.19) 9.6528 (1.19) 0.0000 (1.0) 9.6528 (1.19) 0.0000 (1.0) 0;0 0.1036 (0.84) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_push-push': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.10.2] (0001_test_pu) 6.7807 (1.19) 6.7807 (1.19) 6.7807 (1.19) 0.0000 (1.0) 6.7807 (1.19) 0.0000 (1.0) 0;0 0.1475 (0.84) 1 1
test_push-push[2.11.0] (0001_test_pu) 6.4786 (1.13) 6.4786 (1.13) 6.4786 (1.13) 0.0000 (1.0) 6.4786 (1.13) 0.0000 (1.0) 0;0 0.1544 (0.88) 1 1
test_push-push[2.13.0] (0001_test_pu) 6.6671 (1.17) 6.6671 (1.17) 6.6671 (1.17) 0.0000 (1.0) 6.6671 (1.17) 0.0000 (1.0) 0;0 0.1500 (0.86) 1 1
test_push-push[2.6.3] (0001_test_pu) 5.7168 (1.0) 5.7168 (1.0) 5.7168 (1.0) 0.0000 (1.0) 5.7168 (1.0) 0.0000 (1.0) 0;0 0.1749 (1.0) 1 1
test_push-push[main] (0001_test_pu) 6.3568 (1.11) 6.3568 (1.11) 6.3568 (1.11) 0.0000 (1.0) 6.3568 (1.11) 0.0000 (1.0) 0;0 0.1573 (0.90) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-add': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.10.2] (0001_test_sh) 7.1102 (1.18) 7.1102 (1.18) 7.1102 (1.18) 0.0000 (1.0) 7.1102 (1.18) 0.0000 (1.0) 0;0 0.1406 (0.85) 1 1
test_sharing-add[2.11.0] (0001_test_sh) 7.0000 (1.16) 7.0000 (1.16) 7.0000 (1.16) 0.0000 (1.0) 7.0000 (1.16) 0.0000 (1.0) 0;0 0.1429 (0.86) 1 1
test_sharing-add[2.13.0] (0001_test_sh) 6.5912 (1.09) 6.5912 (1.09) 6.5912 (1.09) 0.0000 (1.0) 6.5912 (1.09) 0.0000 (1.0) 0;0 0.1517 (0.92) 1 1
test_sharing-add[2.6.3] (0001_test_sh) 8.0949 (1.34) 8.0949 (1.34) 8.0949 (1.34) 0.0000 (1.0) 8.0949 (1.34) 0.0000 (1.0) 0;0 0.1235 (0.75) 1 1
test_sharing-add[main] (0001_test_sh) 6.0498 (1.0) 6.0498 (1.0) 6.0498 (1.0) 0.0000 (1.0) 6.0498 (1.0) 0.0000 (1.0) 0;0 0.1653 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing-add-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.10.2] (0001_test_sh) 3.3429 (3.69) 3.3429 (3.69) 3.3429 (3.69) 0.0000 (1.0) 3.3429 (3.69) 0.0000 (1.0) 0;0 0.2991 (0.27) 1 1
test_sharing-add-noop[2.11.0] (0001_test_sh) 3.3583 (3.71) 3.3583 (3.71) 3.3583 (3.71) 0.0000 (1.0) 3.3583 (3.71) 0.0000 (1.0) 0;0 0.2978 (0.27) 1 1
test_sharing-add-noop[2.13.0] (0001_test_sh) 3.5114 (3.88) 3.5114 (3.88) 3.5114 (3.88) 0.0000 (1.0) 3.5114 (3.88) 0.0000 (1.0) 0;0 0.2848 (0.26) 1 1
test_sharing-add-noop[2.6.3] (0001_test_sh) 0.9055 (1.0) 0.9055 (1.0) 0.9055 (1.0) 0.0000 (1.0) 0.9055 (1.0) 0.0000 (1.0) 0;0 1.1044 (1.0) 1 1
test_sharing-add-noop[main] (0001_test_sh) 3.4947 (3.86) 3.4947 (3.86) 3.4947 (3.86) 0.0000 (1.0) 3.4947 (3.86) 0.0000 (1.0) 0;0 0.2861 (0.26) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.10.2] (0001_test_sh) 493.5743 (1.28) 493.5743 (1.28) 493.5743 (1.28) 0.0000 (1.0) 493.5743 (1.28) 0.0000 (1.0) 0;0 2.0260 (0.78) 1 1
test_sharing-checkout-noop[2.11.0] (0001_test_sh) 491.9992 (1.28) 491.9992 (1.28) 491.9992 (1.28) 0.0000 (1.0) 491.9992 (1.28) 0.0000 (1.0) 0;0 2.0325 (0.78) 1 1
test_sharing-checkout-noop[2.13.0] (0001_test_sh) 532.5814 (1.39) 532.5814 (1.39) 532.5814 (1.39) 0.0000 (1.0) 532.5814 (1.39) 0.0000 (1.0) 0;0 1.8776 (0.72) 1 1
test_sharing-checkout-noop[2.6.3] (0001_test_sh) 384.1776 (1.0) 384.1776 (1.0) 384.1776 (1.0) 0.0000 (1.0) 384.1776 (1.0) 0.0000 (1.0) 0;0 2.6030 (1.0) 1 1
test_sharing-checkout-noop[main] (0001_test_sh) 547.2088 (1.42) 547.2088 (1.42) 547.2088 (1.42) 0.0000 (1.0) 547.2088 (1.42) 0.0000 (1.0) 0;0 1.8275 (0.70) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing-pull': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.10.2] (0001_test_sh) 7.5282 (1.24) 7.5282 (1.24) 7.5282 (1.24) 0.0000 (1.0) 7.5282 (1.24) 0.0000 (1.0) 0;0 0.1328 (0.81) 1 1
test_sharing-pull[2.11.0] (0001_test_sh) 7.5082 (1.24) 7.5082 (1.24) 7.5082 (1.24) 0.0000 (1.0) 7.5082 (1.24) 0.0000 (1.0) 0;0 0.1332 (0.81) 1 1
test_sharing-pull[2.13.0] (0001_test_sh) 7.8189 (1.29) 7.8189 (1.29) 7.8189 (1.29) 0.0000 (1.0) 7.8189 (1.29) 0.0000 (1.0) 0;0 0.1279 (0.78) 1 1
test_sharing-pull[2.6.3] (0001_test_sh) 6.0666 (1.0) 6.0666 (1.0) 6.0666 (1.0) 0.0000 (1.0) 6.0666 (1.0) 0.0000 (1.0) 0;0 0.1648 (1.0) 1 1
test_sharing-pull[main] (0001_test_sh) 7.5027 (1.24) 7.5027 (1.24) 7.5027 (1.24) 0.0000 (1.0) 7.5027 (1.24) 0.0000 (1.0) 0;0 0.1333 (0.81) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.10.2] (0001_test_sh) 530.2592 (1.28) 530.2592 (1.28) 530.2592 (1.28) 0.0000 (1.0) 530.2592 (1.28) 0.0000 (1.0) 0;0 1.8859 (0.78) 1 1
test_sharing-pull-noop[2.11.0] (0001_test_sh) 514.9297 (1.24) 514.9297 (1.24) 514.9297 (1.24) 0.0000 (1.0) 514.9297 (1.24) 0.0000 (1.0) 0;0 1.9420 (0.80) 1 1
test_sharing-pull-noop[2.13.0] (0001_test_sh) 573.9068 (1.39) 573.9068 (1.39) 573.9068 (1.39) 0.0000 (1.0) 573.9068 (1.39) 0.0000 (1.0) 0;0 1.7424 (0.72) 1 1
test_sharing-pull-noop[2.6.3] (0001_test_sh) 414.0181 (1.0) 414.0181 (1.0) 414.0181 (1.0) 0.0000 (1.0) 414.0181 (1.0) 0.0000 (1.0) 0;0 2.4154 (1.0) 1 1
test_sharing-pull-noop[main] (0001_test_sh) 573.2471 (1.38) 573.2471 (1.38) 573.2471 (1.38) 0.0000 (1.0) 573.2471 (1.38) 0.0000 (1.0) 0;0 1.7444 (0.72) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing-push': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.10.2] (0001_test_sh) 4.2911 (1.20) 4.2911 (1.20) 4.2911 (1.20) 0.0000 (1.0) 4.2911 (1.20) 0.0000 (1.0) 0;0 0.2330 (0.83) 1 1
test_sharing-push[2.11.0] (0001_test_sh) 4.2252 (1.18) 4.2252 (1.18) 4.2252 (1.18) 0.0000 (1.0) 4.2252 (1.18) 0.0000 (1.0) 0;0 0.2367 (0.84) 1 1
test_sharing-push[2.13.0] (0001_test_sh) 4.3146 (1.21) 4.3146 (1.21) 4.3146 (1.21) 0.0000 (1.0) 4.3146 (1.21) 0.0000 (1.0) 0;0 0.2318 (0.83) 1 1
test_sharing-push[2.6.3] (0001_test_sh) 3.5662 (1.0) 3.5662 (1.0) 3.5662 (1.0) 0.0000 (1.0) 3.5662 (1.0) 0.0000 (1.0) 0;0 0.2804 (1.0) 1 1
test_sharing-push[main] (0001_test_sh) 4.2539 (1.19) 4.2539 (1.19) 4.2539 (1.19) 0.0000 (1.0) 4.2539 (1.19) 0.0000 (1.0) 0;0 0.2351 (0.84) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.10.2] (0001_test_sh) 427.6118 (1.09) 427.6118 (1.09) 427.6118 (1.09) 0.0000 (1.0) 427.6118 (1.09) 0.0000 (1.0) 0;0 2.3386 (0.91) 1 1
test_sharing-push-noop[2.11.0] (0001_test_sh) 429.8029 (1.10) 429.8029 (1.10) 429.8029 (1.10) 0.0000 (1.0) 429.8029 (1.10) 0.0000 (1.0) 0;0 2.3266 (0.91) 1 1
test_sharing-push-noop[2.13.0] (0001_test_sh) 423.1307 (1.08) 423.1307 (1.08) 423.1307 (1.08) 0.0000 (1.0) 423.1307 (1.08) 0.0000 (1.0) 0;0 2.3633 (0.92) 1 1
test_sharing-push-noop[2.6.3] (0001_test_sh) 390.7345 (1.0) 390.7345 (1.0) 390.7345 (1.0) 0.0000 (1.0) 390.7345 (1.0) 0.0000 (1.0) 0;0 2.5593 (1.0) 1 1
test_sharing-push-noop[main] (0001_test_sh) 427.5798 (1.09) 427.5798 (1.09) 427.5798 (1.09) 0.0000 (1.0) 427.5798 (1.09) 0.0000 (1.0) 0;0 2.3387 (0.91) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_status-status': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.10.2] (0001_test_st) 2.1154 (1.63) 2.1154 (1.63) 2.1154 (1.63) 0.0000 (1.0) 2.1154 (1.63) 0.0000 (1.0) 0;0 0.4727 (0.61) 1 1
test_status-status[2.11.0] (0001_test_st) 2.0035 (1.55) 2.0035 (1.55) 2.0035 (1.55) 0.0000 (1.0) 2.0035 (1.55) 0.0000 (1.0) 0;0 0.4991 (0.65) 1 1
test_status-status[2.13.0] (0001_test_st) 1.2959 (1.0) 1.2959 (1.0) 1.2959 (1.0) 0.0000 (1.0) 1.2959 (1.0) 0.0000 (1.0) 0;0 0.7716 (1.0) 1 1
test_status-status[2.6.3] (0001_test_st) 3.3694 (2.60) 3.3694 (2.60) 3.3694 (2.60) 0.0000 (1.0) 3.3694 (2.60) 0.0000 (1.0) 0;0 0.2968 (0.38) 1 1
test_status-status[main] (0001_test_st) 1.2976 (1.00) 1.2976 (1.00) 1.2976 (1.00) 0.0000 (1.0) 1.2976 (1.00) 0.0000 (1.0) 0;0 0.7707 (1.00) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_status-status-changed': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.10.2] (0001_test_st) 1.4400 (2.90) 1.4400 (2.90) 1.4400 (2.90) 0.0000 (1.0) 1.4400 (2.90) 0.0000 (1.0) 0;0 0.6944 (0.35) 1 1
test_status-status-changed[2.11.0] (0001_test_st) 1.4204 (2.86) 1.4204 (2.86) 1.4204 (2.86) 0.0000 (1.0) 1.4204 (2.86) 0.0000 (1.0) 0;0 0.7040 (0.35) 1 1
test_status-status-changed[2.13.0] (0001_test_st) 0.4985 (1.00) 0.4985 (1.00) 0.4985 (1.00) 0.0000 (1.0) 0.4985 (1.00) 0.0000 (1.0) 0;0 2.0059 (1.00) 1 1
test_status-status-changed[2.6.3] (0001_test_st) 2.5285 (5.09) 2.5285 (5.09) 2.5285 (5.09) 0.0000 (1.0) 2.5285 (5.09) 0.0000 (1.0) 0;0 0.3955 (0.20) 1 1
test_status-status-changed[main] (0001_test_st) 0.4972 (1.0) 0.4972 (1.0) 0.4972 (1.0) 0.0000 (1.0) 0.4972 (1.0) 0.0000 (1.0) 0;0 2.0111 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.10.2] (0001_test_st) 1.3862 (2.82) 1.3862 (2.82) 1.3862 (2.82) 0.0000 (1.0) 1.3862 (2.82) 0.0000 (1.0) 0;0 0.7214 (0.36) 1 1
test_status-status-changed-noop[2.11.0] (0001_test_st) 0.6298 (1.28) 0.6298 (1.28) 0.6298 (1.28) 0.0000 (1.0) 0.6298 (1.28) 0.0000 (1.0) 0;0 1.5877 (0.78) 1 1
test_status-status-changed-noop[2.13.0] (0001_test_st) 0.4923 (1.0) 0.4923 (1.0) 0.4923 (1.0) 0.0000 (1.0) 0.4923 (1.0) 0.0000 (1.0) 0;0 2.0312 (1.0) 1 1
test_status-status-changed-noop[2.6.3] (0001_test_st) 2.5104 (5.10) 2.5104 (5.10) 2.5104 (5.10) 0.0000 (1.0) 2.5104 (5.10) 0.0000 (1.0) 0;0 0.3983 (0.20) 1 1
test_status-status-changed-noop[main] (0001_test_st) 0.4932 (1.00) 0.4932 (1.00) 0.4932 (1.00) 0.0000 (1.0) 0.4932 (1.00) 0.0000 (1.0) 0;0 2.0276 (1.00) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_status-status-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.10.2] (0001_test_st) 444.4059 (1.06) 444.4059 (1.06) 444.4059 (1.06) 0.0000 (1.0) 444.4059 (1.06) 0.0000 (1.0) 0;0 2.2502 (0.94) 1 1
test_status-status-noop[2.11.0] (0001_test_st) 428.0747 (1.03) 428.0747 (1.03) 428.0747 (1.03) 0.0000 (1.0) 428.0747 (1.03) 0.0000 (1.0) 0;0 2.3360 (0.98) 1 1
test_status-status-noop[2.13.0] (0001_test_st) 493.1978 (1.18) 493.1978 (1.18) 493.1978 (1.18) 0.0000 (1.0) 493.1978 (1.18) 0.0000 (1.0) 0;0 2.0276 (0.85) 1 1
test_status-status-noop[2.6.3] (0001_test_st) 417.4301 (1.0) 417.4301 (1.0) 417.4301 (1.0) 0.0000 (1.0) 417.4301 (1.0) 0.0000 (1.0) 0;0 2.3956 (1.0) 1 1
test_status-status-noop[main] (0001_test_st) 493.6284 (1.18) 493.6284 (1.18) 493.6284 (1.18) 0.0000 (1.0) 493.6284 (1.18) 0.0000 (1.0) 0;0 2.0258 (0.85) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```
github-actions[bot] commented 2 years ago 

Fri Jul 22 13:25:19 UTC 2022
dataset: small
project: example-get-started

------------------------------------------------------------------------------- benchmark 'test_add-add': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.10.2] (0001_test_ad) 6.6889 (1.14) 6.6889 (1.14) 6.6889 (1.14) 0.0000 (1.0) 6.6889 (1.14) 0.0000 (1.0) 0;0 0.1495 (0.87) 1 1
test_add-add[2.11.0] (0001_test_ad) 6.8787 (1.18) 6.8787 (1.18) 6.8787 (1.18) 0.0000 (1.0) 6.8787 (1.18) 0.0000 (1.0) 0;0 0.1454 (0.85) 1 1
test_add-add[2.13.0] (0001_test_ad) 6.3060 (1.08) 6.3060 (1.08) 6.3060 (1.08) 0.0000 (1.0) 6.3060 (1.08) 0.0000 (1.0) 0;0 0.1586 (0.93) 1 1
test_add-add[2.6.3] (0001_test_ad) 7.6006 (1.30) 7.6006 (1.30) 7.6006 (1.30) 0.0000 (1.0) 7.6006 (1.30) 0.0000 (1.0) 0;0 0.1316 (0.77) 1 1
test_add-add[main] (0001_test_ad) 5.8509 (1.0) 5.8509 (1.0) 5.8509 (1.0) 0.0000 (1.0) 5.8509 (1.0) 0.0000 (1.0) 0;0 0.1709 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout[2.10.2] (0001_test_ch) 3.5958 (1.44) 3.5958 (1.44) 3.5958 (1.44) 0.0000 (1.0) 3.5958 (1.44) 0.0000 (1.0) 0;0 0.2781 (0.69) 1 1
test_checkout-checkout[2.11.0] (0001_test_ch) 3.5974 (1.44) 3.5974 (1.44) 3.5974 (1.44) 0.0000 (1.0) 3.5974 (1.44) 0.0000 (1.0) 0;0 0.2780 (0.69) 1 1
test_checkout-checkout[2.13.0] (0001_test_ch) 3.0653 (1.23) 3.0653 (1.23) 3.0653 (1.23) 0.0000 (1.0) 3.0653 (1.23) 0.0000 (1.0) 0;0 0.3262 (0.82) 1 1
test_checkout-checkout[2.6.3] (0001_test_ch) 2.4989 (1.0) 2.4989 (1.0) 2.4989 (1.0) 0.0000 (1.0) 2.4989 (1.0) 0.0000 (1.0) 0;0 0.4002 (1.0) 1 1
test_checkout-checkout[main] (0001_test_ch) 3.0228 (1.21) 3.0228 (1.21) 3.0228 (1.21) 0.0000 (1.0) 3.0228 (1.21) 0.0000 (1.0) 0;0 0.3308 (0.83) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_checkout-checkout-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop[2.10.2] (0001_test_ch) 507.6072 (1.26) 507.6072 (1.26) 507.6072 (1.26) 0.0000 (1.0) 507.6072 (1.26) 0.0000 (1.0) 0;0 1.9700 (0.79) 1 1
test_checkout-checkout-noop[2.11.0] (0001_test_ch) 501.1532 (1.24) 501.1532 (1.24) 501.1532 (1.24) 0.0000 (1.0) 501.1532 (1.24) 0.0000 (1.0) 0;0 1.9954 (0.80) 1 1
test_checkout-checkout-noop[2.13.0] (0001_test_ch) 554.2268 (1.38) 554.2268 (1.38) 554.2268 (1.38) 0.0000 (1.0) 554.2268 (1.38) 0.0000 (1.0) 0;0 1.8043 (0.73) 1 1
test_checkout-checkout-noop[2.6.3] (0001_test_ch) 402.7060 (1.0) 402.7060 (1.0) 402.7060 (1.0) 0.0000 (1.0) 402.7060 (1.0) 0.0000 (1.0) 0;0 2.4832 (1.0) 1 1
test_checkout-checkout-noop[main] (0001_test_ch) 564.3359 (1.40) 564.3359 (1.40) 564.3359 (1.40) 0.0000 (1.0) 564.3359 (1.40) 0.0000 (1.0) 0;0 1.7720 (0.71) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-update': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update[2.10.2] (0001_test_ch) 1.3376 (2.09) 1.3376 (2.09) 1.3376 (2.09) 0.0000 (1.0) 1.3376 (2.09) 0.0000 (1.0) 0;0 0.7476 (0.48) 1 1
test_checkout-checkout-update[2.11.0] (0001_test_ch) 1.3193 (2.06) 1.3193 (2.06) 1.3193 (2.06) 0.0000 (1.0) 1.3193 (2.06) 0.0000 (1.0) 0;0 0.7580 (0.49) 1 1
test_checkout-checkout-update[2.13.0] (0001_test_ch) 0.6406 (1.0) 0.6406 (1.0) 0.6406 (1.0) 0.0000 (1.0) 0.6406 (1.0) 0.0000 (1.0) 0;0 1.5609 (1.0) 1 1
test_checkout-checkout-update[2.6.3] (0001_test_ch) 2.9294 (4.57) 2.9294 (4.57) 2.9294 (4.57) 0.0000 (1.0) 2.9294 (4.57) 0.0000 (1.0) 0;0 0.3414 (0.22) 1 1
test_checkout-checkout-update[main] (0001_test_ch) 0.7311 (1.14) 0.7311 (1.14) 0.7311 (1.14) 0.0000 (1.0) 0.7311 (1.14) 0.0000 (1.0) 0;0 1.3679 (0.88) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_diff-diff': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.10.2] (0001_test_di) 3.9622 (1.83) 3.9622 (1.83) 3.9622 (1.83) 0.0000 (1.0) 3.9622 (1.83) 0.0000 (1.0) 0;0 0.2524 (0.55) 1 1
test_diff-diff[2.11.0] (0001_test_di) 2.7416 (1.27) 2.7416 (1.27) 2.7416 (1.27) 0.0000 (1.0) 2.7416 (1.27) 0.0000 (1.0) 0;0 0.3647 (0.79) 1 1
test_diff-diff[2.13.0] (0001_test_di) 2.1620 (1.0) 2.1620 (1.0) 2.1620 (1.0) 0.0000 (1.0) 2.1620 (1.0) 0.0000 (1.0) 0;0 0.4625 (1.0) 1 1
test_diff-diff[2.6.3] (0001_test_di) 5.1773 (2.39) 5.1773 (2.39) 5.1773 (2.39) 0.0000 (1.0) 5.1773 (2.39) 0.0000 (1.0) 0;0 0.1932 (0.42) 1 1
test_diff-diff[main] (0001_test_di) 2.2857 (1.06) 2.2857 (1.06) 2.2857 (1.06) 0.0000 (1.0) 2.2857 (1.06) 0.0000 (1.0) 0;0 0.4375 (0.95) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.10.2] (0001_test_di) 1.9656 (2.22) 1.9656 (2.22) 1.9656 (2.22) 0.0000 (1.0) 1.9656 (2.22) 0.0000 (1.0) 0;0 0.5088 (0.45) 1 1
test_diff-diff-changed[2.11.0] (0001_test_di) 1.8230 (2.06) 1.8230 (2.06) 1.8230 (2.06) 0.0000 (1.0) 1.8230 (2.06) 0.0000 (1.0) 0;0 0.5486 (0.48) 1 1
test_diff-diff-changed[2.13.0] (0001_test_di) 0.8836 (1.0) 0.8836 (1.0) 0.8836 (1.0) 0.0000 (1.0) 0.8836 (1.0) 0.0000 (1.0) 0;0 1.1317 (1.0) 1 1
test_diff-diff-changed[2.6.3] (0001_test_di) 3.9772 (4.50) 3.9772 (4.50) 3.9772 (4.50) 0.0000 (1.0) 3.9772 (4.50) 0.0000 (1.0) 0;0 0.2514 (0.22) 1 1
test_diff-diff-changed[main] (0001_test_di) 0.9710 (1.10) 0.9710 (1.10) 0.9710 (1.10) 0.0000 (1.0) 0.9710 (1.10) 0.0000 (1.0) 0;0 1.0299 (0.91) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.10.2] (0001_test_di) 1.9232 (2.57) 1.9232 (2.57) 1.9232 (2.57) 0.0000 (1.0) 1.9232 (2.57) 0.0000 (1.0) 0;0 0.5200 (0.39) 1 1
test_diff-diff-changed-noop[2.11.0] (0001_test_di) 0.7495 (1.0) 0.7495 (1.0) 0.7495 (1.0) 0.0000 (1.0) 0.7495 (1.0) 0.0000 (1.0) 0;0 1.3342 (1.0) 1 1
test_diff-diff-changed-noop[2.13.0] (0001_test_di) 0.9211 (1.23) 0.9211 (1.23) 0.9211 (1.23) 0.0000 (1.0) 0.9211 (1.23) 0.0000 (1.0) 0;0 1.0856 (0.81) 1 1
test_diff-diff-changed-noop[2.6.3] (0001_test_di) 3.9985 (5.33) 3.9985 (5.33) 3.9985 (5.33) 0.0000 (1.0) 3.9985 (5.33) 0.0000 (1.0) 0;0 0.2501 (0.19) 1 1
test_diff-diff-changed-noop[main] (0001_test_di) 0.9806 (1.31) 0.9806 (1.31) 0.9806 (1.31) 0.0000 (1.0) 0.9806 (1.31) 0.0000 (1.0) 0;0 1.0198 (0.76) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.10.2] (0001_test_di) 741.9236 (1.10) 741.9236 (1.10) 741.9236 (1.10) 0.0000 (1.0) 741.9236 (1.10) 0.0000 (1.0) 0;0 1.3478 (0.91) 1 1
test_diff-diff-noop[2.11.0] (0001_test_di) 673.8718 (1.0) 673.8718 (1.0) 673.8718 (1.0) 0.0000 (1.0) 673.8718 (1.0) 0.0000 (1.0) 0;0 1.4840 (1.0) 1 1
test_diff-diff-noop[2.13.0] (0001_test_di) 800.8827 (1.19) 800.8827 (1.19) 800.8827 (1.19) 0.0000 (1.0) 800.8827 (1.19) 0.0000 (1.0) 0;0 1.2486 (0.84) 1 1
test_diff-diff-noop[2.6.3] (0001_test_di) 2,782.0823 (4.13) 2,782.0823 (4.13) 2,782.0823 (4.13) 0.0000 (1.0) 2,782.0823 (4.13) 0.0000 (1.0) 0;0 0.3594 (0.24) 1 1
test_diff-diff-noop[main] (0001_test_di) 894.4456 (1.33) 894.4456 (1.33) 894.4456 (1.33) 0.0000 (1.0) 894.4456 (1.33) 0.0000 (1.0) 0;0 1.1180 (0.75) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 4 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.10.2] (0001_test_ex) 3.4083 (1.05) 3.4083 (1.05) 3.4083 (1.05) 0.0000 (1.0) 3.4083 (1.05) 0.0000 (1.0) 0;0 0.2934 (0.96) 1 1
test_exp_show-exp[2.11.0] (0001_test_ex) 3.4072 (1.04) 3.4072 (1.04) 3.4072 (1.04) 0.0000 (1.0) 3.4072 (1.04) 0.0000 (1.0) 0;0 0.2935 (0.96) 1 1
test_exp_show-exp[2.13.0] (0001_test_ex) 3.2614 (1.0) 3.2614 (1.0) 3.2614 (1.0) 0.0000 (1.0) 3.2614 (1.0) 0.0000 (1.0) 0;0 0.3066 (1.0) 1 1
test_exp_show-exp[main] (0001_test_ex) 3.4269 (1.05) 3.4269 (1.05) 3.4269 (1.05) 0.0000 (1.0) 3.4269 (1.05) 0.0000 (1.0) 0;0 0.2918 (0.95) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_gc-gc': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.10.2] (0001_test_gc) 457.2413 (1.00) 457.2413 (1.00) 457.2413 (1.00) 0.0000 (1.0) 457.2413 (1.00) 0.0000 (1.0) 0;0 2.1870 (1.00) 1 1
test_gc-gc[2.11.0] (0001_test_gc) 491.6506 (1.08) 491.6506 (1.08) 491.6506 (1.08) 0.0000 (1.0) 491.6506 (1.08) 0.0000 (1.0) 0;0 2.0340 (0.93) 1 1
test_gc-gc[2.13.0] (0001_test_gc) 455.7564 (1.0) 455.7564 (1.0) 455.7564 (1.0) 0.0000 (1.0) 455.7564 (1.0) 0.0000 (1.0) 0;0 2.1942 (1.0) 1 1
test_gc-gc[2.6.3] (0001_test_gc) 567.5276 (1.25) 567.5276 (1.25) 567.5276 (1.25) 0.0000 (1.0) 567.5276 (1.25) 0.0000 (1.0) 0;0 1.7620 (0.80) 1 1
test_gc-gc[main] (0001_test_gc) 459.1939 (1.01) 459.1939 (1.01) 459.1939 (1.01) 0.0000 (1.0) 459.1939 (1.01) 0.0000 (1.0) 0;0 2.1777 (0.99) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_get-get': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.10.2] (0001_test_ge) 6.5547 (1.61) 6.5547 (1.61) 6.5547 (1.61) 0.0000 (1.0) 6.5547 (1.61) 0.0000 (1.0) 0;0 0.1526 (0.62) 1 1
test_get-get[2.11.0] (0001_test_ge) 5.4381 (1.34) 5.4381 (1.34) 5.4381 (1.34) 0.0000 (1.0) 5.4381 (1.34) 0.0000 (1.0) 0;0 0.1839 (0.75) 1 1
test_get-get[2.13.0] (0001_test_ge) 5.3765 (1.32) 5.3765 (1.32) 5.3765 (1.32) 0.0000 (1.0) 5.3765 (1.32) 0.0000 (1.0) 0;0 0.1860 (0.76) 1 1
test_get-get[2.6.3] (0001_test_ge) 4.0596 (1.0) 4.0596 (1.0) 4.0596 (1.0) 0.0000 (1.0) 4.0596 (1.0) 0.0000 (1.0) 0;0 0.2463 (1.0) 1 1
test_get-get[main] (0001_test_ge) 5.5903 (1.38) 5.5903 (1.38) 5.5903 (1.38) 0.0000 (1.0) 5.5903 (1.38) 0.0000 (1.0) 0;0 0.1789 (0.73) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_help---help': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.10.2] (0001_test_he) 145.9443 (1.0) 154.6547 (1.01) 148.2715 (1.0) 1.8993 (1.52) 147.6818 (1.0) 1.6256 (1.04) 20;10 6.7444 (1.0) 100 1
test_help---help[2.11.0] (0001_test_he) 146.7444 (1.01) 155.3845 (1.01) 149.4561 (1.01) 1.7299 (1.38) 149.0318 (1.01) 1.7432 (1.11) 24;7 6.6909 (0.99) 100 1
test_help---help[2.13.0] (0001_test_he) 146.6897 (1.01) 153.6777 (1.0) 149.8863 (1.01) 1.2525 (1.0) 149.8455 (1.01) 1.5702 (1.0) 28;4 6.6717 (0.99) 100 1
test_help---help[2.6.3] (0001_test_he) 188.3563 (1.29) 198.6894 (1.29) 191.8850 (1.29) 1.7979 (1.44) 191.4844 (1.30) 1.8922 (1.21) 23;7 5.2115 (0.77) 100 1
test_help---help[main] (0001_test_he) 151.6606 (1.04) 158.4901 (1.03) 154.4847 (1.04) 1.4897 (1.19) 154.3801 (1.05) 2.1804 (1.39) 33;0 6.4731 (0.96) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_import-import': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.10.2] (0001_test_im) 22.1707 (1.37) 22.1707 (1.37) 22.1707 (1.37) 0.0000 (1.0) 22.1707 (1.37) 0.0000 (1.0) 0;0 0.0451 (0.73) 1 1
test_import-import[2.11.0] (0001_test_im) 20.1125 (1.24) 20.1125 (1.24) 20.1125 (1.24) 0.0000 (1.0) 20.1125 (1.24) 0.0000 (1.0) 0;0 0.0497 (0.81) 1 1
test_import-import[2.13.0] (0001_test_im) 19.3058 (1.19) 19.3058 (1.19) 19.3058 (1.19) 0.0000 (1.0) 19.3058 (1.19) 0.0000 (1.0) 0;0 0.0518 (0.84) 1 1
test_import-import[2.6.3] (0001_test_im) 16.2080 (1.0) 16.2080 (1.0) 16.2080 (1.0) 0.0000 (1.0) 16.2080 (1.0) 0.0000 (1.0) 0;0 0.0617 (1.0) 1 1
test_import-import[main] (0001_test_im) 19.6809 (1.21) 19.6809 (1.21) 19.6809 (1.21) 0.0000 (1.0) 19.6809 (1.21) 0.0000 (1.0) 0;0 0.0508 (0.82) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_init-init': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.10.2] (0001_test_in) 328.1717 (1.06) 427.7074 (1.31) 342.6719 (1.08) 16.6291 (5.45) 338.9549 (1.07) 6.4784 (1.60) 4;7 2.9182 (0.92) 100 1
test_init-init[2.11.0] (0001_test_in) 332.1981 (1.07) 499.9896 (1.53) 341.6934 (1.08) 19.3955 (6.36) 338.0941 (1.07) 4.7745 (1.18) 3;7 2.9266 (0.93) 100 1
test_init-init[2.13.0] (0001_test_in) 333.0616 (1.08) 421.1741 (1.29) 342.4472 (1.08) 13.5716 (4.45) 340.1197 (1.08) 5.1598 (1.28) 3;3 2.9202 (0.92) 100 1
test_init-init[2.6.3] (0001_test_in) 309.1931 (1.0) 326.8141 (1.0) 316.2930 (1.0) 3.0509 (1.0) 315.9537 (1.0) 4.0450 (1.0) 30;1 3.1616 (1.0) 100 1
test_init-init[main] (0001_test_in) 339.2662 (1.10) 456.4366 (1.40) 349.4230 (1.10) 18.0627 (5.92) 345.6940 (1.09) 5.4055 (1.34) 6;6 2.8619 (0.91) 100 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_list-list': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.10.2] (0001_test_li) 420.0235 (1.01) 420.0235 (1.01) 420.0235 (1.01) 0.0000 (1.0) 420.0235 (1.01) 0.0000 (1.0) 0;0 2.3808 (0.99) 1 1
test_list-list[2.11.0] (0001_test_li) 425.8195 (1.02) 425.8195 (1.02) 425.8195 (1.02) 0.0000 (1.0) 425.8195 (1.02) 0.0000 (1.0) 0;0 2.3484 (0.98) 1 1
test_list-list[2.13.0] (0001_test_li) 416.2927 (1.0) 416.2927 (1.0) 416.2927 (1.0) 0.0000 (1.0) 416.2927 (1.0) 0.0000 (1.0) 0;0 2.4022 (1.0) 1 1
test_list-list[2.6.3] (0001_test_li) 756.2752 (1.82) 756.2752 (1.82) 756.2752 (1.82) 0.0000 (1.0) 756.2752 (1.82) 0.0000 (1.0) 0;0 1.3223 (0.55) 1 1
test_list-list[main] (0001_test_li) 438.0691 (1.05) 438.0691 (1.05) 438.0691 (1.05) 0.0000 (1.0) 438.0691 (1.05) 0.0000 (1.0) 0;0 2.2827 (0.95) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_list-list-dvc-only': 5 tests -----------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.10.2] (0001_test_li) 409.8576 (1.0) 409.8576 (1.0) 409.8576 (1.0) 0.0000 (1.0) 409.8576 (1.0) 0.0000 (1.0) 0;0 2.4399 (1.0) 1 1
test_list-list-dvc-only[2.11.0] (0001_test_li) 420.9202 (1.03) 420.9202 (1.03) 420.9202 (1.03) 0.0000 (1.0) 420.9202 (1.03) 0.0000 (1.0) 0;0 2.3757 (0.97) 1 1
test_list-list-dvc-only[2.13.0] (0001_test_li) 418.5493 (1.02) 418.5493 (1.02) 418.5493 (1.02) 0.0000 (1.0) 418.5493 (1.02) 0.0000 (1.0) 0;0 2.3892 (0.98) 1 1
test_list-list-dvc-only[2.6.3] (0001_test_li) 518.9178 (1.27) 518.9178 (1.27) 518.9178 (1.27) 0.0000 (1.0) 518.9178 (1.27) 0.0000 (1.0) 0;0 1.9271 (0.79) 1 1
test_list-list-dvc-only[main] (0001_test_li) 414.1445 (1.01) 414.1445 (1.01) 414.1445 (1.01) 0.0000 (1.0) 414.1445 (1.01) 0.0000 (1.0) 0;0 2.4146 (0.99) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_plots_show-plots': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots_show-plots[2.10.2] (0001_test_pl) 878.7609 (1.41) 878.7609 (1.41) 878.7609 (1.41) 0.0000 (1.0) 878.7609 (1.41) 0.0000 (1.0) 0;0 1.1380 (0.71) 1 1
test_plots_show-plots[2.11.0] (0001_test_pl) 1,339.8262 (2.16) 1,339.8262 (2.16) 1,339.8262 (2.16) 0.0000 (1.0) 1,339.8262 (2.16) 0.0000 (1.0) 0;0 0.7464 (0.46) 1 1
test_plots_show-plots[2.13.0] (0001_test_pl) 1,328.1003 (2.14) 1,328.1003 (2.14) 1,328.1003 (2.14) 0.0000 (1.0) 1,328.1003 (2.14) 0.0000 (1.0) 0;0 0.7530 (0.47) 1 1
test_plots_show-plots[2.6.3] (0001_test_pl) 621.1895 (1.0) 621.1895 (1.0) 621.1895 (1.0) 0.0000 (1.0) 621.1895 (1.0) 0.0000 (1.0) 0;0 1.6098 (1.0) 1 1
test_plots_show-plots[main] (0001_test_pl) 1,696.5611 (2.73) 1,696.5611 (2.73) 1,696.5611 (2.73) 0.0000 (1.0) 1,696.5611 (2.73) 0.0000 (1.0) 0;0 0.5894 (0.37) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_pull-pull': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.10.2] (0001_test_pu) 10.4794 (1.30) 10.4794 (1.30) 10.4794 (1.30) 0.0000 (1.0) 10.4794 (1.30) 0.0000 (1.0) 0;0 0.0954 (0.77) 1 1
test_pull-pull[2.11.0] (0001_test_pu) 9.7619 (1.21) 9.7619 (1.21) 9.7619 (1.21) 0.0000 (1.0) 9.7619 (1.21) 0.0000 (1.0) 0;0 0.1024 (0.82) 1 1
test_pull-pull[2.13.0] (0001_test_pu) 10.1385 (1.26) 10.1385 (1.26) 10.1385 (1.26) 0.0000 (1.0) 10.1385 (1.26) 0.0000 (1.0) 0;0 0.0986 (0.79) 1 1
test_pull-pull[2.6.3] (0001_test_pu) 8.0398 (1.0) 8.0398 (1.0) 8.0398 (1.0) 0.0000 (1.0) 8.0398 (1.0) 0.0000 (1.0) 0;0 0.1244 (1.0) 1 1
test_pull-pull[main] (0001_test_pu) 9.8784 (1.23) 9.8784 (1.23) 9.8784 (1.23) 0.0000 (1.0) 9.8784 (1.23) 0.0000 (1.0) 0;0 0.1012 (0.81) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_push-push': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.10.2] (0001_test_pu) 5.5611 (1.16) 5.5611 (1.16) 5.5611 (1.16) 0.0000 (1.0) 5.5611 (1.16) 0.0000 (1.0) 0;0 0.1798 (0.87) 1 1
test_push-push[2.11.0] (0001_test_pu) 5.6779 (1.18) 5.6779 (1.18) 5.6779 (1.18) 0.0000 (1.0) 5.6779 (1.18) 0.0000 (1.0) 0;0 0.1761 (0.85) 1 1
test_push-push[2.13.0] (0001_test_pu) 5.6648 (1.18) 5.6648 (1.18) 5.6648 (1.18) 0.0000 (1.0) 5.6648 (1.18) 0.0000 (1.0) 0;0 0.1765 (0.85) 1 1
test_push-push[2.6.3] (0001_test_pu) 4.8128 (1.0) 4.8128 (1.0) 4.8128 (1.0) 0.0000 (1.0) 4.8128 (1.0) 0.0000 (1.0) 0;0 0.2078 (1.0) 1 1
test_push-push[main] (0001_test_pu) 5.7498 (1.19) 5.7498 (1.19) 5.7498 (1.19) 0.0000 (1.0) 5.7498 (1.19) 0.0000 (1.0) 0;0 0.1739 (0.84) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_sharing-add': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.10.2] (0001_test_sh) 9.7934 (1.23) 9.7934 (1.23) 9.7934 (1.23) 0.0000 (1.0) 9.7934 (1.23) 0.0000 (1.0) 0;0 0.1021 (0.82) 1 1
test_sharing-add[2.11.0] (0001_test_sh) 9.3161 (1.17) 9.3161 (1.17) 9.3161 (1.17) 0.0000 (1.0) 9.3161 (1.17) 0.0000 (1.0) 0;0 0.1073 (0.86) 1 1
test_sharing-add[2.13.0] (0001_test_sh) 7.9849 (1.0) 7.9849 (1.0) 7.9849 (1.0) 0.0000 (1.0) 7.9849 (1.0) 0.0000 (1.0) 0;0 0.1252 (1.0) 1 1
test_sharing-add[2.6.3] (0001_test_sh) 10.7805 (1.35) 10.7805 (1.35) 10.7805 (1.35) 0.0000 (1.0) 10.7805 (1.35) 0.0000 (1.0) 0;0 0.0928 (0.74) 1 1
test_sharing-add[main] (0001_test_sh) 8.3100 (1.04) 8.3100 (1.04) 8.3100 (1.04) 0.0000 (1.0) 8.3100 (1.04) 0.0000 (1.0) 0;0 0.1203 (0.96) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing-add-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.10.2] (0001_test_sh) 4.4479 (3.81) 4.4479 (3.81) 4.4479 (3.81) 0.0000 (1.0) 4.4479 (3.81) 0.0000 (1.0) 0;0 0.2248 (0.26) 1 1
test_sharing-add-noop[2.11.0] (0001_test_sh) 4.6147 (3.95) 4.6147 (3.95) 4.6147 (3.95) 0.0000 (1.0) 4.6147 (3.95) 0.0000 (1.0) 0;0 0.2167 (0.25) 1 1
test_sharing-add-noop[2.13.0] (0001_test_sh) 4.5102 (3.86) 4.5102 (3.86) 4.5102 (3.86) 0.0000 (1.0) 4.5102 (3.86) 0.0000 (1.0) 0;0 0.2217 (0.26) 1 1
test_sharing-add-noop[2.6.3] (0001_test_sh) 1.1683 (1.0) 1.1683 (1.0) 1.1683 (1.0) 0.0000 (1.0) 1.1683 (1.0) 0.0000 (1.0) 0;0 0.8559 (1.0) 1 1
test_sharing-add-noop[main] (0001_test_sh) 4.6198 (3.95) 4.6198 (3.95) 4.6198 (3.95) 0.0000 (1.0) 4.6198 (3.95) 0.0000 (1.0) 0;0 0.2165 (0.25) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.10.2] (0001_test_sh) 610.8528 (1.26) 610.8528 (1.26) 610.8528 (1.26) 0.0000 (1.0) 610.8528 (1.26) 0.0000 (1.0) 0;0 1.6371 (0.79) 1 1
test_sharing-checkout-noop[2.11.0] (0001_test_sh) 667.1573 (1.37) 667.1573 (1.37) 667.1573 (1.37) 0.0000 (1.0) 667.1573 (1.37) 0.0000 (1.0) 0;0 1.4989 (0.73) 1 1
test_sharing-checkout-noop[2.13.0] (0001_test_sh) 652.6643 (1.34) 652.6643 (1.34) 652.6643 (1.34) 0.0000 (1.0) 652.6643 (1.34) 0.0000 (1.0) 0;0 1.5322 (0.74) 1 1
test_sharing-checkout-noop[2.6.3] (0001_test_sh) 485.6135 (1.0) 485.6135 (1.0) 485.6135 (1.0) 0.0000 (1.0) 485.6135 (1.0) 0.0000 (1.0) 0;0 2.0593 (1.0) 1 1
test_sharing-checkout-noop[main] (0001_test_sh) 665.5102 (1.37) 665.5102 (1.37) 665.5102 (1.37) 0.0000 (1.0) 665.5102 (1.37) 0.0000 (1.0) 0;0 1.5026 (0.73) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_sharing-pull': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.10.2] (0001_test_sh) 10.4664 (1.22) 10.4664 (1.22) 10.4664 (1.22) 0.0000 (1.0) 10.4664 (1.22) 0.0000 (1.0) 0;0 0.0955 (0.82) 1 1
test_sharing-pull[2.11.0] (0001_test_sh) 10.9395 (1.28) 10.9395 (1.28) 10.9395 (1.28) 0.0000 (1.0) 10.9395 (1.28) 0.0000 (1.0) 0;0 0.0914 (0.78) 1 1
test_sharing-pull[2.13.0] (0001_test_sh) 9.8987 (1.16) 9.8987 (1.16) 9.8987 (1.16) 0.0000 (1.0) 9.8987 (1.16) 0.0000 (1.0) 0;0 0.1010 (0.87) 1 1
test_sharing-pull[2.6.3] (0001_test_sh) 8.5656 (1.0) 8.5656 (1.0) 8.5656 (1.0) 0.0000 (1.0) 8.5656 (1.0) 0.0000 (1.0) 0;0 0.1167 (1.0) 1 1
test_sharing-pull[main] (0001_test_sh) 10.0708 (1.18) 10.0708 (1.18) 10.0708 (1.18) 0.0000 (1.0) 10.0708 (1.18) 0.0000 (1.0) 0;0 0.0993 (0.85) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.10.2] (0001_test_sh) 661.9738 (1.23) 661.9738 (1.23) 661.9738 (1.23) 0.0000 (1.0) 661.9738 (1.23) 0.0000 (1.0) 0;0 1.5106 (0.81) 1 1
test_sharing-pull-noop[2.11.0] (0001_test_sh) 692.0063 (1.29) 692.0063 (1.29) 692.0063 (1.29) 0.0000 (1.0) 692.0063 (1.29) 0.0000 (1.0) 0;0 1.4451 (0.78) 1 1
test_sharing-pull-noop[2.13.0] (0001_test_sh) 735.8164 (1.37) 735.8164 (1.37) 735.8164 (1.37) 0.0000 (1.0) 735.8164 (1.37) 0.0000 (1.0) 0;0 1.3590 (0.73) 1 1
test_sharing-pull-noop[2.6.3] (0001_test_sh) 537.7029 (1.0) 537.7029 (1.0) 537.7029 (1.0) 0.0000 (1.0) 537.7029 (1.0) 0.0000 (1.0) 0;0 1.8598 (1.0) 1 1
test_sharing-pull-noop[main] (0001_test_sh) 706.6731 (1.31) 706.6731 (1.31) 706.6731 (1.31) 0.0000 (1.0) 706.6731 (1.31) 0.0000 (1.0) 0;0 1.4151 (0.76) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_sharing-push': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.10.2] (0001_test_sh) 5.9385 (1.26) 5.9385 (1.26) 5.9385 (1.26) 0.0000 (1.0) 5.9385 (1.26) 0.0000 (1.0) 0;0 0.1684 (0.80) 1 1
test_sharing-push[2.11.0] (0001_test_sh) 6.0133 (1.27) 6.0133 (1.27) 6.0133 (1.27) 0.0000 (1.0) 6.0133 (1.27) 0.0000 (1.0) 0;0 0.1663 (0.79) 1 1
test_sharing-push[2.13.0] (0001_test_sh) 5.5845 (1.18) 5.5845 (1.18) 5.5845 (1.18) 0.0000 (1.0) 5.5845 (1.18) 0.0000 (1.0) 0;0 0.1791 (0.85) 1 1
test_sharing-push[2.6.3] (0001_test_sh) 4.7283 (1.0) 4.7283 (1.0) 4.7283 (1.0) 0.0000 (1.0) 4.7283 (1.0) 0.0000 (1.0) 0;0 0.2115 (1.0) 1 1
test_sharing-push[main] (0001_test_sh) 5.6979 (1.21) 5.6979 (1.21) 5.6979 (1.21) 0.0000 (1.0) 5.6979 (1.21) 0.0000 (1.0) 0;0 0.1755 (0.83) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.10.2] (0001_test_sh) 537.7718 (1.12) 537.7718 (1.12) 537.7718 (1.12) 0.0000 (1.0) 537.7718 (1.12) 0.0000 (1.0) 0;0 1.8595 (0.90) 1 1
test_sharing-push-noop[2.11.0] (0001_test_sh) 526.1463 (1.09) 526.1463 (1.09) 526.1463 (1.09) 0.0000 (1.0) 526.1463 (1.09) 0.0000 (1.0) 0;0 1.9006 (0.92) 1 1
test_sharing-push-noop[2.13.0] (0001_test_sh) 541.4283 (1.12) 541.4283 (1.12) 541.4283 (1.12) 0.0000 (1.0) 541.4283 (1.12) 0.0000 (1.0) 0;0 1.8470 (0.89) 1 1
test_sharing-push-noop[2.6.3] (0001_test_sh) 482.1875 (1.0) 482.1875 (1.0) 482.1875 (1.0) 0.0000 (1.0) 482.1875 (1.0) 0.0000 (1.0) 0;0 2.0739 (1.0) 1 1
test_sharing-push-noop[main] (0001_test_sh) 534.1535 (1.11) 534.1535 (1.11) 534.1535 (1.11) 0.0000 (1.0) 534.1535 (1.11) 0.0000 (1.0) 0;0 1.8721 (0.90) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_status-status': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.10.2] (0001_test_st) 2.4570 (1.72) 2.4570 (1.72) 2.4570 (1.72) 0.0000 (1.0) 2.4570 (1.72) 0.0000 (1.0) 0;0 0.4070 (0.58) 1 1
test_status-status[2.11.0] (0001_test_st) 2.3561 (1.65) 2.3561 (1.65) 2.3561 (1.65) 0.0000 (1.0) 2.3561 (1.65) 0.0000 (1.0) 0;0 0.4244 (0.61) 1 1
test_status-status[2.13.0] (0001_test_st) 1.4453 (1.01) 1.4453 (1.01) 1.4453 (1.01) 0.0000 (1.0) 1.4453 (1.01) 0.0000 (1.0) 0;0 0.6919 (0.99) 1 1
test_status-status[2.6.3] (0001_test_st) 3.7919 (2.65) 3.7919 (2.65) 3.7919 (2.65) 0.0000 (1.0) 3.7919 (2.65) 0.0000 (1.0) 0;0 0.2637 (0.38) 1 1
test_status-status[main] (0001_test_st) 1.4294 (1.0) 1.4294 (1.0) 1.4294 (1.0) 0.0000 (1.0) 1.4294 (1.0) 0.0000 (1.0) 0;0 0.6996 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_status-status-changed': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.10.2] (0001_test_st) 1.6377 (2.99) 1.6377 (2.99) 1.6377 (2.99) 0.0000 (1.0) 1.6377 (2.99) 0.0000 (1.0) 0;0 0.6106 (0.33) 1 1
test_status-status-changed[2.11.0] (0001_test_st) 1.6325 (2.98) 1.6325 (2.98) 1.6325 (2.98) 0.0000 (1.0) 1.6325 (2.98) 0.0000 (1.0) 0;0 0.6126 (0.34) 1 1
test_status-status-changed[2.13.0] (0001_test_st) 0.5485 (1.0) 0.5485 (1.0) 0.5485 (1.0) 0.0000 (1.0) 0.5485 (1.0) 0.0000 (1.0) 0;0 1.8231 (1.0) 1 1
test_status-status-changed[2.6.3] (0001_test_st) 2.8504 (5.20) 2.8504 (5.20) 2.8504 (5.20) 0.0000 (1.0) 2.8504 (5.20) 0.0000 (1.0) 0;0 0.3508 (0.19) 1 1
test_status-status-changed[main] (0001_test_st) 0.5516 (1.01) 0.5516 (1.01) 0.5516 (1.01) 0.0000 (1.0) 0.5516 (1.01) 0.0000 (1.0) 0;0 1.8130 (0.99) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.10.2] (0001_test_st) 1.6417 (3.09) 1.6417 (3.09) 1.6417 (3.09) 0.0000 (1.0) 1.6417 (3.09) 0.0000 (1.0) 0;0 0.6091 (0.32) 1 1
test_status-status-changed-noop[2.11.0] (0001_test_st) 0.6640 (1.25) 0.6640 (1.25) 0.6640 (1.25) 0.0000 (1.0) 0.6640 (1.25) 0.0000 (1.0) 0;0 1.5059 (0.80) 1 1
test_status-status-changed-noop[2.13.0] (0001_test_st) 0.5370 (1.01) 0.5370 (1.01) 0.5370 (1.01) 0.0000 (1.0) 0.5370 (1.01) 0.0000 (1.0) 0;0 1.8621 (0.99) 1 1
test_status-status-changed-noop[2.6.3] (0001_test_st) 2.7950 (5.25) 2.7950 (5.25) 2.7950 (5.25) 0.0000 (1.0) 2.7950 (5.25) 0.0000 (1.0) 0;0 0.3578 (0.19) 1 1
test_status-status-changed-noop[main] (0001_test_st) 0.5319 (1.0) 0.5319 (1.0) 0.5319 (1.0) 0.0000 (1.0) 0.5319 (1.0) 0.0000 (1.0) 0;0 1.8801 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_status-status-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.10.2] (0001_test_st) 480.2292 (1.04) 480.2292 (1.04) 480.2292 (1.04) 0.0000 (1.0) 480.2292 (1.04) 0.0000 (1.0) 0;0 2.0823 (0.96) 1 1
test_status-status-noop[2.11.0] (0001_test_st) 467.4259 (1.01) 467.4259 (1.01) 467.4259 (1.01) 0.0000 (1.0) 467.4259 (1.01) 0.0000 (1.0) 0;0 2.1394 (0.99) 1 1
test_status-status-noop[2.13.0] (0001_test_st) 530.9804 (1.15) 530.9804 (1.15) 530.9804 (1.15) 0.0000 (1.0) 530.9804 (1.15) 0.0000 (1.0) 0;0 1.8833 (0.87) 1 1
test_status-status-noop[2.6.3] (0001_test_st) 461.8229 (1.0) 461.8229 (1.0) 461.8229 (1.0) 0.0000 (1.0) 461.8229 (1.0) 0.0000 (1.0) 0;0 2.1653 (1.0) 1 1
test_status-status-noop[main] (0001_test_st) 528.1332 (1.14) 528.1332 (1.14) 528.1332 (1.14) 0.0000 (1.0) 528.1332 (1.14) 0.0000 (1.0) 0;0 1.8935 (0.87) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```
efiop commented 2 years ago

Thank you!