search

iterative / dvc-bench

Benchmarks for DVC
http://bench.dvc.org/
Apache License 2.0
20 stars 10 forks source link

plots: hacky plots #355

Closed dberenbaum closed 2 years ago

dberenbaum commented 2 years ago

Initial PR to add some basic plots. See the results in Studio: https://studio.iterative.ai/user/dberenbaum/views/dvc-bench-v8wdyctd8n.

dberenbaum commented 2 years ago

We probably want to cache the csv files in a dvc remote even though they are small.

github-actions[bot] commented 2 years ago 

dataset: mnist
project: example-get-started

----------------------------------------------------------------------------------- benchmark 'test_add-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.10.2] (0001_test_ad) 260.9161 (1.09) 260.9161 (1.09) 260.9161 (1.09) 0.0000 (1.0) 260.9161 (1.09) 0.0000 (1.0) 0;0 0.0038 (0.92) 1 1
test_add-add[2.11.0] (0001_test_ad) 260.3609 (1.09) 260.3609 (1.09) 260.3609 (1.09) 0.0000 (1.0) 260.3609 (1.09) 0.0000 (1.0) 0;0 0.0038 (0.92) 1 1
test_add-add[2.6.3] (0001_test_ad) 309.6566 (1.30) 309.6566 (1.30) 309.6566 (1.30) 0.0000 (1.0) 309.6566 (1.30) 0.0000 (1.0) 0;0 0.0032 (0.77) 1 1
test_add-add[2.9.5] (0001_test_ad) 320.1123 (1.34) 320.1123 (1.34) 320.1123 (1.34) 0.0000 (1.0) 320.1123 (1.34) 0.0000 (1.0) 0;0 0.0031 (0.75) 1 1
test_add-add[main] (0001_test_ad) 239.0331 (1.0) 239.0331 (1.0) 239.0331 (1.0) 0.0000 (1.0) 239.0331 (1.0) 0.0000 (1.0) 0;0 0.0042 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout[2.10.2] (0001_test_ch) 84.9488 (1.40) 84.9488 (1.40) 84.9488 (1.40) 0.0000 (1.0) 84.9488 (1.40) 0.0000 (1.0) 0;0 0.0118 (0.72) 1 1
test_checkout-checkout[2.11.0] (0001_test_ch) 83.4990 (1.37) 83.4990 (1.37) 83.4990 (1.37) 0.0000 (1.0) 83.4990 (1.37) 0.0000 (1.0) 0;0 0.0120 (0.73) 1 1
test_checkout-checkout[2.6.3] (0001_test_ch) 60.7446 (1.0) 60.7446 (1.0) 60.7446 (1.0) 0.0000 (1.0) 60.7446 (1.0) 0.0000 (1.0) 0;0 0.0165 (1.0) 1 1
test_checkout-checkout[2.9.5] (0001_test_ch) 95.9669 (1.58) 95.9669 (1.58) 95.9669 (1.58) 0.0000 (1.0) 95.9669 (1.58) 0.0000 (1.0) 0;0 0.0104 (0.63) 1 1
test_checkout-checkout[main] (0001_test_ch) 86.1466 (1.42) 86.1466 (1.42) 86.1466 (1.42) 0.0000 (1.0) 86.1466 (1.42) 0.0000 (1.0) 0;0 0.0116 (0.71) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop[2.10.2] (0001_test_ch) 5.2942 (2.10) 5.2942 (2.10) 5.2942 (2.10) 0.0000 (1.0) 5.2942 (2.10) 0.0000 (1.0) 0;0 0.1889 (0.48) 1 1
test_checkout-checkout-noop[2.11.0] (0001_test_ch) 5.2564 (2.08) 5.2564 (2.08) 5.2564 (2.08) 0.0000 (1.0) 5.2564 (2.08) 0.0000 (1.0) 0;0 0.1902 (0.48) 1 1
test_checkout-checkout-noop[2.6.3] (0001_test_ch) 2.5244 (1.0) 2.5244 (1.0) 2.5244 (1.0) 0.0000 (1.0) 2.5244 (1.0) 0.0000 (1.0) 0;0 0.3961 (1.0) 1 1
test_checkout-checkout-noop[2.9.5] (0001_test_ch) 5.2649 (2.09) 5.2649 (2.09) 5.2649 (2.09) 0.0000 (1.0) 5.2649 (2.09) 0.0000 (1.0) 0;0 0.1899 (0.48) 1 1
test_checkout-checkout-noop[main] (0001_test_ch) 7.3826 (2.92) 7.3826 (2.92) 7.3826 (2.92) 0.0000 (1.0) 7.3826 (2.92) 0.0000 (1.0) 0;0 0.1355 (0.34) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-update': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update[2.10.2] (0001_test_ch) 34.2454 (3.86) 34.2454 (3.86) 34.2454 (3.86) 0.0000 (1.0) 34.2454 (3.86) 0.0000 (1.0) 0;0 0.0292 (0.26) 1 1
test_checkout-checkout-update[2.11.0] (0001_test_ch) 33.4273 (3.77) 33.4273 (3.77) 33.4273 (3.77) 0.0000 (1.0) 33.4273 (3.77) 0.0000 (1.0) 0;0 0.0299 (0.27) 1 1
test_checkout-checkout-update[2.6.3] (0001_test_ch) 86.3572 (9.73) 86.3572 (9.73) 86.3572 (9.73) 0.0000 (1.0) 86.3572 (9.73) 0.0000 (1.0) 0;0 0.0116 (0.10) 1 1
test_checkout-checkout-update[2.9.5] (0001_test_ch) 34.0236 (3.83) 34.0236 (3.83) 34.0236 (3.83) 0.0000 (1.0) 34.0236 (3.83) 0.0000 (1.0) 0;0 0.0294 (0.26) 1 1
test_checkout-checkout-update[main] (0001_test_ch) 8.8760 (1.0) 8.8760 (1.0) 8.8760 (1.0) 0.0000 (1.0) 8.8760 (1.0) 0.0000 (1.0) 0;0 0.1127 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_diff-diff': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.10.2] (0001_test_di) 62.7207 (1.63) 62.7207 (1.63) 62.7207 (1.63) 0.0000 (1.0) 62.7207 (1.63) 0.0000 (1.0) 0;0 0.0159 (0.61) 1 1
test_diff-diff[2.11.0] (0001_test_di) 55.4616 (1.45) 55.4616 (1.45) 55.4616 (1.45) 0.0000 (1.0) 55.4616 (1.45) 0.0000 (1.0) 0;0 0.0180 (0.69) 1 1
test_diff-diff[2.6.3] (0001_test_di) 114.5233 (2.98) 114.5233 (2.98) 114.5233 (2.98) 0.0000 (1.0) 114.5233 (2.98) 0.0000 (1.0) 0;0 0.0087 (0.34) 1 1
test_diff-diff[2.9.5] (0001_test_di) 60.8565 (1.59) 60.8565 (1.59) 60.8565 (1.59) 0.0000 (1.0) 60.8565 (1.59) 0.0000 (1.0) 0;0 0.0164 (0.63) 1 1
test_diff-diff[main] (0001_test_di) 38.3708 (1.0) 38.3708 (1.0) 38.3708 (1.0) 0.0000 (1.0) 38.3708 (1.0) 0.0000 (1.0) 0;0 0.0261 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.10.2] (0001_test_di) 35.5687 (4.29) 35.5687 (4.29) 35.5687 (4.29) 0.0000 (1.0) 35.5687 (4.29) 0.0000 (1.0) 0;0 0.0281 (0.23) 1 1
test_diff-diff-changed[2.11.0] (0001_test_di) 34.6231 (4.17) 34.6231 (4.17) 34.6231 (4.17) 0.0000 (1.0) 34.6231 (4.17) 0.0000 (1.0) 0;0 0.0289 (0.24) 1 1
test_diff-diff-changed[2.6.3] (0001_test_di) 85.1530 (10.26) 85.1530 (10.26) 85.1530 (10.26) 0.0000 (1.0) 85.1530 (10.26) 0.0000 (1.0) 0;0 0.0117 (0.10) 1 1
test_diff-diff-changed[2.9.5] (0001_test_di) 35.0990 (4.23) 35.0990 (4.23) 35.0990 (4.23) 0.0000 (1.0) 35.0990 (4.23) 0.0000 (1.0) 0;0 0.0285 (0.24) 1 1
test_diff-diff-changed[main] (0001_test_di) 8.3000 (1.0) 8.3000 (1.0) 8.3000 (1.0) 0.0000 (1.0) 8.3000 (1.0) 0.0000 (1.0) 0;0 0.1205 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.10.2] (0001_test_di) 36.0183 (11.66) 36.0183 (11.66) 36.0183 (11.66) 0.0000 (1.0) 36.0183 (11.66) 0.0000 (1.0) 0;0 0.0278 (0.09) 1 1
test_diff-diff-changed-noop[2.11.0] (0001_test_di) 3.0888 (1.0) 3.0888 (1.0) 3.0888 (1.0) 0.0000 (1.0) 3.0888 (1.0) 0.0000 (1.0) 0;0 0.3238 (1.0) 1 1
test_diff-diff-changed-noop[2.6.3] (0001_test_di) 85.8605 (27.80) 85.8605 (27.80) 85.8605 (27.80) 0.0000 (1.0) 85.8605 (27.80) 0.0000 (1.0) 0;0 0.0116 (0.04) 1 1
test_diff-diff-changed-noop[2.9.5] (0001_test_di) 35.2891 (11.42) 35.2891 (11.42) 35.2891 (11.42) 0.0000 (1.0) 35.2891 (11.42) 0.0000 (1.0) 0;0 0.0283 (0.09) 1 1
test_diff-diff-changed-noop[main] (0001_test_di) 8.3667 (2.71) 8.3667 (2.71) 8.3667 (2.71) 0.0000 (1.0) 8.3667 (2.71) 0.0000 (1.0) 0;0 0.1195 (0.37) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.10.2] (0001_test_di) 3.7761 (1.06) 3.7761 (1.06) 3.7761 (1.06) 0.0000 (1.0) 3.7761 (1.06) 0.0000 (1.0) 0;0 0.2648 (0.94) 1 1
test_diff-diff-noop[2.11.0] (0001_test_di) 3.5622 (1.0) 3.5622 (1.0) 3.5622 (1.0) 0.0000 (1.0) 3.5622 (1.0) 0.0000 (1.0) 0;0 0.2807 (1.0) 1 1
test_diff-diff-noop[2.6.3] (0001_test_di) 57.6776 (16.19) 57.6776 (16.19) 57.6776 (16.19) 0.0000 (1.0) 57.6776 (16.19) 0.0000 (1.0) 0;0 0.0173 (0.06) 1 1
test_diff-diff-noop[2.9.5] (0001_test_di) 3.6818 (1.03) 3.6818 (1.03) 3.6818 (1.03) 0.0000 (1.0) 3.6818 (1.03) 0.0000 (1.0) 0;0 0.2716 (0.97) 1 1
test_diff-diff-noop[main] (0001_test_di) 8.3960 (2.36) 8.3960 (2.36) 8.3960 (2.36) 0.0000 (1.0) 8.3960 (2.36) 0.0000 (1.0) 0;0 0.1191 (0.42) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 3 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.10.2] (0001_test_ex) 1.3303 (1.03) 1.3303 (1.03) 1.3303 (1.03) 0.0000 (1.0) 1.3303 (1.03) 0.0000 (1.0) 0;0 0.7517 (0.97) 1 1
test_exp_show-exp[2.11.0] (0001_test_ex) 1.2942 (1.0) 1.2942 (1.0) 1.2942 (1.0) 0.0000 (1.0) 1.2942 (1.0) 0.0000 (1.0) 0;0 0.7727 (1.0) 1 1
test_exp_show-exp[main] (0001_test_ex) 1.3311 (1.03) 1.3311 (1.03) 1.3311 (1.03) 0.0000 (1.0) 1.3311 (1.03) 0.0000 (1.0) 0;0 0.7513 (0.97) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_gc-gc': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.10.2] (0001_test_gc) 2.8938 (1.21) 2.8938 (1.21) 2.8938 (1.21) 0.0000 (1.0) 2.8938 (1.21) 0.0000 (1.0) 0;0 0.3456 (0.83) 1 1
test_gc-gc[2.11.0] (0001_test_gc) 2.3883 (1.0) 2.3883 (1.0) 2.3883 (1.0) 0.0000 (1.0) 2.3883 (1.0) 0.0000 (1.0) 0;0 0.4187 (1.0) 1 1
test_gc-gc[2.6.3] (0001_test_gc) 3.4441 (1.44) 3.4441 (1.44) 3.4441 (1.44) 0.0000 (1.0) 3.4441 (1.44) 0.0000 (1.0) 0;0 0.2904 (0.69) 1 1
test_gc-gc[2.9.5] (0001_test_gc) 2.9180 (1.22) 2.9180 (1.22) 2.9180 (1.22) 0.0000 (1.0) 2.9180 (1.22) 0.0000 (1.0) 0;0 0.3427 (0.82) 1 1
test_gc-gc[main] (0001_test_gc) 2.6683 (1.12) 2.6683 (1.12) 2.6683 (1.12) 0.0000 (1.0) 2.6683 (1.12) 0.0000 (1.0) 0;0 0.3748 (0.90) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_get-get': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.10.2] (0001_test_ge) 195.1101 (1.99) 195.1101 (1.99) 195.1101 (1.99) 0.0000 (1.0) 195.1101 (1.99) 0.0000 (1.0) 0;0 0.0051 (0.50) 1 1
test_get-get[2.11.0] (0001_test_ge) 160.9436 (1.64) 160.9436 (1.64) 160.9436 (1.64) 0.0000 (1.0) 160.9436 (1.64) 0.0000 (1.0) 0;0 0.0062 (0.61) 1 1
test_get-get[2.6.3] (0001_test_ge) 117.3716 (1.20) 117.3716 (1.20) 117.3716 (1.20) 0.0000 (1.0) 117.3716 (1.20) 0.0000 (1.0) 0;0 0.0085 (0.83) 1 1
test_get-get[2.9.5] (0001_test_ge) 97.9051 (1.0) 97.9051 (1.0) 97.9051 (1.0) 0.0000 (1.0) 97.9051 (1.0) 0.0000 (1.0) 0;0 0.0102 (1.0) 1 1
test_get-get[main] (0001_test_ge) 159.2082 (1.63) 159.2082 (1.63) 159.2082 (1.63) 0.0000 (1.0) 159.2082 (1.63) 0.0000 (1.0) 0;0 0.0063 (0.61) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_help---help': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.10.2] (0001_test_he) 172.7645 (1.01) 200.9464 (1.05) 178.8547 (1.00) 5.4151 (1.34) 177.4281 (1.00) 4.9868 (1.07) 19;6 5.5911 (1.00) 100 1
test_help---help[2.11.0] (0001_test_he) 171.8950 (1.0) 190.6721 (1.0) 178.0697 (1.00) 4.0365 (1.0) 177.2370 (1.00) 4.6434 (1.0) 28;6 5.6158 (1.00) 100 1
test_help---help[2.6.3] (0001_test_he) 225.3638 (1.31) 252.2038 (1.32) 233.3315 (1.31) 6.4702 (1.60) 232.1169 (1.31) 7.9961 (1.72) 31;5 4.2857 (0.76) 100 1
test_help---help[2.9.5] (0001_test_he) 173.3505 (1.01) 215.1173 (1.13) 183.2729 (1.03) 7.5826 (1.88) 181.0023 (1.02) 9.9751 (2.15) 25;1 5.4563 (0.97) 100 1
test_help---help[main] (0001_test_he) 172.0619 (1.00) 191.5935 (1.00) 177.9917 (1.0) 4.4378 (1.10) 176.9878 (1.0) 5.0370 (1.08) 22;5 5.6182 (1.0) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_import-import': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.10.2] (0001_test_im) 430.1419 (1.41) 430.1419 (1.41) 430.1419 (1.41) 0.0000 (1.0) 430.1419 (1.41) 0.0000 (1.0) 0;0 0.0023 (0.71) 1 1
test_import-import[2.11.0] (0001_test_im) 386.6931 (1.26) 386.6931 (1.26) 386.6931 (1.26) 0.0000 (1.0) 386.6931 (1.26) 0.0000 (1.0) 0;0 0.0026 (0.79) 1 1
test_import-import[2.6.3] (0001_test_im) 306.0718 (1.0) 306.0718 (1.0) 306.0718 (1.0) 0.0000 (1.0) 306.0718 (1.0) 0.0000 (1.0) 0;0 0.0033 (1.0) 1 1
test_import-import[2.9.5] (0001_test_im) 445.8613 (1.46) 445.8613 (1.46) 445.8613 (1.46) 0.0000 (1.0) 445.8613 (1.46) 0.0000 (1.0) 0;0 0.0022 (0.69) 1 1
test_import-import[main] (0001_test_im) 367.3697 (1.20) 367.3697 (1.20) 367.3697 (1.20) 0.0000 (1.0) 367.3697 (1.20) 0.0000 (1.0) 0;0 0.0027 (0.83) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_init-init': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.10.2] (0001_test_in) 324.0802 (1.06) 345.9626 (1.06) 333.5680 (1.07) 4.1041 (1.23) 332.9397 (1.07) 5.4226 (1.18) 29;2 2.9979 (0.94) 100 1
test_init-init[2.11.0] (0001_test_in) 331.2489 (1.08) 346.3462 (1.06) 337.2924 (1.08) 3.3405 (1.0) 337.2056 (1.08) 4.9026 (1.06) 37;0 2.9648 (0.93) 100 1
test_init-init[2.6.3] (0001_test_in) 306.5157 (1.0) 325.2501 (1.0) 313.1967 (1.0) 3.6010 (1.08) 312.5353 (1.0) 4.6081 (1.0) 25;4 3.1929 (1.0) 100 1
test_init-init[2.9.5] (0001_test_in) 332.3894 (1.08) 455.1599 (1.40) 340.6001 (1.09) 13.8448 (4.14) 338.3053 (1.08) 5.0727 (1.10) 2;2 2.9360 (0.92) 100 1
test_init-init[main] (0001_test_in) 330.7010 (1.08) 356.0406 (1.09) 338.2793 (1.08) 3.9663 (1.19) 338.2196 (1.08) 4.7925 (1.04) 28;1 2.9561 (0.93) 100 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_list-list': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.10.2] (0001_test_li) 2.3098 (4.06) 2.3098 (4.06) 2.3098 (4.06) 0.0000 (1.0) 2.3098 (4.06) 0.0000 (1.0) 0;0 0.4329 (0.25) 1 1
test_list-list[2.11.0] (0001_test_li) 2.2990 (4.04) 2.2990 (4.04) 2.2990 (4.04) 0.0000 (1.0) 2.2990 (4.04) 0.0000 (1.0) 0;0 0.4350 (0.25) 1 1
test_list-list[2.6.3] (0001_test_li) 0.5695 (1.0) 0.5695 (1.0) 0.5695 (1.0) 0.0000 (1.0) 0.5695 (1.0) 0.0000 (1.0) 0;0 1.7560 (1.0) 1 1
test_list-list[2.9.5] (0001_test_li) 2.5273 (4.44) 2.5273 (4.44) 2.5273 (4.44) 0.0000 (1.0) 2.5273 (4.44) 0.0000 (1.0) 0;0 0.3957 (0.23) 1 1
test_list-list[main] (0001_test_li) 2.2909 (4.02) 2.2909 (4.02) 2.2909 (4.02) 0.0000 (1.0) 2.2909 (4.02) 0.0000 (1.0) 0;0 0.4365 (0.25) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_list-list-dvc-only': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.10.2] (0001_test_li) 2.2694 (4.93) 2.2694 (4.93) 2.2694 (4.93) 0.0000 (1.0) 2.2694 (4.93) 0.0000 (1.0) 0;0 0.4406 (0.20) 1 1
test_list-list-dvc-only[2.11.0] (0001_test_li) 2.2962 (4.99) 2.2962 (4.99) 2.2962 (4.99) 0.0000 (1.0) 2.2962 (4.99) 0.0000 (1.0) 0;0 0.4355 (0.20) 1 1
test_list-list-dvc-only[2.6.3] (0001_test_li) 0.4603 (1.0) 0.4603 (1.0) 0.4603 (1.0) 0.0000 (1.0) 0.4603 (1.0) 0.0000 (1.0) 0;0 2.1726 (1.0) 1 1
test_list-list-dvc-only[2.9.5] (0001_test_li) 2.4798 (5.39) 2.4798 (5.39) 2.4798 (5.39) 0.0000 (1.0) 2.4798 (5.39) 0.0000 (1.0) 0;0 0.4033 (0.19) 1 1
test_list-list-dvc-only[main] (0001_test_li) 2.3024 (5.00) 2.3024 (5.00) 2.3024 (5.00) 0.0000 (1.0) 2.3024 (5.00) 0.0000 (1.0) 0;0 0.4343 (0.20) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_plots_show-plots': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots_show-plots[2.10.2] (0001_test_pl) 801.8697 (1.37) 801.8697 (1.37) 801.8697 (1.37) 0.0000 (1.0) 801.8697 (1.37) 0.0000 (1.0) 0;0 1.2471 (0.73) 1 1
test_plots_show-plots[2.11.0] (0001_test_pl) 1,213.5939 (2.07) 1,213.5939 (2.07) 1,213.5939 (2.07) 0.0000 (1.0) 1,213.5939 (2.07) 0.0000 (1.0) 0;0 0.8240 (0.48) 1 1
test_plots_show-plots[2.6.3] (0001_test_pl) 585.7536 (1.0) 585.7536 (1.0) 585.7536 (1.0) 0.0000 (1.0) 585.7536 (1.0) 0.0000 (1.0) 0;0 1.7072 (1.0) 1 1
test_plots_show-plots[2.9.5] (0001_test_pl) 1,325.3269 (2.26) 1,325.3269 (2.26) 1,325.3269 (2.26) 0.0000 (1.0) 1,325.3269 (2.26) 0.0000 (1.0) 0;0 0.7545 (0.44) 1 1
test_plots_show-plots[main] (0001_test_pl) 1,646.1306 (2.81) 1,646.1306 (2.81) 1,646.1306 (2.81) 0.0000 (1.0) 1,646.1306 (2.81) 0.0000 (1.0) 0;0 0.6075 (0.36) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_pull-pull': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.10.2] (0001_test_pu) 237.6018 (1.28) 237.6018 (1.28) 237.6018 (1.28) 0.0000 (1.0) 237.6018 (1.28) 0.0000 (1.0) 0;0 0.0042 (0.78) 1 1
test_pull-pull[2.11.0] (0001_test_pu) 240.3375 (1.29) 240.3375 (1.29) 240.3375 (1.29) 0.0000 (1.0) 240.3375 (1.29) 0.0000 (1.0) 0;0 0.0042 (0.77) 1 1
test_pull-pull[2.6.3] (0001_test_pu) 185.7631 (1.0) 185.7631 (1.0) 185.7631 (1.0) 0.0000 (1.0) 185.7631 (1.0) 0.0000 (1.0) 0;0 0.0054 (1.0) 1 1
test_pull-pull[2.9.5] (0001_test_pu) 274.8929 (1.48) 274.8929 (1.48) 274.8929 (1.48) 0.0000 (1.0) 274.8929 (1.48) 0.0000 (1.0) 0;0 0.0036 (0.68) 1 1
test_pull-pull[main] (0001_test_pu) 237.6174 (1.28) 237.6174 (1.28) 237.6174 (1.28) 0.0000 (1.0) 237.6174 (1.28) 0.0000 (1.0) 0;0 0.0042 (0.78) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_push-push': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.10.2] (0001_test_pu) 130.7279 (1.25) 130.7279 (1.25) 130.7279 (1.25) 0.0000 (1.0) 130.7279 (1.25) 0.0000 (1.0) 0;0 0.0076 (0.80) 1 1
test_push-push[2.11.0] (0001_test_pu) 132.9340 (1.27) 132.9340 (1.27) 132.9340 (1.27) 0.0000 (1.0) 132.9340 (1.27) 0.0000 (1.0) 0;0 0.0075 (0.79) 1 1
test_push-push[2.6.3] (0001_test_pu) 104.8414 (1.0) 104.8414 (1.0) 104.8414 (1.0) 0.0000 (1.0) 104.8414 (1.0) 0.0000 (1.0) 0;0 0.0095 (1.0) 1 1
test_push-push[2.9.5] (0001_test_pu) 146.6534 (1.40) 146.6534 (1.40) 146.6534 (1.40) 0.0000 (1.0) 146.6534 (1.40) 0.0000 (1.0) 0;0 0.0068 (0.71) 1 1
test_push-push[main] (0001_test_pu) 130.1872 (1.24) 130.1872 (1.24) 130.1872 (1.24) 0.0000 (1.0) 130.1872 (1.24) 0.0000 (1.0) 0;0 0.0077 (0.81) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.10.2] (0001_test_sh) 210.8475 (1.12) 210.8475 (1.12) 210.8475 (1.12) 0.0000 (1.0) 210.8475 (1.12) 0.0000 (1.0) 0;0 0.0047 (0.90) 1 1
test_sharing-add[2.11.0] (0001_test_sh) 212.7079 (1.12) 212.7079 (1.12) 212.7079 (1.12) 0.0000 (1.0) 212.7079 (1.12) 0.0000 (1.0) 0;0 0.0047 (0.89) 1 1
test_sharing-add[2.6.3] (0001_test_sh) 243.2342 (1.29) 243.2342 (1.29) 243.2342 (1.29) 0.0000 (1.0) 243.2342 (1.29) 0.0000 (1.0) 0;0 0.0041 (0.78) 1 1
test_sharing-add[2.9.5] (0001_test_sh) 260.3807 (1.38) 260.3807 (1.38) 260.3807 (1.38) 0.0000 (1.0) 260.3807 (1.38) 0.0000 (1.0) 0;0 0.0038 (0.73) 1 1
test_sharing-add[main] (0001_test_sh) 189.0888 (1.0) 189.0888 (1.0) 189.0888 (1.0) 0.0000 (1.0) 189.0888 (1.0) 0.0000 (1.0) 0;0 0.0053 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.10.2] (0001_test_sh) 105.8532 (6.12) 105.8532 (6.12) 105.8532 (6.12) 0.0000 (1.0) 105.8532 (6.12) 0.0000 (1.0) 0;0 0.0094 (0.16) 1 1
test_sharing-add-noop[2.11.0] (0001_test_sh) 103.1437 (5.97) 103.1437 (5.97) 103.1437 (5.97) 0.0000 (1.0) 103.1437 (5.97) 0.0000 (1.0) 0;0 0.0097 (0.17) 1 1
test_sharing-add-noop[2.6.3] (0001_test_sh) 17.2875 (1.0) 17.2875 (1.0) 17.2875 (1.0) 0.0000 (1.0) 17.2875 (1.0) 0.0000 (1.0) 0;0 0.0578 (1.0) 1 1
test_sharing-add-noop[2.9.5] (0001_test_sh) 119.7008 (6.92) 119.7008 (6.92) 119.7008 (6.92) 0.0000 (1.0) 119.7008 (6.92) 0.0000 (1.0) 0;0 0.0084 (0.14) 1 1
test_sharing-add-noop[main] (0001_test_sh) 116.9745 (6.77) 116.9745 (6.77) 116.9745 (6.77) 0.0000 (1.0) 116.9745 (6.77) 0.0000 (1.0) 0;0 0.0085 (0.15) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.10.2] (0001_test_sh) 5.3045 (2.10) 5.3045 (2.10) 5.3045 (2.10) 0.0000 (1.0) 5.3045 (2.10) 0.0000 (1.0) 0;0 0.1885 (0.48) 1 1
test_sharing-checkout-noop[2.11.0] (0001_test_sh) 5.2615 (2.08) 5.2615 (2.08) 5.2615 (2.08) 0.0000 (1.0) 5.2615 (2.08) 0.0000 (1.0) 0;0 0.1901 (0.48) 1 1
test_sharing-checkout-noop[2.6.3] (0001_test_sh) 2.5272 (1.0) 2.5272 (1.0) 2.5272 (1.0) 0.0000 (1.0) 2.5272 (1.0) 0.0000 (1.0) 0;0 0.3957 (1.0) 1 1
test_sharing-checkout-noop[2.9.5] (0001_test_sh) 5.3037 (2.10) 5.3037 (2.10) 5.3037 (2.10) 0.0000 (1.0) 5.3037 (2.10) 0.0000 (1.0) 0;0 0.1885 (0.48) 1 1
test_sharing-checkout-noop[main] (0001_test_sh) 7.3933 (2.93) 7.3933 (2.93) 7.3933 (2.93) 0.0000 (1.0) 7.3933 (2.93) 0.0000 (1.0) 0;0 0.1353 (0.34) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.10.2] (0001_test_sh) 242.6524 (1.27) 242.6524 (1.27) 242.6524 (1.27) 0.0000 (1.0) 242.6524 (1.27) 0.0000 (1.0) 0;0 0.0041 (0.79) 1 1
test_sharing-pull[2.11.0] (0001_test_sh) 241.8670 (1.27) 241.8670 (1.27) 241.8670 (1.27) 0.0000 (1.0) 241.8670 (1.27) 0.0000 (1.0) 0;0 0.0041 (0.79) 1 1
test_sharing-pull[2.6.3] (0001_test_sh) 190.5181 (1.0) 190.5181 (1.0) 190.5181 (1.0) 0.0000 (1.0) 190.5181 (1.0) 0.0000 (1.0) 0;0 0.0052 (1.0) 1 1
test_sharing-pull[2.9.5] (0001_test_sh) 274.8589 (1.44) 274.8589 (1.44) 274.8589 (1.44) 0.0000 (1.0) 274.8589 (1.44) 0.0000 (1.0) 0;0 0.0036 (0.69) 1 1
test_sharing-pull[main] (0001_test_sh) 245.0479 (1.29) 245.0479 (1.29) 245.0479 (1.29) 0.0000 (1.0) 245.0479 (1.29) 0.0000 (1.0) 0;0 0.0041 (0.78) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.10.2] (0001_test_sh) 6.1537 (1.82) 6.1537 (1.82) 6.1537 (1.82) 0.0000 (1.0) 6.1537 (1.82) 0.0000 (1.0) 0;0 0.1625 (0.55) 1 1
test_sharing-pull-noop[2.11.0] (0001_test_sh) 6.0604 (1.79) 6.0604 (1.79) 6.0604 (1.79) 0.0000 (1.0) 6.0604 (1.79) 0.0000 (1.0) 0;0 0.1650 (0.56) 1 1
test_sharing-pull-noop[2.6.3] (0001_test_sh) 3.3900 (1.0) 3.3900 (1.0) 3.3900 (1.0) 0.0000 (1.0) 3.3900 (1.0) 0.0000 (1.0) 0;0 0.2950 (1.0) 1 1
test_sharing-pull-noop[2.9.5] (0001_test_sh) 6.0708 (1.79) 6.0708 (1.79) 6.0708 (1.79) 0.0000 (1.0) 6.0708 (1.79) 0.0000 (1.0) 0;0 0.1647 (0.56) 1 1
test_sharing-pull-noop[main] (0001_test_sh) 8.4075 (2.48) 8.4075 (2.48) 8.4075 (2.48) 0.0000 (1.0) 8.4075 (2.48) 0.0000 (1.0) 0;0 0.1189 (0.40) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.10.2] (0001_test_sh) 131.7985 (1.25) 131.7985 (1.25) 131.7985 (1.25) 0.0000 (1.0) 131.7985 (1.25) 0.0000 (1.0) 0;0 0.0076 (0.80) 1 1
test_sharing-push[2.11.0] (0001_test_sh) 135.4953 (1.28) 135.4953 (1.28) 135.4953 (1.28) 0.0000 (1.0) 135.4953 (1.28) 0.0000 (1.0) 0;0 0.0074 (0.78) 1 1
test_sharing-push[2.6.3] (0001_test_sh) 105.6246 (1.0) 105.6246 (1.0) 105.6246 (1.0) 0.0000 (1.0) 105.6246 (1.0) 0.0000 (1.0) 0;0 0.0095 (1.0) 1 1
test_sharing-push[2.9.5] (0001_test_sh) 149.5376 (1.42) 149.5376 (1.42) 149.5376 (1.42) 0.0000 (1.0) 149.5376 (1.42) 0.0000 (1.0) 0;0 0.0067 (0.71) 1 1
test_sharing-push[main] (0001_test_sh) 132.9633 (1.26) 132.9633 (1.26) 132.9633 (1.26) 0.0000 (1.0) 132.9633 (1.26) 0.0000 (1.0) 0;0 0.0075 (0.79) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.10.2] (0001_test_sh) 2.6622 (1.04) 2.6622 (1.04) 2.6622 (1.04) 0.0000 (1.0) 2.6622 (1.04) 0.0000 (1.0) 0;0 0.3756 (0.96) 1 1
test_sharing-push-noop[2.11.0] (0001_test_sh) 2.8274 (1.10) 2.8274 (1.10) 2.8274 (1.10) 0.0000 (1.0) 2.8274 (1.10) 0.0000 (1.0) 0;0 0.3537 (0.91) 1 1
test_sharing-push-noop[2.6.3] (0001_test_sh) 2.9889 (1.17) 2.9889 (1.17) 2.9889 (1.17) 0.0000 (1.0) 2.9889 (1.17) 0.0000 (1.0) 0;0 0.3346 (0.86) 1 1
test_sharing-push-noop[2.9.5] (0001_test_sh) 2.7969 (1.09) 2.7969 (1.09) 2.7969 (1.09) 0.0000 (1.0) 2.7969 (1.09) 0.0000 (1.0) 0;0 0.3575 (0.92) 1 1
test_sharing-push-noop[main] (0001_test_sh) 2.5620 (1.0) 2.5620 (1.0) 2.5620 (1.0) 0.0000 (1.0) 2.5620 (1.0) 0.0000 (1.0) 0;0 0.3903 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_status-status': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.10.2] (0001_test_st) 68.2922 (1.92) 68.2922 (1.92) 68.2922 (1.92) 0.0000 (1.0) 68.2922 (1.92) 0.0000 (1.0) 0;0 0.0146 (0.52) 1 1
test_status-status[2.11.0] (0001_test_st) 64.6216 (1.82) 64.6216 (1.82) 64.6216 (1.82) 0.0000 (1.0) 64.6216 (1.82) 0.0000 (1.0) 0;0 0.0155 (0.55) 1 1
test_status-status[2.6.3] (0001_test_st) 117.6808 (3.31) 117.6808 (3.31) 117.6808 (3.31) 0.0000 (1.0) 117.6808 (3.31) 0.0000 (1.0) 0;0 0.0085 (0.30) 1 1
test_status-status[2.9.5] (0001_test_st) 112.9001 (3.18) 112.9001 (3.18) 112.9001 (3.18) 0.0000 (1.0) 112.9001 (3.18) 0.0000 (1.0) 0;0 0.0089 (0.31) 1 1
test_status-status[main] (0001_test_st) 35.5157 (1.0) 35.5157 (1.0) 35.5157 (1.0) 0.0000 (1.0) 35.5157 (1.0) 0.0000 (1.0) 0;0 0.0282 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_status-status-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.10.2] (0001_test_st) 47.0905 (6.84) 47.0905 (6.84) 47.0905 (6.84) 0.0000 (1.0) 47.0905 (6.84) 0.0000 (1.0) 0;0 0.0212 (0.15) 1 1
test_status-status-changed[2.11.0] (0001_test_st) 47.9452 (6.97) 47.9452 (6.97) 47.9452 (6.97) 0.0000 (1.0) 47.9452 (6.97) 0.0000 (1.0) 0;0 0.0209 (0.14) 1 1
test_status-status-changed[2.6.3] (0001_test_st) 92.5071 (13.44) 92.5071 (13.44) 92.5071 (13.44) 0.0000 (1.0) 92.5071 (13.44) 0.0000 (1.0) 0;0 0.0108 (0.07) 1 1
test_status-status-changed[2.9.5] (0001_test_st) 90.1673 (13.10) 90.1673 (13.10) 90.1673 (13.10) 0.0000 (1.0) 90.1673 (13.10) 0.0000 (1.0) 0;0 0.0111 (0.08) 1 1
test_status-status-changed[main] (0001_test_st) 6.8809 (1.0) 6.8809 (1.0) 6.8809 (1.0) 0.0000 (1.0) 6.8809 (1.0) 0.0000 (1.0) 0;0 0.1453 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.10.2] (0001_test_st) 48.0814 (7.13) 48.0814 (7.13) 48.0814 (7.13) 0.0000 (1.0) 48.0814 (7.13) 0.0000 (1.0) 0;0 0.0208 (0.14) 1 1
test_status-status-changed-noop[2.11.0] (0001_test_st) 11.3024 (1.68) 11.3024 (1.68) 11.3024 (1.68) 0.0000 (1.0) 11.3024 (1.68) 0.0000 (1.0) 0;0 0.0885 (0.60) 1 1
test_status-status-changed-noop[2.6.3] (0001_test_st) 91.3878 (13.55) 91.3878 (13.55) 91.3878 (13.55) 0.0000 (1.0) 91.3878 (13.55) 0.0000 (1.0) 0;0 0.0109 (0.07) 1 1
test_status-status-changed-noop[2.9.5] (0001_test_st) 91.3883 (13.55) 91.3883 (13.55) 91.3883 (13.55) 0.0000 (1.0) 91.3883 (13.55) 0.0000 (1.0) 0;0 0.0109 (0.07) 1 1
test_status-status-changed-noop[main] (0001_test_st) 6.7422 (1.0) 6.7422 (1.0) 6.7422 (1.0) 0.0000 (1.0) 6.7422 (1.0) 0.0000 (1.0) 0;0 0.1483 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_status-status-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.10.2] (0001_test_st) 3.3808 (1.02) 3.3808 (1.02) 3.3808 (1.02) 0.0000 (1.0) 3.3808 (1.02) 0.0000 (1.0) 0;0 0.2958 (0.98) 1 1
test_status-status-noop[2.11.0] (0001_test_st) 3.3020 (1.0) 3.3020 (1.0) 3.3020 (1.0) 0.0000 (1.0) 3.3020 (1.0) 0.0000 (1.0) 0;0 0.3028 (1.0) 1 1
test_status-status-noop[2.6.3] (0001_test_st) 4.6317 (1.40) 4.6317 (1.40) 4.6317 (1.40) 0.0000 (1.0) 4.6317 (1.40) 0.0000 (1.0) 0;0 0.2159 (0.71) 1 1
test_status-status-noop[2.9.5] (0001_test_st) 3.3859 (1.03) 3.3859 (1.03) 3.3859 (1.03) 0.0000 (1.0) 3.3859 (1.03) 0.0000 (1.0) 0;0 0.2953 (0.98) 1 1
test_status-status-noop[main] (0001_test_st) 6.5699 (1.99) 6.5699 (1.99) 6.5699 (1.99) 0.0000 (1.0) 6.5699 (1.99) 0.0000 (1.0) 0;0 0.1522 (0.50) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```
github-actions[bot] commented 2 years ago 

dataset: mnist
project: example-get-started

----------------------------------------------------------------------------------- benchmark 'test_add-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.10.2] (0001_test_ad) 209.9422 (1.12) 209.9422 (1.12) 209.9422 (1.12) 0.0000 (1.0) 209.9422 (1.12) 0.0000 (1.0) 0;0 0.0048 (0.90) 1 1
test_add-add[2.11.0] (0001_test_ad) 209.3522 (1.11) 209.3522 (1.11) 209.3522 (1.11) 0.0000 (1.0) 209.3522 (1.11) 0.0000 (1.0) 0;0 0.0048 (0.90) 1 1
test_add-add[2.6.3] (0001_test_ad) 239.7808 (1.28) 239.7808 (1.28) 239.7808 (1.28) 0.0000 (1.0) 239.7808 (1.28) 0.0000 (1.0) 0;0 0.0042 (0.78) 1 1
test_add-add[2.9.5] (0001_test_ad) 257.4881 (1.37) 257.4881 (1.37) 257.4881 (1.37) 0.0000 (1.0) 257.4881 (1.37) 0.0000 (1.0) 0;0 0.0039 (0.73) 1 1
test_add-add[main] (0001_test_ad) 187.9575 (1.0) 187.9575 (1.0) 187.9575 (1.0) 0.0000 (1.0) 187.9575 (1.0) 0.0000 (1.0) 0;0 0.0053 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout[2.10.2] (0001_test_ch) 98.9339 (1.40) 98.9339 (1.40) 98.9339 (1.40) 0.0000 (1.0) 98.9339 (1.40) 0.0000 (1.0) 0;0 0.0101 (0.72) 1 1
test_checkout-checkout[2.11.0] (0001_test_ch) 97.3991 (1.37) 97.3991 (1.37) 97.3991 (1.37) 0.0000 (1.0) 97.3991 (1.37) 0.0000 (1.0) 0;0 0.0103 (0.73) 1 1
test_checkout-checkout[2.6.3] (0001_test_ch) 70.8392 (1.0) 70.8392 (1.0) 70.8392 (1.0) 0.0000 (1.0) 70.8392 (1.0) 0.0000 (1.0) 0;0 0.0141 (1.0) 1 1
test_checkout-checkout[2.9.5] (0001_test_ch) 110.9775 (1.57) 110.9775 (1.57) 110.9775 (1.57) 0.0000 (1.0) 110.9775 (1.57) 0.0000 (1.0) 0;0 0.0090 (0.64) 1 1
test_checkout-checkout[main] (0001_test_ch) 98.6829 (1.39) 98.6829 (1.39) 98.6829 (1.39) 0.0000 (1.0) 98.6829 (1.39) 0.0000 (1.0) 0;0 0.0101 (0.72) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop[2.10.2] (0001_test_ch) 5.9083 (2.13) 5.9083 (2.13) 5.9083 (2.13) 0.0000 (1.0) 5.9083 (2.13) 0.0000 (1.0) 0;0 0.1693 (0.47) 1 1
test_checkout-checkout-noop[2.11.0] (0001_test_ch) 5.9542 (2.14) 5.9542 (2.14) 5.9542 (2.14) 0.0000 (1.0) 5.9542 (2.14) 0.0000 (1.0) 0;0 0.1679 (0.47) 1 1
test_checkout-checkout-noop[2.6.3] (0001_test_ch) 2.7794 (1.0) 2.7794 (1.0) 2.7794 (1.0) 0.0000 (1.0) 2.7794 (1.0) 0.0000 (1.0) 0;0 0.3598 (1.0) 1 1
test_checkout-checkout-noop[2.9.5] (0001_test_ch) 5.5421 (1.99) 5.5421 (1.99) 5.5421 (1.99) 0.0000 (1.0) 5.5421 (1.99) 0.0000 (1.0) 0;0 0.1804 (0.50) 1 1
test_checkout-checkout-noop[main] (0001_test_ch) 7.8769 (2.83) 7.8769 (2.83) 7.8769 (2.83) 0.0000 (1.0) 7.8769 (2.83) 0.0000 (1.0) 0;0 0.1270 (0.35) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-update': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update[2.10.2] (0001_test_ch) 38.4260 (4.14) 38.4260 (4.14) 38.4260 (4.14) 0.0000 (1.0) 38.4260 (4.14) 0.0000 (1.0) 0;0 0.0260 (0.24) 1 1
test_checkout-checkout-update[2.11.0] (0001_test_ch) 37.7889 (4.07) 37.7889 (4.07) 37.7889 (4.07) 0.0000 (1.0) 37.7889 (4.07) 0.0000 (1.0) 0;0 0.0265 (0.25) 1 1
test_checkout-checkout-update[2.6.3] (0001_test_ch) 93.2882 (10.05) 93.2882 (10.05) 93.2882 (10.05) 0.0000 (1.0) 93.2882 (10.05) 0.0000 (1.0) 0;0 0.0107 (0.10) 1 1
test_checkout-checkout-update[2.9.5] (0001_test_ch) 38.7451 (4.18) 38.7451 (4.18) 38.7451 (4.18) 0.0000 (1.0) 38.7451 (4.18) 0.0000 (1.0) 0;0 0.0258 (0.24) 1 1
test_checkout-checkout-update[main] (0001_test_ch) 9.2779 (1.0) 9.2779 (1.0) 9.2779 (1.0) 0.0000 (1.0) 9.2779 (1.0) 0.0000 (1.0) 0;0 0.1078 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.10.2] (0001_test_di) 50.7482 (1.54) 50.7482 (1.54) 50.7482 (1.54) 0.0000 (1.0) 50.7482 (1.54) 0.0000 (1.0) 0;0 0.0197 (0.65) 1 1
test_diff-diff[2.11.0] (0001_test_di) 46.0364 (1.40) 46.0364 (1.40) 46.0364 (1.40) 0.0000 (1.0) 46.0364 (1.40) 0.0000 (1.0) 0;0 0.0217 (0.71) 1 1
test_diff-diff[2.6.3] (0001_test_di) 93.4018 (2.84) 93.4018 (2.84) 93.4018 (2.84) 0.0000 (1.0) 93.4018 (2.84) 0.0000 (1.0) 0;0 0.0107 (0.35) 1 1
test_diff-diff[2.9.5] (0001_test_di) 50.4803 (1.54) 50.4803 (1.54) 50.4803 (1.54) 0.0000 (1.0) 50.4803 (1.54) 0.0000 (1.0) 0;0 0.0198 (0.65) 1 1
test_diff-diff[main] (0001_test_di) 32.8478 (1.0) 32.8478 (1.0) 32.8478 (1.0) 0.0000 (1.0) 32.8478 (1.0) 0.0000 (1.0) 0;0 0.0304 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.10.2] (0001_test_di) 31.1955 (4.97) 31.1955 (4.97) 31.1955 (4.97) 0.0000 (1.0) 31.1955 (4.97) 0.0000 (1.0) 0;0 0.0321 (0.20) 1 1
test_diff-diff-changed[2.11.0] (0001_test_di) 29.5324 (4.70) 29.5324 (4.70) 29.5324 (4.70) 0.0000 (1.0) 29.5324 (4.70) 0.0000 (1.0) 0;0 0.0339 (0.21) 1 1
test_diff-diff-changed[2.6.3] (0001_test_di) 70.8736 (11.29) 70.8736 (11.29) 70.8736 (11.29) 0.0000 (1.0) 70.8736 (11.29) 0.0000 (1.0) 0;0 0.0141 (0.09) 1 1
test_diff-diff-changed[2.9.5] (0001_test_di) 30.4763 (4.85) 30.4763 (4.85) 30.4763 (4.85) 0.0000 (1.0) 30.4763 (4.85) 0.0000 (1.0) 0;0 0.0328 (0.21) 1 1
test_diff-diff-changed[main] (0001_test_di) 6.2791 (1.0) 6.2791 (1.0) 6.2791 (1.0) 0.0000 (1.0) 6.2791 (1.0) 0.0000 (1.0) 0;0 0.1593 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.10.2] (0001_test_di) 31.3693 (10.00) 31.3693 (10.00) 31.3693 (10.00) 0.0000 (1.0) 31.3693 (10.00) 0.0000 (1.0) 0;0 0.0319 (0.10) 1 1
test_diff-diff-changed-noop[2.11.0] (0001_test_di) 3.1381 (1.0) 3.1381 (1.0) 3.1381 (1.0) 0.0000 (1.0) 3.1381 (1.0) 0.0000 (1.0) 0;0 0.3187 (1.0) 1 1
test_diff-diff-changed-noop[2.6.3] (0001_test_di) 71.5999 (22.82) 71.5999 (22.82) 71.5999 (22.82) 0.0000 (1.0) 71.5999 (22.82) 0.0000 (1.0) 0;0 0.0140 (0.04) 1 1
test_diff-diff-changed-noop[2.9.5] (0001_test_di) 30.4221 (9.69) 30.4221 (9.69) 30.4221 (9.69) 0.0000 (1.0) 30.4221 (9.69) 0.0000 (1.0) 0;0 0.0329 (0.10) 1 1
test_diff-diff-changed-noop[main] (0001_test_di) 6.2969 (2.01) 6.2969 (2.01) 6.2969 (2.01) 0.0000 (1.0) 6.2969 (2.01) 0.0000 (1.0) 0;0 0.1588 (0.50) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.10.2] (0001_test_di) 3.7190 (1.06) 3.7190 (1.06) 3.7190 (1.06) 0.0000 (1.0) 3.7190 (1.06) 0.0000 (1.0) 0;0 0.2689 (0.94) 1 1
test_diff-diff-noop[2.11.0] (0001_test_di) 3.6017 (1.03) 3.6017 (1.03) 3.6017 (1.03) 0.0000 (1.0) 3.6017 (1.03) 0.0000 (1.0) 0;0 0.2776 (0.97) 1 1
test_diff-diff-noop[2.6.3] (0001_test_di) 46.5892 (13.33) 46.5892 (13.33) 46.5892 (13.33) 0.0000 (1.0) 46.5892 (13.33) 0.0000 (1.0) 0;0 0.0215 (0.08) 1 1
test_diff-diff-noop[2.9.5] (0001_test_di) 3.4952 (1.0) 3.4952 (1.0) 3.4952 (1.0) 0.0000 (1.0) 3.4952 (1.0) 0.0000 (1.0) 0;0 0.2861 (1.0) 1 1
test_diff-diff-noop[main] (0001_test_di) 6.3080 (1.80) 6.3080 (1.80) 6.3080 (1.80) 0.0000 (1.0) 6.3080 (1.80) 0.0000 (1.0) 0;0 0.1585 (0.55) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 3 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.10.2] (0001_test_ex) 1.3710 (1.01) 1.3710 (1.01) 1.3710 (1.01) 0.0000 (1.0) 1.3710 (1.01) 0.0000 (1.0) 0;0 0.7294 (0.99) 1 1
test_exp_show-exp[2.11.0] (0001_test_ex) 1.3612 (1.0) 1.3612 (1.0) 1.3612 (1.0) 0.0000 (1.0) 1.3612 (1.0) 0.0000 (1.0) 0;0 0.7347 (1.0) 1 1
test_exp_show-exp[main] (0001_test_ex) 1.5147 (1.11) 1.5147 (1.11) 1.5147 (1.11) 0.0000 (1.0) 1.5147 (1.11) 0.0000 (1.0) 0;0 0.6602 (0.90) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_gc-gc': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.10.2] (0001_test_gc) 2.9309 (1.21) 2.9309 (1.21) 2.9309 (1.21) 0.0000 (1.0) 2.9309 (1.21) 0.0000 (1.0) 0;0 0.3412 (0.83) 1 1
test_gc-gc[2.11.0] (0001_test_gc) 2.4234 (1.0) 2.4234 (1.0) 2.4234 (1.0) 0.0000 (1.0) 2.4234 (1.0) 0.0000 (1.0) 0;0 0.4126 (1.0) 1 1
test_gc-gc[2.6.3] (0001_test_gc) 3.5873 (1.48) 3.5873 (1.48) 3.5873 (1.48) 0.0000 (1.0) 3.5873 (1.48) 0.0000 (1.0) 0;0 0.2788 (0.68) 1 1
test_gc-gc[2.9.5] (0001_test_gc) 2.9569 (1.22) 2.9569 (1.22) 2.9569 (1.22) 0.0000 (1.0) 2.9569 (1.22) 0.0000 (1.0) 0;0 0.3382 (0.82) 1 1
test_gc-gc[main] (0001_test_gc) 2.7337 (1.13) 2.7337 (1.13) 2.7337 (1.13) 0.0000 (1.0) 2.7337 (1.13) 0.0000 (1.0) 0;0 0.3658 (0.89) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_get-get': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.10.2] (0001_test_ge) 258.9282 (2.03) 258.9282 (2.03) 258.9282 (2.03) 0.0000 (1.0) 258.9282 (2.03) 0.0000 (1.0) 0;0 0.0039 (0.49) 1 1
test_get-get[2.11.0] (0001_test_ge) 204.8103 (1.60) 204.8103 (1.60) 204.8103 (1.60) 0.0000 (1.0) 204.8103 (1.60) 0.0000 (1.0) 0;0 0.0049 (0.62) 1 1
test_get-get[2.6.3] (0001_test_ge) 150.6264 (1.18) 150.6264 (1.18) 150.6264 (1.18) 0.0000 (1.0) 150.6264 (1.18) 0.0000 (1.0) 0;0 0.0066 (0.85) 1 1
test_get-get[2.9.5] (0001_test_ge) 127.6233 (1.0) 127.6233 (1.0) 127.6233 (1.0) 0.0000 (1.0) 127.6233 (1.0) 0.0000 (1.0) 0;0 0.0078 (1.0) 1 1
test_get-get[main] (0001_test_ge) 206.2499 (1.62) 206.2499 (1.62) 206.2499 (1.62) 0.0000 (1.0) 206.2499 (1.62) 0.0000 (1.0) 0;0 0.0048 (0.62) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_help---help': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.10.2] (0001_test_he) 138.2665 (1.0) 145.0602 (1.02) 139.8244 (1.0) 0.9946 (1.32) 139.7578 (1.00) 1.1983 (1.31) 28;1 7.1518 (1.0) 100 1
test_help---help[2.11.0] (0001_test_he) 138.2737 (1.00) 147.9622 (1.04) 139.9473 (1.00) 1.1562 (1.54) 139.7219 (1.0) 1.0715 (1.17) 16;2 7.1455 (1.00) 100 1
test_help---help[2.6.3] (0001_test_he) 179.0982 (1.30) 185.1680 (1.30) 180.7014 (1.29) 0.9070 (1.21) 180.5740 (1.29) 1.1925 (1.31) 29;1 5.5340 (0.77) 100 1
test_help---help[2.9.5] (0001_test_he) 138.6266 (1.00) 152.9804 (1.07) 140.4216 (1.00) 1.4503 (1.93) 140.3136 (1.00) 0.9121 (1.0) 6;2 7.1214 (1.00) 100 1
test_help---help[main] (0001_test_he) 138.4478 (1.00) 142.8050 (1.0) 140.3922 (1.00) 0.7523 (1.0) 140.2873 (1.00) 0.9440 (1.03) 33;3 7.1229 (1.00) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_import-import': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.10.2] (0001_test_im) 490.8868 (1.37) 490.8868 (1.37) 490.8868 (1.37) 0.0000 (1.0) 490.8868 (1.37) 0.0000 (1.0) 0;0 0.0020 (0.73) 1 1
test_import-import[2.11.0] (0001_test_im) 436.7386 (1.22) 436.7386 (1.22) 436.7386 (1.22) 0.0000 (1.0) 436.7386 (1.22) 0.0000 (1.0) 0;0 0.0023 (0.82) 1 1
test_import-import[2.6.3] (0001_test_im) 357.1664 (1.0) 357.1664 (1.0) 357.1664 (1.0) 0.0000 (1.0) 357.1664 (1.0) 0.0000 (1.0) 0;0 0.0028 (1.0) 1 1
test_import-import[2.9.5] (0001_test_im) 555.8184 (1.56) 555.8184 (1.56) 555.8184 (1.56) 0.0000 (1.0) 555.8184 (1.56) 0.0000 (1.0) 0;0 0.0018 (0.64) 1 1
test_import-import[main] (0001_test_im) 419.5574 (1.17) 419.5574 (1.17) 419.5574 (1.17) 0.0000 (1.0) 419.5574 (1.17) 0.0000 (1.0) 0;0 0.0024 (0.85) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_init-init': 5 tests -------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.10.2] (0001_test_in) 345.5033 (1.04) 447.2214 (1.04) 380.4203 (1.05) 15.7739 (1.03) 378.5976 (1.04) 16.5535 (1.0) 26;3 2.6287 (0.95) 100 1
test_init-init[2.11.0] (0001_test_in) 361.7601 (1.09) 435.4980 (1.01) 390.8342 (1.08) 15.4433 (1.01) 387.8608 (1.07) 18.3937 (1.11) 27;4 2.5586 (0.93) 100 1
test_init-init[2.6.3] (0001_test_in) 332.3680 (1.0) 429.4248 (1.0) 362.4441 (1.0) 16.1100 (1.06) 363.6303 (1.0) 16.8834 (1.02) 21;4 2.7590 (1.0) 100 1
test_init-init[2.9.5] (0001_test_in) 356.9697 (1.07) 473.9623 (1.10) 392.1685 (1.08) 19.2770 (1.26) 389.5786 (1.07) 24.6183 (1.49) 27;2 2.5499 (0.92) 100 1
test_init-init[main] (0001_test_in) 361.1134 (1.09) 431.8740 (1.01) 390.1603 (1.08) 15.2677 (1.0) 389.6179 (1.07) 20.8028 (1.26) 33;1 2.5630 (0.93) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_list-list': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.10.2] (0001_test_li) 2.5181 (3.21) 2.5181 (3.21) 2.5181 (3.21) 0.0000 (1.0) 2.5181 (3.21) 0.0000 (1.0) 0;0 0.3971 (0.31) 1 1
test_list-list[2.11.0] (0001_test_li) 2.5050 (3.19) 2.5050 (3.19) 2.5050 (3.19) 0.0000 (1.0) 2.5050 (3.19) 0.0000 (1.0) 0;0 0.3992 (0.31) 1 1
test_list-list[2.6.3] (0001_test_li) 0.7841 (1.0) 0.7841 (1.0) 0.7841 (1.0) 0.0000 (1.0) 0.7841 (1.0) 0.0000 (1.0) 0;0 1.2754 (1.0) 1 1
test_list-list[2.9.5] (0001_test_li) 2.8119 (3.59) 2.8119 (3.59) 2.8119 (3.59) 0.0000 (1.0) 2.8119 (3.59) 0.0000 (1.0) 0;0 0.3556 (0.28) 1 1
test_list-list[main] (0001_test_li) 2.5295 (3.23) 2.5295 (3.23) 2.5295 (3.23) 0.0000 (1.0) 2.5295 (3.23) 0.0000 (1.0) 0;0 0.3953 (0.31) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_list-list-dvc-only': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.10.2] (0001_test_li) 2.5374 (4.68) 2.5374 (4.68) 2.5374 (4.68) 0.0000 (1.0) 2.5374 (4.68) 0.0000 (1.0) 0;0 0.3941 (0.21) 1 1
test_list-list-dvc-only[2.11.0] (0001_test_li) 2.5738 (4.74) 2.5738 (4.74) 2.5738 (4.74) 0.0000 (1.0) 2.5738 (4.74) 0.0000 (1.0) 0;0 0.3885 (0.21) 1 1
test_list-list-dvc-only[2.6.3] (0001_test_li) 0.5427 (1.0) 0.5427 (1.0) 0.5427 (1.0) 0.0000 (1.0) 0.5427 (1.0) 0.0000 (1.0) 0;0 1.8427 (1.0) 1 1
test_list-list-dvc-only[2.9.5] (0001_test_li) 2.9462 (5.43) 2.9462 (5.43) 2.9462 (5.43) 0.0000 (1.0) 2.9462 (5.43) 0.0000 (1.0) 0;0 0.3394 (0.18) 1 1
test_list-list-dvc-only[main] (0001_test_li) 2.4896 (4.59) 2.4896 (4.59) 2.4896 (4.59) 0.0000 (1.0) 2.4896 (4.59) 0.0000 (1.0) 0;0 0.4017 (0.22) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_plots_show-plots': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots_show-plots[2.10.2] (0001_test_pl) 1.0229 (1.22) 1.0229 (1.22) 1.0229 (1.22) 0.0000 (1.0) 1.0229 (1.22) 0.0000 (1.0) 0;0 0.9776 (0.82) 1 1
test_plots_show-plots[2.11.0] (0001_test_pl) 1.6552 (1.98) 1.6552 (1.98) 1.6552 (1.98) 0.0000 (1.0) 1.6552 (1.98) 0.0000 (1.0) 0;0 0.6041 (0.51) 1 1
test_plots_show-plots[2.6.3] (0001_test_pl) 0.8366 (1.0) 0.8366 (1.0) 0.8366 (1.0) 0.0000 (1.0) 0.8366 (1.0) 0.0000 (1.0) 0;0 1.1953 (1.0) 1 1
test_plots_show-plots[2.9.5] (0001_test_pl) 1.7244 (2.06) 1.7244 (2.06) 1.7244 (2.06) 0.0000 (1.0) 1.7244 (2.06) 0.0000 (1.0) 0;0 0.5799 (0.49) 1 1
test_plots_show-plots[main] (0001_test_pl) 1.8229 (2.18) 1.8229 (2.18) 1.8229 (2.18) 0.0000 (1.0) 1.8229 (2.18) 0.0000 (1.0) 0;0 0.5486 (0.46) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_pull-pull': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.10.2] (0001_test_pu) 265.4294 (1.26) 265.4294 (1.26) 265.4294 (1.26) 0.0000 (1.0) 265.4294 (1.26) 0.0000 (1.0) 0;0 0.0038 (0.79) 1 1
test_pull-pull[2.11.0] (0001_test_pu) 263.9355 (1.25) 263.9355 (1.25) 263.9355 (1.25) 0.0000 (1.0) 263.9355 (1.25) 0.0000 (1.0) 0;0 0.0038 (0.80) 1 1
test_pull-pull[2.6.3] (0001_test_pu) 210.7075 (1.0) 210.7075 (1.0) 210.7075 (1.0) 0.0000 (1.0) 210.7075 (1.0) 0.0000 (1.0) 0;0 0.0047 (1.0) 1 1
test_pull-pull[2.9.5] (0001_test_pu) 299.2079 (1.42) 299.2079 (1.42) 299.2079 (1.42) 0.0000 (1.0) 299.2079 (1.42) 0.0000 (1.0) 0;0 0.0033 (0.70) 1 1
test_pull-pull[main] (0001_test_pu) 264.6732 (1.26) 264.6732 (1.26) 264.6732 (1.26) 0.0000 (1.0) 264.6732 (1.26) 0.0000 (1.0) 0;0 0.0038 (0.80) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_push-push': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.10.2] (0001_test_pu) 118.2706 (1.25) 118.2706 (1.25) 118.2706 (1.25) 0.0000 (1.0) 118.2706 (1.25) 0.0000 (1.0) 0;0 0.0085 (0.80) 1 1
test_push-push[2.11.0] (0001_test_pu) 116.6895 (1.23) 116.6895 (1.23) 116.6895 (1.23) 0.0000 (1.0) 116.6895 (1.23) 0.0000 (1.0) 0;0 0.0086 (0.81) 1 1
test_push-push[2.6.3] (0001_test_pu) 94.5892 (1.0) 94.5892 (1.0) 94.5892 (1.0) 0.0000 (1.0) 94.5892 (1.0) 0.0000 (1.0) 0;0 0.0106 (1.0) 1 1
test_push-push[2.9.5] (0001_test_pu) 129.4935 (1.37) 129.4935 (1.37) 129.4935 (1.37) 0.0000 (1.0) 129.4935 (1.37) 0.0000 (1.0) 0;0 0.0077 (0.73) 1 1
test_push-push[main] (0001_test_pu) 116.4042 (1.23) 116.4042 (1.23) 116.4042 (1.23) 0.0000 (1.0) 116.4042 (1.23) 0.0000 (1.0) 0;0 0.0086 (0.81) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.10.2] (0001_test_sh) 251.9915 (1.10) 251.9915 (1.10) 251.9915 (1.10) 0.0000 (1.0) 251.9915 (1.10) 0.0000 (1.0) 0;0 0.0040 (0.91) 1 1
test_sharing-add[2.11.0] (0001_test_sh) 251.6912 (1.10) 251.6912 (1.10) 251.6912 (1.10) 0.0000 (1.0) 251.6912 (1.10) 0.0000 (1.0) 0;0 0.0040 (0.91) 1 1
test_sharing-add[2.6.3] (0001_test_sh) 284.6669 (1.24) 284.6669 (1.24) 284.6669 (1.24) 0.0000 (1.0) 284.6669 (1.24) 0.0000 (1.0) 0;0 0.0035 (0.81) 1 1
test_sharing-add[2.9.5] (0001_test_sh) 303.8742 (1.33) 303.8742 (1.33) 303.8742 (1.33) 0.0000 (1.0) 303.8742 (1.33) 0.0000 (1.0) 0;0 0.0033 (0.75) 1 1
test_sharing-add[main] (0001_test_sh) 229.2286 (1.0) 229.2286 (1.0) 229.2286 (1.0) 0.0000 (1.0) 229.2286 (1.0) 0.0000 (1.0) 0;0 0.0044 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.10.2] (0001_test_sh) 127.1666 (6.50) 127.1666 (6.50) 127.1666 (6.50) 0.0000 (1.0) 127.1666 (6.50) 0.0000 (1.0) 0;0 0.0079 (0.15) 1 1
test_sharing-add-noop[2.11.0] (0001_test_sh) 123.8219 (6.33) 123.8219 (6.33) 123.8219 (6.33) 0.0000 (1.0) 123.8219 (6.33) 0.0000 (1.0) 0;0 0.0081 (0.16) 1 1
test_sharing-add-noop[2.6.3] (0001_test_sh) 19.5744 (1.0) 19.5744 (1.0) 19.5744 (1.0) 0.0000 (1.0) 19.5744 (1.0) 0.0000 (1.0) 0;0 0.0511 (1.0) 1 1
test_sharing-add-noop[2.9.5] (0001_test_sh) 140.3573 (7.17) 140.3573 (7.17) 140.3573 (7.17) 0.0000 (1.0) 140.3573 (7.17) 0.0000 (1.0) 0;0 0.0071 (0.14) 1 1
test_sharing-add-noop[main] (0001_test_sh) 137.9857 (7.05) 137.9857 (7.05) 137.9857 (7.05) 0.0000 (1.0) 137.9857 (7.05) 0.0000 (1.0) 0;0 0.0072 (0.14) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.10.2] (0001_test_sh) 5.9594 (2.10) 5.9594 (2.10) 5.9594 (2.10) 0.0000 (1.0) 5.9594 (2.10) 0.0000 (1.0) 0;0 0.1678 (0.48) 1 1
test_sharing-checkout-noop[2.11.0] (0001_test_sh) 5.8059 (2.05) 5.8059 (2.05) 5.8059 (2.05) 0.0000 (1.0) 5.8059 (2.05) 0.0000 (1.0) 0;0 0.1722 (0.49) 1 1
test_sharing-checkout-noop[2.6.3] (0001_test_sh) 2.8384 (1.0) 2.8384 (1.0) 2.8384 (1.0) 0.0000 (1.0) 2.8384 (1.0) 0.0000 (1.0) 0;0 0.3523 (1.0) 1 1
test_sharing-checkout-noop[2.9.5] (0001_test_sh) 5.8611 (2.06) 5.8611 (2.06) 5.8611 (2.06) 0.0000 (1.0) 5.8611 (2.06) 0.0000 (1.0) 0;0 0.1706 (0.48) 1 1
test_sharing-checkout-noop[main] (0001_test_sh) 8.4733 (2.99) 8.4733 (2.99) 8.4733 (2.99) 0.0000 (1.0) 8.4733 (2.99) 0.0000 (1.0) 0;0 0.1180 (0.33) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.10.2] (0001_test_sh) 287.4220 (1.29) 287.4220 (1.29) 287.4220 (1.29) 0.0000 (1.0) 287.4220 (1.29) 0.0000 (1.0) 0;0 0.0035 (0.77) 1 1
test_sharing-pull[2.11.0] (0001_test_sh) 286.1716 (1.29) 286.1716 (1.29) 286.1716 (1.29) 0.0000 (1.0) 286.1716 (1.29) 0.0000 (1.0) 0;0 0.0035 (0.78) 1 1
test_sharing-pull[2.6.3] (0001_test_sh) 222.6061 (1.0) 222.6061 (1.0) 222.6061 (1.0) 0.0000 (1.0) 222.6061 (1.0) 0.0000 (1.0) 0;0 0.0045 (1.0) 1 1
test_sharing-pull[2.9.5] (0001_test_sh) 322.2253 (1.45) 322.2253 (1.45) 322.2253 (1.45) 0.0000 (1.0) 322.2253 (1.45) 0.0000 (1.0) 0;0 0.0031 (0.69) 1 1
test_sharing-pull[main] (0001_test_sh) 295.3508 (1.33) 295.3508 (1.33) 295.3508 (1.33) 0.0000 (1.0) 295.3508 (1.33) 0.0000 (1.0) 0;0 0.0034 (0.75) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.10.2] (0001_test_sh) 6.9256 (1.86) 6.9256 (1.86) 6.9256 (1.86) 0.0000 (1.0) 6.9256 (1.86) 0.0000 (1.0) 0;0 0.1444 (0.54) 1 1
test_sharing-pull-noop[2.11.0] (0001_test_sh) 6.7140 (1.81) 6.7140 (1.81) 6.7140 (1.81) 0.0000 (1.0) 6.7140 (1.81) 0.0000 (1.0) 0;0 0.1489 (0.55) 1 1
test_sharing-pull-noop[2.6.3] (0001_test_sh) 3.7167 (1.0) 3.7167 (1.0) 3.7167 (1.0) 0.0000 (1.0) 3.7167 (1.0) 0.0000 (1.0) 0;0 0.2691 (1.0) 1 1
test_sharing-pull-noop[2.9.5] (0001_test_sh) 6.6503 (1.79) 6.6503 (1.79) 6.6503 (1.79) 0.0000 (1.0) 6.6503 (1.79) 0.0000 (1.0) 0;0 0.1504 (0.56) 1 1
test_sharing-pull-noop[main] (0001_test_sh) 9.5380 (2.57) 9.5380 (2.57) 9.5380 (2.57) 0.0000 (1.0) 9.5380 (2.57) 0.0000 (1.0) 0;0 0.1048 (0.39) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.10.2] (0001_test_sh) 157.1878 (1.27) 157.1878 (1.27) 157.1878 (1.27) 0.0000 (1.0) 157.1878 (1.27) 0.0000 (1.0) 0;0 0.0064 (0.79) 1 1
test_sharing-push[2.11.0] (0001_test_sh) 160.0464 (1.30) 160.0464 (1.30) 160.0464 (1.30) 0.0000 (1.0) 160.0464 (1.30) 0.0000 (1.0) 0;0 0.0062 (0.77) 1 1
test_sharing-push[2.6.3] (0001_test_sh) 123.4257 (1.0) 123.4257 (1.0) 123.4257 (1.0) 0.0000 (1.0) 123.4257 (1.0) 0.0000 (1.0) 0;0 0.0081 (1.0) 1 1
test_sharing-push[2.9.5] (0001_test_sh) 171.6794 (1.39) 171.6794 (1.39) 171.6794 (1.39) 0.0000 (1.0) 171.6794 (1.39) 0.0000 (1.0) 0;0 0.0058 (0.72) 1 1
test_sharing-push[main] (0001_test_sh) 158.1329 (1.28) 158.1329 (1.28) 158.1329 (1.28) 0.0000 (1.0) 158.1329 (1.28) 0.0000 (1.0) 0;0 0.0063 (0.78) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.10.2] (0001_test_sh) 3.0380 (1.05) 3.0380 (1.05) 3.0380 (1.05) 0.0000 (1.0) 3.0380 (1.05) 0.0000 (1.0) 0;0 0.3292 (0.95) 1 1
test_sharing-push-noop[2.11.0] (0001_test_sh) 3.1178 (1.08) 3.1178 (1.08) 3.1178 (1.08) 0.0000 (1.0) 3.1178 (1.08) 0.0000 (1.0) 0;0 0.3207 (0.93) 1 1
test_sharing-push-noop[2.6.3] (0001_test_sh) 3.3354 (1.16) 3.3354 (1.16) 3.3354 (1.16) 0.0000 (1.0) 3.3354 (1.16) 0.0000 (1.0) 0;0 0.2998 (0.86) 1 1
test_sharing-push-noop[2.9.5] (0001_test_sh) 2.8845 (1.0) 2.8845 (1.0) 2.8845 (1.0) 0.0000 (1.0) 2.8845 (1.0) 0.0000 (1.0) 0;0 0.3467 (1.0) 1 1
test_sharing-push-noop[main] (0001_test_sh) 2.9423 (1.02) 2.9423 (1.02) 2.9423 (1.02) 0.0000 (1.0) 2.9423 (1.02) 0.0000 (1.0) 0;0 0.3399 (0.98) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_status-status': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.10.2] (0001_test_st) 55.1309 (1.90) 55.1309 (1.90) 55.1309 (1.90) 0.0000 (1.0) 55.1309 (1.90) 0.0000 (1.0) 0;0 0.0181 (0.53) 1 1
test_status-status[2.11.0] (0001_test_st) 51.5786 (1.78) 51.5786 (1.78) 51.5786 (1.78) 0.0000 (1.0) 51.5786 (1.78) 0.0000 (1.0) 0;0 0.0194 (0.56) 1 1
test_status-status[2.6.3] (0001_test_st) 92.6805 (3.19) 92.6805 (3.19) 92.6805 (3.19) 0.0000 (1.0) 92.6805 (3.19) 0.0000 (1.0) 0;0 0.0108 (0.31) 1 1
test_status-status[2.9.5] (0001_test_st) 87.3640 (3.01) 87.3640 (3.01) 87.3640 (3.01) 0.0000 (1.0) 87.3640 (3.01) 0.0000 (1.0) 0;0 0.0114 (0.33) 1 1
test_status-status[main] (0001_test_st) 29.0292 (1.0) 29.0292 (1.0) 29.0292 (1.0) 0.0000 (1.0) 29.0292 (1.0) 0.0000 (1.0) 0;0 0.0344 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_status-status-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.10.2] (0001_test_st) 37.8828 (7.57) 37.8828 (7.57) 37.8828 (7.57) 0.0000 (1.0) 37.8828 (7.57) 0.0000 (1.0) 0;0 0.0264 (0.13) 1 1
test_status-status-changed[2.11.0] (0001_test_st) 38.1406 (7.62) 38.1406 (7.62) 38.1406 (7.62) 0.0000 (1.0) 38.1406 (7.62) 0.0000 (1.0) 0;0 0.0262 (0.13) 1 1
test_status-status-changed[2.6.3] (0001_test_st) 75.5625 (15.10) 75.5625 (15.10) 75.5625 (15.10) 0.0000 (1.0) 75.5625 (15.10) 0.0000 (1.0) 0;0 0.0132 (0.07) 1 1
test_status-status-changed[2.9.5] (0001_test_st) 72.1067 (14.41) 72.1067 (14.41) 72.1067 (14.41) 0.0000 (1.0) 72.1067 (14.41) 0.0000 (1.0) 0;0 0.0139 (0.07) 1 1
test_status-status-changed[main] (0001_test_st) 5.0046 (1.0) 5.0046 (1.0) 5.0046 (1.0) 0.0000 (1.0) 5.0046 (1.0) 0.0000 (1.0) 0;0 0.1998 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.10.2] (0001_test_st) 37.2107 (6.77) 37.2107 (6.77) 37.2107 (6.77) 0.0000 (1.0) 37.2107 (6.77) 0.0000 (1.0) 0;0 0.0269 (0.15) 1 1
test_status-status-changed-noop[2.11.0] (0001_test_st) 8.0329 (1.46) 8.0329 (1.46) 8.0329 (1.46) 0.0000 (1.0) 8.0329 (1.46) 0.0000 (1.0) 0;0 0.1245 (0.68) 1 1
test_status-status-changed-noop[2.6.3] (0001_test_st) 74.5995 (13.57) 74.5995 (13.57) 74.5995 (13.57) 0.0000 (1.0) 74.5995 (13.57) 0.0000 (1.0) 0;0 0.0134 (0.07) 1 1
test_status-status-changed-noop[2.9.5] (0001_test_st) 71.1835 (12.95) 71.1835 (12.95) 71.1835 (12.95) 0.0000 (1.0) 71.1835 (12.95) 0.0000 (1.0) 0;0 0.0140 (0.08) 1 1
test_status-status-changed-noop[main] (0001_test_st) 5.4985 (1.0) 5.4985 (1.0) 5.4985 (1.0) 0.0000 (1.0) 5.4985 (1.0) 0.0000 (1.0) 0;0 0.1819 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_status-status-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.10.2] (0001_test_st) 2.7378 (1.12) 2.7378 (1.12) 2.7378 (1.12) 0.0000 (1.0) 2.7378 (1.12) 0.0000 (1.0) 0;0 0.3653 (0.90) 1 1
test_status-status-noop[2.11.0] (0001_test_st) 2.5906 (1.06) 2.5906 (1.06) 2.5906 (1.06) 0.0000 (1.0) 2.5906 (1.06) 0.0000 (1.0) 0;0 0.3860 (0.95) 1 1
test_status-status-noop[2.6.3] (0001_test_st) 3.7478 (1.53) 3.7478 (1.53) 3.7478 (1.53) 0.0000 (1.0) 3.7478 (1.53) 0.0000 (1.0) 0;0 0.2668 (0.65) 1 1
test_status-status-noop[2.9.5] (0001_test_st) 2.4533 (1.0) 2.4533 (1.0) 2.4533 (1.0) 0.0000 (1.0) 2.4533 (1.0) 0.0000 (1.0) 0;0 0.4076 (1.0) 1 1
test_status-status-noop[main] (0001_test_st) 4.9653 (2.02) 4.9653 (2.02) 4.9653 (2.02) 0.0000 (1.0) 4.9653 (2.02) 0.0000 (1.0) 0;0 0.2014 (0.49) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```
github-actions[bot] commented 2 years ago 

dataset: mnist
project: example-get-started

----------------------------------------------------------------------------------- benchmark 'test_add-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.10.2] (0001_test_ad) 213.2422 (1.12) 213.2422 (1.12) 213.2422 (1.12) 0.0000 (1.0) 213.2422 (1.12) 0.0000 (1.0) 0;0 0.0047 (0.90) 1 1
test_add-add[2.11.0] (0001_test_ad) 210.3909 (1.10) 210.3909 (1.10) 210.3909 (1.10) 0.0000 (1.0) 210.3909 (1.10) 0.0000 (1.0) 0;0 0.0048 (0.91) 1 1
test_add-add[2.6.3] (0001_test_ad) 241.8898 (1.27) 241.8898 (1.27) 241.8898 (1.27) 0.0000 (1.0) 241.8898 (1.27) 0.0000 (1.0) 0;0 0.0041 (0.79) 1 1
test_add-add[2.9.5] (0001_test_ad) 260.2895 (1.36) 260.2895 (1.36) 260.2895 (1.36) 0.0000 (1.0) 260.2895 (1.36) 0.0000 (1.0) 0;0 0.0038 (0.73) 1 1
test_add-add[main] (0001_test_ad) 190.9583 (1.0) 190.9583 (1.0) 190.9583 (1.0) 0.0000 (1.0) 190.9583 (1.0) 0.0000 (1.0) 0;0 0.0052 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout[2.10.2] (0001_test_ch) 76.1725 (1.44) 76.1725 (1.44) 76.1725 (1.44) 0.0000 (1.0) 76.1725 (1.44) 0.0000 (1.0) 0;0 0.0131 (0.69) 1 1
test_checkout-checkout[2.11.0] (0001_test_ch) 74.8679 (1.42) 74.8679 (1.42) 74.8679 (1.42) 0.0000 (1.0) 74.8679 (1.42) 0.0000 (1.0) 0;0 0.0134 (0.71) 1 1
test_checkout-checkout[2.6.3] (0001_test_ch) 52.8857 (1.0) 52.8857 (1.0) 52.8857 (1.0) 0.0000 (1.0) 52.8857 (1.0) 0.0000 (1.0) 0;0 0.0189 (1.0) 1 1
test_checkout-checkout[2.9.5] (0001_test_ch) 86.4000 (1.63) 86.4000 (1.63) 86.4000 (1.63) 0.0000 (1.0) 86.4000 (1.63) 0.0000 (1.0) 0;0 0.0116 (0.61) 1 1
test_checkout-checkout[main] (0001_test_ch) 78.7280 (1.49) 78.7280 (1.49) 78.7280 (1.49) 0.0000 (1.0) 78.7280 (1.49) 0.0000 (1.0) 0;0 0.0127 (0.67) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop[2.10.2] (0001_test_ch) 4.7716 (2.03) 4.7716 (2.03) 4.7716 (2.03) 0.0000 (1.0) 4.7716 (2.03) 0.0000 (1.0) 0;0 0.2096 (0.49) 1 1
test_checkout-checkout-noop[2.11.0] (0001_test_ch) 4.7297 (2.02) 4.7297 (2.02) 4.7297 (2.02) 0.0000 (1.0) 4.7297 (2.02) 0.0000 (1.0) 0;0 0.2114 (0.50) 1 1
test_checkout-checkout-noop[2.6.3] (0001_test_ch) 2.3453 (1.0) 2.3453 (1.0) 2.3453 (1.0) 0.0000 (1.0) 2.3453 (1.0) 0.0000 (1.0) 0;0 0.4264 (1.0) 1 1
test_checkout-checkout-noop[2.9.5] (0001_test_ch) 4.8061 (2.05) 4.8061 (2.05) 4.8061 (2.05) 0.0000 (1.0) 4.8061 (2.05) 0.0000 (1.0) 0;0 0.2081 (0.49) 1 1
test_checkout-checkout-noop[main] (0001_test_ch) 6.7814 (2.89) 6.7814 (2.89) 6.7814 (2.89) 0.0000 (1.0) 6.7814 (2.89) 0.0000 (1.0) 0;0 0.1475 (0.35) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-update': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update[2.10.2] (0001_test_ch) 30.1982 (3.77) 30.1982 (3.77) 30.1982 (3.77) 0.0000 (1.0) 30.1982 (3.77) 0.0000 (1.0) 0;0 0.0331 (0.27) 1 1
test_checkout-checkout-update[2.11.0] (0001_test_ch) 29.5748 (3.69) 29.5748 (3.69) 29.5748 (3.69) 0.0000 (1.0) 29.5748 (3.69) 0.0000 (1.0) 0;0 0.0338 (0.27) 1 1
test_checkout-checkout-update[2.6.3] (0001_test_ch) 69.6471 (8.69) 69.6471 (8.69) 69.6471 (8.69) 0.0000 (1.0) 69.6471 (8.69) 0.0000 (1.0) 0;0 0.0144 (0.12) 1 1
test_checkout-checkout-update[2.9.5] (0001_test_ch) 30.0799 (3.75) 30.0799 (3.75) 30.0799 (3.75) 0.0000 (1.0) 30.0799 (3.75) 0.0000 (1.0) 0;0 0.0332 (0.27) 1 1
test_checkout-checkout-update[main] (0001_test_ch) 8.0146 (1.0) 8.0146 (1.0) 8.0146 (1.0) 0.0000 (1.0) 8.0146 (1.0) 0.0000 (1.0) 0;0 0.1248 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.10.2] (0001_test_di) 44.6769 (1.57) 44.6769 (1.57) 44.6769 (1.57) 0.0000 (1.0) 44.6769 (1.57) 0.0000 (1.0) 0;0 0.0224 (0.64) 1 1
test_diff-diff[2.11.0] (0001_test_di) 39.8264 (1.40) 39.8264 (1.40) 39.8264 (1.40) 0.0000 (1.0) 39.8264 (1.40) 0.0000 (1.0) 0;0 0.0251 (0.72) 1 1
test_diff-diff[2.6.3] (0001_test_di) 80.5752 (2.83) 80.5752 (2.83) 80.5752 (2.83) 0.0000 (1.0) 80.5752 (2.83) 0.0000 (1.0) 0;0 0.0124 (0.35) 1 1
test_diff-diff[2.9.5] (0001_test_di) 43.6235 (1.53) 43.6235 (1.53) 43.6235 (1.53) 0.0000 (1.0) 43.6235 (1.53) 0.0000 (1.0) 0;0 0.0229 (0.65) 1 1
test_diff-diff[main] (0001_test_di) 28.5084 (1.0) 28.5084 (1.0) 28.5084 (1.0) 0.0000 (1.0) 28.5084 (1.0) 0.0000 (1.0) 0;0 0.0351 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.10.2] (0001_test_di) 26.6952 (4.81) 26.6952 (4.81) 26.6952 (4.81) 0.0000 (1.0) 26.6952 (4.81) 0.0000 (1.0) 0;0 0.0375 (0.21) 1 1
test_diff-diff-changed[2.11.0] (0001_test_di) 25.6949 (4.63) 25.6949 (4.63) 25.6949 (4.63) 0.0000 (1.0) 25.6949 (4.63) 0.0000 (1.0) 0;0 0.0389 (0.22) 1 1
test_diff-diff-changed[2.6.3] (0001_test_di) 60.6245 (10.91) 60.6245 (10.91) 60.6245 (10.91) 0.0000 (1.0) 60.6245 (10.91) 0.0000 (1.0) 0;0 0.0165 (0.09) 1 1
test_diff-diff-changed[2.9.5] (0001_test_di) 26.5177 (4.77) 26.5177 (4.77) 26.5177 (4.77) 0.0000 (1.0) 26.5177 (4.77) 0.0000 (1.0) 0;0 0.0377 (0.21) 1 1
test_diff-diff-changed[main] (0001_test_di) 5.5551 (1.0) 5.5551 (1.0) 5.5551 (1.0) 0.0000 (1.0) 5.5551 (1.0) 0.0000 (1.0) 0;0 0.1800 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.10.2] (0001_test_di) 26.5188 (9.67) 26.5188 (9.67) 26.5188 (9.67) 0.0000 (1.0) 26.5188 (9.67) 0.0000 (1.0) 0;0 0.0377 (0.10) 1 1
test_diff-diff-changed-noop[2.11.0] (0001_test_di) 2.7415 (1.0) 2.7415 (1.0) 2.7415 (1.0) 0.0000 (1.0) 2.7415 (1.0) 0.0000 (1.0) 0;0 0.3648 (1.0) 1 1
test_diff-diff-changed-noop[2.6.3] (0001_test_di) 60.7526 (22.16) 60.7526 (22.16) 60.7526 (22.16) 0.0000 (1.0) 60.7526 (22.16) 0.0000 (1.0) 0;0 0.0165 (0.05) 1 1
test_diff-diff-changed-noop[2.9.5] (0001_test_di) 26.5232 (9.67) 26.5232 (9.67) 26.5232 (9.67) 0.0000 (1.0) 26.5232 (9.67) 0.0000 (1.0) 0;0 0.0377 (0.10) 1 1
test_diff-diff-changed-noop[main] (0001_test_di) 5.4903 (2.00) 5.4903 (2.00) 5.4903 (2.00) 0.0000 (1.0) 5.4903 (2.00) 0.0000 (1.0) 0;0 0.1821 (0.50) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.10.2] (0001_test_di) 3.1026 (1.01) 3.1026 (1.01) 3.1026 (1.01) 0.0000 (1.0) 3.1026 (1.01) 0.0000 (1.0) 0;0 0.3223 (0.99) 1 1
test_diff-diff-noop[2.11.0] (0001_test_di) 3.0793 (1.0) 3.0793 (1.0) 3.0793 (1.0) 0.0000 (1.0) 3.0793 (1.0) 0.0000 (1.0) 0;0 0.3247 (1.0) 1 1
test_diff-diff-noop[2.6.3] (0001_test_di) 39.8219 (12.93) 39.8219 (12.93) 39.8219 (12.93) 0.0000 (1.0) 39.8219 (12.93) 0.0000 (1.0) 0;0 0.0251 (0.08) 1 1
test_diff-diff-noop[2.9.5] (0001_test_di) 3.0814 (1.00) 3.0814 (1.00) 3.0814 (1.00) 0.0000 (1.0) 3.0814 (1.00) 0.0000 (1.0) 0;0 0.3245 (1.00) 1 1
test_diff-diff-noop[main] (0001_test_di) 5.4668 (1.78) 5.4668 (1.78) 5.4668 (1.78) 0.0000 (1.0) 5.4668 (1.78) 0.0000 (1.0) 0;0 0.1829 (0.56) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 3 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.10.2] (0001_test_ex) 1.3453 (1.0) 1.3453 (1.0) 1.3453 (1.0) 0.0000 (1.0) 1.3453 (1.0) 0.0000 (1.0) 0;0 0.7433 (1.0) 1 1
test_exp_show-exp[2.11.0] (0001_test_ex) 1.3710 (1.02) 1.3710 (1.02) 1.3710 (1.02) 0.0000 (1.0) 1.3710 (1.02) 0.0000 (1.0) 0;0 0.7294 (0.98) 1 1
test_exp_show-exp[main] (0001_test_ex) 1.4132 (1.05) 1.4132 (1.05) 1.4132 (1.05) 0.0000 (1.0) 1.4132 (1.05) 0.0000 (1.0) 0;0 0.7076 (0.95) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_gc-gc': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.10.2] (0001_test_gc) 2.8776 (1.19) 2.8776 (1.19) 2.8776 (1.19) 0.0000 (1.0) 2.8776 (1.19) 0.0000 (1.0) 0;0 0.3475 (0.84) 1 1
test_gc-gc[2.11.0] (0001_test_gc) 2.4134 (1.0) 2.4134 (1.0) 2.4134 (1.0) 0.0000 (1.0) 2.4134 (1.0) 0.0000 (1.0) 0;0 0.4144 (1.0) 1 1
test_gc-gc[2.6.3] (0001_test_gc) 3.3581 (1.39) 3.3581 (1.39) 3.3581 (1.39) 0.0000 (1.0) 3.3581 (1.39) 0.0000 (1.0) 0;0 0.2978 (0.72) 1 1
test_gc-gc[2.9.5] (0001_test_gc) 2.9521 (1.22) 2.9521 (1.22) 2.9521 (1.22) 0.0000 (1.0) 2.9521 (1.22) 0.0000 (1.0) 0;0 0.3387 (0.82) 1 1
test_gc-gc[main] (0001_test_gc) 2.5545 (1.06) 2.5545 (1.06) 2.5545 (1.06) 0.0000 (1.0) 2.5545 (1.06) 0.0000 (1.0) 0;0 0.3915 (0.94) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_get-get': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.10.2] (0001_test_ge) 255.5104 (2.00) 255.5104 (2.00) 255.5104 (2.00) 0.0000 (1.0) 255.5104 (2.00) 0.0000 (1.0) 0;0 0.0039 (0.50) 1 1
test_get-get[2.11.0] (0001_test_ge) 204.9277 (1.61) 204.9277 (1.61) 204.9277 (1.61) 0.0000 (1.0) 204.9277 (1.61) 0.0000 (1.0) 0;0 0.0049 (0.62) 1 1
test_get-get[2.6.3] (0001_test_ge) 149.1962 (1.17) 149.1962 (1.17) 149.1962 (1.17) 0.0000 (1.0) 149.1962 (1.17) 0.0000 (1.0) 0;0 0.0067 (0.86) 1 1
test_get-get[2.9.5] (0001_test_ge) 127.6205 (1.0) 127.6205 (1.0) 127.6205 (1.0) 0.0000 (1.0) 127.6205 (1.0) 0.0000 (1.0) 0;0 0.0078 (1.0) 1 1
test_get-get[main] (0001_test_ge) 203.0147 (1.59) 203.0147 (1.59) 203.0147 (1.59) 0.0000 (1.0) 203.0147 (1.59) 0.0000 (1.0) 0;0 0.0049 (0.63) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_help---help': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.10.2] (0001_test_he) 144.5984 (1.00) 150.0778 (1.0) 147.4534 (1.00) 1.0743 (1.0) 147.3024 (1.00) 1.2178 (1.0) 31;4 6.7818 (1.00) 100 1
test_help---help[2.11.0] (0001_test_he) 145.4752 (1.01) 150.2014 (1.00) 147.3947 (1.00) 1.1326 (1.05) 147.2350 (1.0) 1.4341 (1.18) 28;0 6.7845 (1.00) 100 1
test_help---help[2.6.3] (0001_test_he) 189.6040 (1.31) 195.2199 (1.30) 191.8384 (1.30) 1.1355 (1.06) 191.7516 (1.30) 1.3142 (1.08) 25;4 5.2127 (0.77) 100 1
test_help---help[2.9.5] (0001_test_he) 145.6003 (1.01) 152.7000 (1.02) 147.9084 (1.00) 1.0984 (1.02) 147.7378 (1.00) 1.2380 (1.02) 27;2 6.7609 (1.00) 100 1
test_help---help[main] (0001_test_he) 144.3551 (1.0) 150.2578 (1.00) 147.1982 (1.0) 1.1255 (1.05) 147.3554 (1.00) 1.5883 (1.30) 31;0 6.7936 (1.0) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_import-import': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.10.2] (0001_test_im) 429.0465 (1.40) 429.0465 (1.40) 429.0465 (1.40) 0.0000 (1.0) 429.0465 (1.40) 0.0000 (1.0) 0;0 0.0023 (0.71) 1 1
test_import-import[2.11.0] (0001_test_im) 387.7582 (1.27) 387.7582 (1.27) 387.7582 (1.27) 0.0000 (1.0) 387.7582 (1.27) 0.0000 (1.0) 0;0 0.0026 (0.79) 1 1
test_import-import[2.6.3] (0001_test_im) 305.5136 (1.0) 305.5136 (1.0) 305.5136 (1.0) 0.0000 (1.0) 305.5136 (1.0) 0.0000 (1.0) 0;0 0.0033 (1.0) 1 1
test_import-import[2.9.5] (0001_test_im) 456.8004 (1.50) 456.8004 (1.50) 456.8004 (1.50) 0.0000 (1.0) 456.8004 (1.50) 0.0000 (1.0) 0;0 0.0022 (0.67) 1 1
test_import-import[main] (0001_test_im) 368.0077 (1.20) 368.0077 (1.20) 368.0077 (1.20) 0.0000 (1.0) 368.0077 (1.20) 0.0000 (1.0) 0;0 0.0027 (0.83) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_init-init': 5 tests -------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.10.2] (0001_test_in) 408.0608 (1.05) 535.0870 (1.05) 432.9643 (1.04) 16.6977 (1.18) 429.9892 (1.05) 17.1675 (1.22) 19;3 2.3097 (0.96) 100 1
test_init-init[2.11.0] (0001_test_in) 414.8647 (1.06) 508.6714 (1.0) 433.9259 (1.04) 14.1301 (1.0) 430.9149 (1.05) 14.0280 (1.0) 20;5 2.3045 (0.96) 100 1
test_init-init[2.6.3] (0001_test_in) 390.3895 (1.0) 533.4600 (1.05) 416.2293 (1.0) 20.0238 (1.42) 411.2840 (1.0) 22.3988 (1.60) 24;2 2.4025 (1.0) 100 1
test_init-init[2.9.5] (0001_test_in) 415.2601 (1.06) 573.6417 (1.13) 436.9612 (1.05) 20.6415 (1.46) 432.4324 (1.05) 16.6579 (1.19) 7;4 2.2885 (0.95) 100 1
test_init-init[main] (0001_test_in) 416.5123 (1.07) 512.5475 (1.01) 450.6655 (1.08) 23.9906 (1.70) 444.2151 (1.08) 28.4096 (2.03) 27;7 2.2189 (0.92) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_list-list': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.10.2] (0001_test_li) 2.2640 (3.97) 2.2640 (3.97) 2.2640 (3.97) 0.0000 (1.0) 2.2640 (3.97) 0.0000 (1.0) 0;0 0.4417 (0.25) 1 1
test_list-list[2.11.0] (0001_test_li) 2.2389 (3.93) 2.2389 (3.93) 2.2389 (3.93) 0.0000 (1.0) 2.2389 (3.93) 0.0000 (1.0) 0;0 0.4467 (0.25) 1 1
test_list-list[2.6.3] (0001_test_li) 0.5704 (1.0) 0.5704 (1.0) 0.5704 (1.0) 0.0000 (1.0) 0.5704 (1.0) 0.0000 (1.0) 0;0 1.7533 (1.0) 1 1
test_list-list[2.9.5] (0001_test_li) 2.5088 (4.40) 2.5088 (4.40) 2.5088 (4.40) 0.0000 (1.0) 2.5088 (4.40) 0.0000 (1.0) 0;0 0.3986 (0.23) 1 1
test_list-list[main] (0001_test_li) 2.2583 (3.96) 2.2583 (3.96) 2.2583 (3.96) 0.0000 (1.0) 2.2583 (3.96) 0.0000 (1.0) 0;0 0.4428 (0.25) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_list-list-dvc-only': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.10.2] (0001_test_li) 2.2561 (5.36) 2.2561 (5.36) 2.2561 (5.36) 0.0000 (1.0) 2.2561 (5.36) 0.0000 (1.0) 0;0 0.4432 (0.19) 1 1
test_list-list-dvc-only[2.11.0] (0001_test_li) 2.2329 (5.31) 2.2329 (5.31) 2.2329 (5.31) 0.0000 (1.0) 2.2329 (5.31) 0.0000 (1.0) 0;0 0.4478 (0.19) 1 1
test_list-list-dvc-only[2.6.3] (0001_test_li) 0.4208 (1.0) 0.4208 (1.0) 0.4208 (1.0) 0.0000 (1.0) 0.4208 (1.0) 0.0000 (1.0) 0;0 2.3764 (1.0) 1 1
test_list-list-dvc-only[2.9.5] (0001_test_li) 2.2358 (5.31) 2.2358 (5.31) 2.2358 (5.31) 0.0000 (1.0) 2.2358 (5.31) 0.0000 (1.0) 0;0 0.4473 (0.19) 1 1
test_list-list-dvc-only[main] (0001_test_li) 1.9954 (4.74) 1.9954 (4.74) 1.9954 (4.74) 0.0000 (1.0) 1.9954 (4.74) 0.0000 (1.0) 0;0 0.5011 (0.21) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_plots_show-plots': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots_show-plots[2.10.2] (0001_test_pl) 799.3388 (1.26) 799.3388 (1.26) 799.3388 (1.26) 0.0000 (1.0) 799.3388 (1.26) 0.0000 (1.0) 0;0 1.2510 (0.79) 1 1
test_plots_show-plots[2.11.0] (0001_test_pl) 983.2387 (1.55) 983.2387 (1.55) 983.2387 (1.55) 0.0000 (1.0) 983.2387 (1.55) 0.0000 (1.0) 0;0 1.0170 (0.64) 1 1
test_plots_show-plots[2.6.3] (0001_test_pl) 633.3506 (1.0) 633.3506 (1.0) 633.3506 (1.0) 0.0000 (1.0) 633.3506 (1.0) 0.0000 (1.0) 0;0 1.5789 (1.0) 1 1
test_plots_show-plots[2.9.5] (0001_test_pl) 1,092.3357 (1.72) 1,092.3357 (1.72) 1,092.3357 (1.72) 0.0000 (1.0) 1,092.3357 (1.72) 0.0000 (1.0) 0;0 0.9155 (0.58) 1 1
test_plots_show-plots[main] (0001_test_pl) 1,395.5103 (2.20) 1,395.5103 (2.20) 1,395.5103 (2.20) 0.0000 (1.0) 1,395.5103 (2.20) 0.0000 (1.0) 0;0 0.7166 (0.45) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_pull-pull': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.10.2] (0001_test_pu) 334.9837 (1.06) 334.9837 (1.06) 334.9837 (1.06) 0.0000 (1.0) 334.9837 (1.06) 0.0000 (1.0) 0;0 0.0030 (0.94) 1 1
test_pull-pull[2.11.0] (0001_test_pu) 332.8868 (1.06) 332.8868 (1.06) 332.8868 (1.06) 0.0000 (1.0) 332.8868 (1.06) 0.0000 (1.0) 0;0 0.0030 (0.95) 1 1
test_pull-pull[2.6.3] (0001_test_pu) 314.8623 (1.0) 314.8623 (1.0) 314.8623 (1.0) 0.0000 (1.0) 314.8623 (1.0) 0.0000 (1.0) 0;0 0.0032 (1.0) 1 1
test_pull-pull[2.9.5] (0001_test_pu) 369.5347 (1.17) 369.5347 (1.17) 369.5347 (1.17) 0.0000 (1.0) 369.5347 (1.17) 0.0000 (1.0) 0;0 0.0027 (0.85) 1 1
test_pull-pull[main] (0001_test_pu) 333.9752 (1.06) 333.9752 (1.06) 333.9752 (1.06) 0.0000 (1.0) 333.9752 (1.06) 0.0000 (1.0) 0;0 0.0030 (0.94) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_push-push': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.10.2] (0001_test_pu) 127.3040 (1.22) 127.3040 (1.22) 127.3040 (1.22) 0.0000 (1.0) 127.3040 (1.22) 0.0000 (1.0) 0;0 0.0079 (0.82) 1 1
test_push-push[2.11.0] (0001_test_pu) 128.3662 (1.23) 128.3662 (1.23) 128.3662 (1.23) 0.0000 (1.0) 128.3662 (1.23) 0.0000 (1.0) 0;0 0.0078 (0.81) 1 1
test_push-push[2.6.3] (0001_test_pu) 104.1334 (1.0) 104.1334 (1.0) 104.1334 (1.0) 0.0000 (1.0) 104.1334 (1.0) 0.0000 (1.0) 0;0 0.0096 (1.0) 1 1
test_push-push[2.9.5] (0001_test_pu) 141.8735 (1.36) 141.8735 (1.36) 141.8735 (1.36) 0.0000 (1.0) 141.8735 (1.36) 0.0000 (1.0) 0;0 0.0070 (0.73) 1 1
test_push-push[main] (0001_test_pu) 125.4696 (1.20) 125.4696 (1.20) 125.4696 (1.20) 0.0000 (1.0) 125.4696 (1.20) 0.0000 (1.0) 0;0 0.0080 (0.83) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.10.2] (0001_test_sh) 187.3565 (1.09) 187.3565 (1.09) 187.3565 (1.09) 0.0000 (1.0) 187.3565 (1.09) 0.0000 (1.0) 0;0 0.0053 (0.92) 1 1
test_sharing-add[2.11.0] (0001_test_sh) 185.8942 (1.08) 185.8942 (1.08) 185.8942 (1.08) 0.0000 (1.0) 185.8942 (1.08) 0.0000 (1.0) 0;0 0.0054 (0.93) 1 1
test_sharing-add[2.6.3] (0001_test_sh) 214.5056 (1.25) 214.5056 (1.25) 214.5056 (1.25) 0.0000 (1.0) 214.5056 (1.25) 0.0000 (1.0) 0;0 0.0047 (0.80) 1 1
test_sharing-add[2.9.5] (0001_test_sh) 230.4205 (1.34) 230.4205 (1.34) 230.4205 (1.34) 0.0000 (1.0) 230.4205 (1.34) 0.0000 (1.0) 0;0 0.0043 (0.75) 1 1
test_sharing-add[main] (0001_test_sh) 172.1756 (1.0) 172.1756 (1.0) 172.1756 (1.0) 0.0000 (1.0) 172.1756 (1.0) 0.0000 (1.0) 0;0 0.0058 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.10.2] (0001_test_sh) 98.0001 (6.17) 98.0001 (6.17) 98.0001 (6.17) 0.0000 (1.0) 98.0001 (6.17) 0.0000 (1.0) 0;0 0.0102 (0.16) 1 1
test_sharing-add-noop[2.11.0] (0001_test_sh) 95.2286 (6.00) 95.2286 (6.00) 95.2286 (6.00) 0.0000 (1.0) 95.2286 (6.00) 0.0000 (1.0) 0;0 0.0105 (0.17) 1 1
test_sharing-add-noop[2.6.3] (0001_test_sh) 15.8825 (1.0) 15.8825 (1.0) 15.8825 (1.0) 0.0000 (1.0) 15.8825 (1.0) 0.0000 (1.0) 0;0 0.0630 (1.0) 1 1
test_sharing-add-noop[2.9.5] (0001_test_sh) 105.1775 (6.62) 105.1775 (6.62) 105.1775 (6.62) 0.0000 (1.0) 105.1775 (6.62) 0.0000 (1.0) 0;0 0.0095 (0.15) 1 1
test_sharing-add-noop[main] (0001_test_sh) 108.2297 (6.81) 108.2297 (6.81) 108.2297 (6.81) 0.0000 (1.0) 108.2297 (6.81) 0.0000 (1.0) 0;0 0.0092 (0.15) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.10.2] (0001_test_sh) 4.7445 (2.02) 4.7445 (2.02) 4.7445 (2.02) 0.0000 (1.0) 4.7445 (2.02) 0.0000 (1.0) 0;0 0.2108 (0.49) 1 1
test_sharing-checkout-noop[2.11.0] (0001_test_sh) 4.6231 (1.97) 4.6231 (1.97) 4.6231 (1.97) 0.0000 (1.0) 4.6231 (1.97) 0.0000 (1.0) 0;0 0.2163 (0.51) 1 1
test_sharing-checkout-noop[2.6.3] (0001_test_sh) 2.3459 (1.0) 2.3459 (1.0) 2.3459 (1.0) 0.0000 (1.0) 2.3459 (1.0) 0.0000 (1.0) 0;0 0.4263 (1.0) 1 1
test_sharing-checkout-noop[2.9.5] (0001_test_sh) 4.7463 (2.02) 4.7463 (2.02) 4.7463 (2.02) 0.0000 (1.0) 4.7463 (2.02) 0.0000 (1.0) 0;0 0.2107 (0.49) 1 1
test_sharing-checkout-noop[main] (0001_test_sh) 6.7108 (2.86) 6.7108 (2.86) 6.7108 (2.86) 0.0000 (1.0) 6.7108 (2.86) 0.0000 (1.0) 0;0 0.1490 (0.35) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.10.2] (0001_test_sh) 206.6486 (1.23) 206.6486 (1.23) 206.6486 (1.23) 0.0000 (1.0) 206.6486 (1.23) 0.0000 (1.0) 0;0 0.0048 (0.81) 1 1
test_sharing-pull[2.11.0] (0001_test_sh) 208.7814 (1.24) 208.7814 (1.24) 208.7814 (1.24) 0.0000 (1.0) 208.7814 (1.24) 0.0000 (1.0) 0;0 0.0048 (0.81) 1 1
test_sharing-pull[2.6.3] (0001_test_sh) 168.1650 (1.0) 168.1650 (1.0) 168.1650 (1.0) 0.0000 (1.0) 168.1650 (1.0) 0.0000 (1.0) 0;0 0.0059 (1.0) 1 1
test_sharing-pull[2.9.5] (0001_test_sh) 228.5996 (1.36) 228.5996 (1.36) 228.5996 (1.36) 0.0000 (1.0) 228.5996 (1.36) 0.0000 (1.0) 0;0 0.0044 (0.74) 1 1
test_sharing-pull[main] (0001_test_sh) 218.8905 (1.30) 218.8905 (1.30) 218.8905 (1.30) 0.0000 (1.0) 218.8905 (1.30) 0.0000 (1.0) 0;0 0.0046 (0.77) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.10.2] (0001_test_sh) 5.5189 (1.80) 5.5189 (1.80) 5.5189 (1.80) 0.0000 (1.0) 5.5189 (1.80) 0.0000 (1.0) 0;0 0.1812 (0.56) 1 1
test_sharing-pull-noop[2.11.0] (0001_test_sh) 5.3606 (1.75) 5.3606 (1.75) 5.3606 (1.75) 0.0000 (1.0) 5.3606 (1.75) 0.0000 (1.0) 0;0 0.1865 (0.57) 1 1
test_sharing-pull-noop[2.6.3] (0001_test_sh) 3.0672 (1.0) 3.0672 (1.0) 3.0672 (1.0) 0.0000 (1.0) 3.0672 (1.0) 0.0000 (1.0) 0;0 0.3260 (1.0) 1 1
test_sharing-pull-noop[2.9.5] (0001_test_sh) 5.3856 (1.76) 5.3856 (1.76) 5.3856 (1.76) 0.0000 (1.0) 5.3856 (1.76) 0.0000 (1.0) 0;0 0.1857 (0.57) 1 1
test_sharing-pull-noop[main] (0001_test_sh) 7.3756 (2.40) 7.3756 (2.40) 7.3756 (2.40) 0.0000 (1.0) 7.3756 (2.40) 0.0000 (1.0) 0;0 0.1356 (0.42) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.10.2] (0001_test_sh) 120.0760 (1.23) 120.0760 (1.23) 120.0760 (1.23) 0.0000 (1.0) 120.0760 (1.23) 0.0000 (1.0) 0;0 0.0083 (0.81) 1 1
test_sharing-push[2.11.0] (0001_test_sh) 118.3673 (1.21) 118.3673 (1.21) 118.3673 (1.21) 0.0000 (1.0) 118.3673 (1.21) 0.0000 (1.0) 0;0 0.0084 (0.83) 1 1
test_sharing-push[2.6.3] (0001_test_sh) 97.7788 (1.0) 97.7788 (1.0) 97.7788 (1.0) 0.0000 (1.0) 97.7788 (1.0) 0.0000 (1.0) 0;0 0.0102 (1.0) 1 1
test_sharing-push[2.9.5] (0001_test_sh) 132.0507 (1.35) 132.0507 (1.35) 132.0507 (1.35) 0.0000 (1.0) 132.0507 (1.35) 0.0000 (1.0) 0;0 0.0076 (0.74) 1 1
test_sharing-push[main] (0001_test_sh) 118.6483 (1.21) 118.6483 (1.21) 118.6483 (1.21) 0.0000 (1.0) 118.6483 (1.21) 0.0000 (1.0) 0;0 0.0084 (0.82) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.10.2] (0001_test_sh) 2.3701 (1.00) 2.3701 (1.00) 2.3701 (1.00) 0.0000 (1.0) 2.3701 (1.00) 0.0000 (1.0) 0;0 0.4219 (1.00) 1 1
test_sharing-push-noop[2.11.0] (0001_test_sh) 2.4948 (1.05) 2.4948 (1.05) 2.4948 (1.05) 0.0000 (1.0) 2.4948 (1.05) 0.0000 (1.0) 0;0 0.4008 (0.95) 1 1
test_sharing-push-noop[2.6.3] (0001_test_sh) 2.7045 (1.14) 2.7045 (1.14) 2.7045 (1.14) 0.0000 (1.0) 2.7045 (1.14) 0.0000 (1.0) 0;0 0.3698 (0.88) 1 1
test_sharing-push-noop[2.9.5] (0001_test_sh) 2.3970 (1.01) 2.3970 (1.01) 2.3970 (1.01) 0.0000 (1.0) 2.3970 (1.01) 0.0000 (1.0) 0;0 0.4172 (0.99) 1 1
test_sharing-push-noop[main] (0001_test_sh) 2.3689 (1.0) 2.3689 (1.0) 2.3689 (1.0) 0.0000 (1.0) 2.3689 (1.0) 0.0000 (1.0) 0;0 0.4221 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_status-status': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.10.2] (0001_test_st) 76.3826 (2.04) 76.3826 (2.04) 76.3826 (2.04) 0.0000 (1.0) 76.3826 (2.04) 0.0000 (1.0) 0;0 0.0131 (0.49) 1 1
test_status-status[2.11.0] (0001_test_st) 71.5208 (1.91) 71.5208 (1.91) 71.5208 (1.91) 0.0000 (1.0) 71.5208 (1.91) 0.0000 (1.0) 0;0 0.0140 (0.52) 1 1
test_status-status[2.6.3] (0001_test_st) 126.9031 (3.40) 126.9031 (3.40) 126.9031 (3.40) 0.0000 (1.0) 126.9031 (3.40) 0.0000 (1.0) 0;0 0.0079 (0.29) 1 1
test_status-status[2.9.5] (0001_test_st) 119.6444 (3.20) 119.6444 (3.20) 119.6444 (3.20) 0.0000 (1.0) 119.6444 (3.20) 0.0000 (1.0) 0;0 0.0084 (0.31) 1 1
test_status-status[main] (0001_test_st) 37.3537 (1.0) 37.3537 (1.0) 37.3537 (1.0) 0.0000 (1.0) 37.3537 (1.0) 0.0000 (1.0) 0;0 0.0268 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_status-status-changed': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.10.2] (0001_test_st) 50.0997 (5.74) 50.0997 (5.74) 50.0997 (5.74) 0.0000 (1.0) 50.0997 (5.74) 0.0000 (1.0) 0;0 0.0200 (0.17) 1 1
test_status-status-changed[2.11.0] (0001_test_st) 49.6090 (5.69) 49.6090 (5.69) 49.6090 (5.69) 0.0000 (1.0) 49.6090 (5.69) 0.0000 (1.0) 0;0 0.0202 (0.18) 1 1
test_status-status-changed[2.6.3] (0001_test_st) 104.4149 (11.97) 104.4149 (11.97) 104.4149 (11.97) 0.0000 (1.0) 104.4149 (11.97) 0.0000 (1.0) 0;0 0.0096 (0.08) 1 1
test_status-status-changed[2.9.5] (0001_test_st) 96.1066 (11.02) 96.1066 (11.02) 96.1066 (11.02) 0.0000 (1.0) 96.1066 (11.02) 0.0000 (1.0) 0;0 0.0104 (0.09) 1 1
test_status-status-changed[main] (0001_test_st) 8.7219 (1.0) 8.7219 (1.0) 8.7219 (1.0) 0.0000 (1.0) 8.7219 (1.0) 0.0000 (1.0) 0;0 0.1147 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_status-status-changed-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.10.2] (0001_test_st) 50.4673 (5.75) 50.4673 (5.75) 50.4673 (5.75) 0.0000 (1.0) 50.4673 (5.75) 0.0000 (1.0) 0;0 0.0198 (0.17) 1 1
test_status-status-changed-noop[2.11.0] (0001_test_st) 12.7746 (1.46) 12.7746 (1.46) 12.7746 (1.46) 0.0000 (1.0) 12.7746 (1.46) 0.0000 (1.0) 0;0 0.0783 (0.69) 1 1
test_status-status-changed-noop[2.6.3] (0001_test_st) 102.2937 (11.65) 102.2937 (11.65) 102.2937 (11.65) 0.0000 (1.0) 102.2937 (11.65) 0.0000 (1.0) 0;0 0.0098 (0.09) 1 1
test_status-status-changed-noop[2.9.5] (0001_test_st) 93.7182 (10.68) 93.7182 (10.68) 93.7182 (10.68) 0.0000 (1.0) 93.7182 (10.68) 0.0000 (1.0) 0;0 0.0107 (0.09) 1 1
test_status-status-changed-noop[main] (0001_test_st) 8.7776 (1.0) 8.7776 (1.0) 8.7776 (1.0) 0.0000 (1.0) 8.7776 (1.0) 0.0000 (1.0) 0;0 0.1139 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_status-status-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.10.2] (0001_test_st) 3.0117 (1.03) 3.0117 (1.03) 3.0117 (1.03) 0.0000 (1.0) 3.0117 (1.03) 0.0000 (1.0) 0;0 0.3320 (0.98) 1 1
test_status-status-noop[2.11.0] (0001_test_st) 2.9369 (1.0) 2.9369 (1.0) 2.9369 (1.0) 0.0000 (1.0) 2.9369 (1.0) 0.0000 (1.0) 0;0 0.3405 (1.0) 1 1
test_status-status-noop[2.6.3] (0001_test_st) 4.1981 (1.43) 4.1981 (1.43) 4.1981 (1.43) 0.0000 (1.0) 4.1981 (1.43) 0.0000 (1.0) 0;0 0.2382 (0.70) 1 1
test_status-status-noop[2.9.5] (0001_test_st) 3.0325 (1.03) 3.0325 (1.03) 3.0325 (1.03) 0.0000 (1.0) 3.0325 (1.03) 0.0000 (1.0) 0;0 0.3298 (0.97) 1 1
test_status-status-noop[main] (0001_test_st) 8.3578 (2.85) 8.3578 (2.85) 8.3578 (2.85) 0.0000 (1.0) 8.3578 (2.85) 0.0000 (1.0) 0;0 0.1196 (0.35) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```
github-actions[bot] commented 2 years ago 

dataset: mnist
project: example-get-started

----------------------------------------------------------------------------------- benchmark 'test_add-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.10.2] (0001_test_ad) 215.1774 (1.03) 215.1774 (1.03) 215.1774 (1.03) 0.0000 (1.0) 215.1774 (1.03) 0.0000 (1.0) 0;0 0.0046 (0.97) 1 1
test_add-add[2.11.0] (0001_test_ad) 214.2072 (1.02) 214.2072 (1.02) 214.2072 (1.02) 0.0000 (1.0) 214.2072 (1.02) 0.0000 (1.0) 0;0 0.0047 (0.98) 1 1
test_add-add[2.6.3] (0001_test_ad) 240.6357 (1.15) 240.6357 (1.15) 240.6357 (1.15) 0.0000 (1.0) 240.6357 (1.15) 0.0000 (1.0) 0;0 0.0042 (0.87) 1 1
test_add-add[2.9.5] (0001_test_ad) 263.3863 (1.26) 263.3863 (1.26) 263.3863 (1.26) 0.0000 (1.0) 263.3863 (1.26) 0.0000 (1.0) 0;0 0.0038 (0.80) 1 1
test_add-add[main] (0001_test_ad) 209.7501 (1.0) 209.7501 (1.0) 209.7501 (1.0) 0.0000 (1.0) 209.7501 (1.0) 0.0000 (1.0) 0;0 0.0048 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_checkout-checkout': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout[2.10.2] (0001_test_ch) 75.0261 (1.43) 75.0261 (1.43) 75.0261 (1.43) 0.0000 (1.0) 75.0261 (1.43) 0.0000 (1.0) 0;0 0.0133 (0.70) 1 1
test_checkout-checkout[2.11.0] (0001_test_ch) 74.0059 (1.41) 74.0059 (1.41) 74.0059 (1.41) 0.0000 (1.0) 74.0059 (1.41) 0.0000 (1.0) 0;0 0.0135 (0.71) 1 1
test_checkout-checkout[2.6.3] (0001_test_ch) 52.5684 (1.0) 52.5684 (1.0) 52.5684 (1.0) 0.0000 (1.0) 52.5684 (1.0) 0.0000 (1.0) 0;0 0.0190 (1.0) 1 1
test_checkout-checkout[2.9.5] (0001_test_ch) 85.4837 (1.63) 85.4837 (1.63) 85.4837 (1.63) 0.0000 (1.0) 85.4837 (1.63) 0.0000 (1.0) 0;0 0.0117 (0.61) 1 1
test_checkout-checkout[main] (0001_test_ch) 78.2785 (1.49) 78.2785 (1.49) 78.2785 (1.49) 0.0000 (1.0) 78.2785 (1.49) 0.0000 (1.0) 0;0 0.0128 (0.67) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop[2.10.2] (0001_test_ch) 4.6061 (2.04) 4.6061 (2.04) 4.6061 (2.04) 0.0000 (1.0) 4.6061 (2.04) 0.0000 (1.0) 0;0 0.2171 (0.49) 1 1
test_checkout-checkout-noop[2.11.0] (0001_test_ch) 4.5621 (2.02) 4.5621 (2.02) 4.5621 (2.02) 0.0000 (1.0) 4.5621 (2.02) 0.0000 (1.0) 0;0 0.2192 (0.50) 1 1
test_checkout-checkout-noop[2.6.3] (0001_test_ch) 2.2610 (1.0) 2.2610 (1.0) 2.2610 (1.0) 0.0000 (1.0) 2.2610 (1.0) 0.0000 (1.0) 0;0 0.4423 (1.0) 1 1
test_checkout-checkout-noop[2.9.5] (0001_test_ch) 4.6188 (2.04) 4.6188 (2.04) 4.6188 (2.04) 0.0000 (1.0) 4.6188 (2.04) 0.0000 (1.0) 0;0 0.2165 (0.49) 1 1
test_checkout-checkout-noop[main] (0001_test_ch) 6.5960 (2.92) 6.5960 (2.92) 6.5960 (2.92) 0.0000 (1.0) 6.5960 (2.92) 0.0000 (1.0) 0;0 0.1516 (0.34) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-update': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update[2.10.2] (0001_test_ch) 29.1497 (3.77) 29.1497 (3.77) 29.1497 (3.77) 0.0000 (1.0) 29.1497 (3.77) 0.0000 (1.0) 0;0 0.0343 (0.27) 1 1
test_checkout-checkout-update[2.11.0] (0001_test_ch) 28.6612 (3.71) 28.6612 (3.71) 28.6612 (3.71) 0.0000 (1.0) 28.6612 (3.71) 0.0000 (1.0) 0;0 0.0349 (0.27) 1 1
test_checkout-checkout-update[2.6.3] (0001_test_ch) 68.0895 (8.80) 68.0895 (8.80) 68.0895 (8.80) 0.0000 (1.0) 68.0895 (8.80) 0.0000 (1.0) 0;0 0.0147 (0.11) 1 1
test_checkout-checkout-update[2.9.5] (0001_test_ch) 29.1169 (3.76) 29.1169 (3.76) 29.1169 (3.76) 0.0000 (1.0) 29.1169 (3.76) 0.0000 (1.0) 0;0 0.0343 (0.27) 1 1
test_checkout-checkout-update[main] (0001_test_ch) 7.7342 (1.0) 7.7342 (1.0) 7.7342 (1.0) 0.0000 (1.0) 7.7342 (1.0) 0.0000 (1.0) 0;0 0.1293 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.10.2] (0001_test_di) 44.1555 (1.64) 44.1555 (1.64) 44.1555 (1.64) 0.0000 (1.0) 44.1555 (1.64) 0.0000 (1.0) 0;0 0.0226 (0.61) 1 1
test_diff-diff[2.11.0] (0001_test_di) 39.7182 (1.48) 39.7182 (1.48) 39.7182 (1.48) 0.0000 (1.0) 39.7182 (1.48) 0.0000 (1.0) 0;0 0.0252 (0.68) 1 1
test_diff-diff[2.6.3] (0001_test_di) 80.9793 (3.01) 80.9793 (3.01) 80.9793 (3.01) 0.0000 (1.0) 80.9793 (3.01) 0.0000 (1.0) 0;0 0.0123 (0.33) 1 1
test_diff-diff[2.9.5] (0001_test_di) 44.3958 (1.65) 44.3958 (1.65) 44.3958 (1.65) 0.0000 (1.0) 44.3958 (1.65) 0.0000 (1.0) 0;0 0.0225 (0.61) 1 1
test_diff-diff[main] (0001_test_di) 26.8886 (1.0) 26.8886 (1.0) 26.8886 (1.0) 0.0000 (1.0) 26.8886 (1.0) 0.0000 (1.0) 0;0 0.0372 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.10.2] (0001_test_di) 26.4380 (4.76) 26.4380 (4.76) 26.4380 (4.76) 0.0000 (1.0) 26.4380 (4.76) 0.0000 (1.0) 0;0 0.0378 (0.21) 1 1
test_diff-diff-changed[2.11.0] (0001_test_di) 25.7909 (4.64) 25.7909 (4.64) 25.7909 (4.64) 0.0000 (1.0) 25.7909 (4.64) 0.0000 (1.0) 0;0 0.0388 (0.22) 1 1
test_diff-diff-changed[2.6.3] (0001_test_di) 60.7682 (10.93) 60.7682 (10.93) 60.7682 (10.93) 0.0000 (1.0) 60.7682 (10.93) 0.0000 (1.0) 0;0 0.0165 (0.09) 1 1
test_diff-diff-changed[2.9.5] (0001_test_di) 26.7379 (4.81) 26.7379 (4.81) 26.7379 (4.81) 0.0000 (1.0) 26.7379 (4.81) 0.0000 (1.0) 0;0 0.0374 (0.21) 1 1
test_diff-diff-changed[main] (0001_test_di) 5.5578 (1.0) 5.5578 (1.0) 5.5578 (1.0) 0.0000 (1.0) 5.5578 (1.0) 0.0000 (1.0) 0;0 0.1799 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.10.2] (0001_test_di) 26.8498 (9.91) 26.8498 (9.91) 26.8498 (9.91) 0.0000 (1.0) 26.8498 (9.91) 0.0000 (1.0) 0;0 0.0372 (0.10) 1 1
test_diff-diff-changed-noop[2.11.0] (0001_test_di) 2.7085 (1.0) 2.7085 (1.0) 2.7085 (1.0) 0.0000 (1.0) 2.7085 (1.0) 0.0000 (1.0) 0;0 0.3692 (1.0) 1 1
test_diff-diff-changed-noop[2.6.3] (0001_test_di) 60.9090 (22.49) 60.9090 (22.49) 60.9090 (22.49) 0.0000 (1.0) 60.9090 (22.49) 0.0000 (1.0) 0;0 0.0164 (0.04) 1 1
test_diff-diff-changed-noop[2.9.5] (0001_test_di) 26.5054 (9.79) 26.5054 (9.79) 26.5054 (9.79) 0.0000 (1.0) 26.5054 (9.79) 0.0000 (1.0) 0;0 0.0377 (0.10) 1 1
test_diff-diff-changed-noop[main] (0001_test_di) 5.4870 (2.03) 5.4870 (2.03) 5.4870 (2.03) 0.0000 (1.0) 5.4870 (2.03) 0.0000 (1.0) 0;0 0.1822 (0.49) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.10.2] (0001_test_di) 3.1534 (1.03) 3.1534 (1.03) 3.1534 (1.03) 0.0000 (1.0) 3.1534 (1.03) 0.0000 (1.0) 0;0 0.3171 (0.97) 1 1
test_diff-diff-noop[2.11.0] (0001_test_di) 3.0473 (1.0) 3.0473 (1.0) 3.0473 (1.0) 0.0000 (1.0) 3.0473 (1.0) 0.0000 (1.0) 0;0 0.3282 (1.0) 1 1
test_diff-diff-noop[2.6.3] (0001_test_di) 39.7500 (13.04) 39.7500 (13.04) 39.7500 (13.04) 0.0000 (1.0) 39.7500 (13.04) 0.0000 (1.0) 0;0 0.0252 (0.08) 1 1
test_diff-diff-noop[2.9.5] (0001_test_di) 3.0792 (1.01) 3.0792 (1.01) 3.0792 (1.01) 0.0000 (1.0) 3.0792 (1.01) 0.0000 (1.0) 0;0 0.3248 (0.99) 1 1
test_diff-diff-noop[main] (0001_test_di) 5.5060 (1.81) 5.5060 (1.81) 5.5060 (1.81) 0.0000 (1.0) 5.5060 (1.81) 0.0000 (1.0) 0;0 0.1816 (0.55) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 3 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.10.2] (0001_test_ex) 1.3670 (1.04) 1.3670 (1.04) 1.3670 (1.04) 0.0000 (1.0) 1.3670 (1.04) 0.0000 (1.0) 0;0 0.7315 (0.96) 1 1
test_exp_show-exp[2.11.0] (0001_test_ex) 1.3194 (1.01) 1.3194 (1.01) 1.3194 (1.01) 0.0000 (1.0) 1.3194 (1.01) 0.0000 (1.0) 0;0 0.7579 (0.99) 1 1
test_exp_show-exp[main] (0001_test_ex) 1.3093 (1.0) 1.3093 (1.0) 1.3093 (1.0) 0.0000 (1.0) 1.3093 (1.0) 0.0000 (1.0) 0;0 0.7637 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_gc-gc': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.10.2] (0001_test_gc) 2.9139 (1.20) 2.9139 (1.20) 2.9139 (1.20) 0.0000 (1.0) 2.9139 (1.20) 0.0000 (1.0) 0;0 0.3432 (0.83) 1 1
test_gc-gc[2.11.0] (0001_test_gc) 2.4329 (1.0) 2.4329 (1.0) 2.4329 (1.0) 0.0000 (1.0) 2.4329 (1.0) 0.0000 (1.0) 0;0 0.4110 (1.0) 1 1
test_gc-gc[2.6.3] (0001_test_gc) 3.4761 (1.43) 3.4761 (1.43) 3.4761 (1.43) 0.0000 (1.0) 3.4761 (1.43) 0.0000 (1.0) 0;0 0.2877 (0.70) 1 1
test_gc-gc[2.9.5] (0001_test_gc) 2.9675 (1.22) 2.9675 (1.22) 2.9675 (1.22) 0.0000 (1.0) 2.9675 (1.22) 0.0000 (1.0) 0;0 0.3370 (0.82) 1 1
test_gc-gc[main] (0001_test_gc) 2.7231 (1.12) 2.7231 (1.12) 2.7231 (1.12) 0.0000 (1.0) 2.7231 (1.12) 0.0000 (1.0) 0;0 0.3672 (0.89) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_get-get': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.10.2] (0001_test_ge) 284.8329 (2.04) 284.8329 (2.04) 284.8329 (2.04) 0.0000 (1.0) 284.8329 (2.04) 0.0000 (1.0) 0;0 0.0035 (0.49) 1 1
test_get-get[2.11.0] (0001_test_ge) 220.5125 (1.58) 220.5125 (1.58) 220.5125 (1.58) 0.0000 (1.0) 220.5125 (1.58) 0.0000 (1.0) 0;0 0.0045 (0.63) 1 1
test_get-get[2.6.3] (0001_test_ge) 161.6835 (1.16) 161.6835 (1.16) 161.6835 (1.16) 0.0000 (1.0) 161.6835 (1.16) 0.0000 (1.0) 0;0 0.0062 (0.86) 1 1
test_get-get[2.9.5] (0001_test_ge) 139.7685 (1.0) 139.7685 (1.0) 139.7685 (1.0) 0.0000 (1.0) 139.7685 (1.0) 0.0000 (1.0) 0;0 0.0072 (1.0) 1 1
test_get-get[main] (0001_test_ge) 222.2095 (1.59) 222.2095 (1.59) 222.2095 (1.59) 0.0000 (1.0) 222.2095 (1.59) 0.0000 (1.0) 0;0 0.0045 (0.63) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_help---help': 5 tests ------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.10.2] (0001_test_he) 140.3767 (1.00) 147.3810 (1.0) 143.1473 (1.00) 1.3119 (1.0) 143.1374 (1.00) 1.5669 (1.08) 30;2 6.9858 (1.00) 100 1
test_help---help[2.11.0] (0001_test_he) 139.8124 (1.0) 159.7748 (1.08) 143.0977 (1.00) 2.6452 (2.02) 142.4836 (1.0) 2.0869 (1.43) 8;5 6.9882 (1.00) 100 1
test_help---help[2.6.3] (0001_test_he) 183.1627 (1.31) 192.8048 (1.31) 186.8648 (1.31) 1.7320 (1.32) 186.6908 (1.31) 2.1311 (1.46) 24;2 5.3515 (0.77) 100 1
test_help---help[2.9.5] (0001_test_he) 141.3955 (1.01) 148.4751 (1.01) 144.7016 (1.01) 1.3415 (1.02) 144.6683 (1.02) 1.4571 (1.0) 31;3 6.9108 (0.99) 100 1
test_help---help[main] (0001_test_he) 140.7264 (1.01) 150.3326 (1.02) 143.0013 (1.0) 1.3575 (1.03) 142.8488 (1.00) 1.7163 (1.18) 24;1 6.9929 (1.0) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_import-import': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.10.2] (0001_test_im) 377.0093 (1.38) 377.0093 (1.38) 377.0093 (1.38) 0.0000 (1.0) 377.0093 (1.38) 0.0000 (1.0) 0;0 0.0027 (0.72) 1 1
test_import-import[2.11.0] (0001_test_im) 346.8436 (1.27) 346.8436 (1.27) 346.8436 (1.27) 0.0000 (1.0) 346.8436 (1.27) 0.0000 (1.0) 0;0 0.0029 (0.79) 1 1
test_import-import[2.6.3] (0001_test_im) 272.3356 (1.0) 272.3356 (1.0) 272.3356 (1.0) 0.0000 (1.0) 272.3356 (1.0) 0.0000 (1.0) 0;0 0.0037 (1.0) 1 1
test_import-import[2.9.5] (0001_test_im) 392.0530 (1.44) 392.0530 (1.44) 392.0530 (1.44) 0.0000 (1.0) 392.0530 (1.44) 0.0000 (1.0) 0;0 0.0026 (0.69) 1 1
test_import-import[main] (0001_test_im) 332.6427 (1.22) 332.6427 (1.22) 332.6427 (1.22) 0.0000 (1.0) 332.6427 (1.22) 0.0000 (1.0) 0;0 0.0030 (0.82) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_init-init': 5 tests -------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.10.2] (0001_test_in) 323.8389 (1.03) 460.4685 (1.20) 351.4863 (1.05) 13.6670 (1.15) 350.5492 (1.05) 11.2382 (1.0) 9;2 2.8451 (0.95) 100 1
test_init-init[2.11.0] (0001_test_in) 332.2648 (1.05) 495.6184 (1.29) 354.7803 (1.06) 17.0578 (1.43) 353.2093 (1.06) 11.4238 (1.02) 7;3 2.8186 (0.94) 100 1
test_init-init[2.6.3] (0001_test_in) 315.7700 (1.0) 401.0912 (1.05) 334.8230 (1.0) 12.4397 (1.04) 333.9432 (1.0) 15.7672 (1.40) 29;1 2.9867 (1.0) 100 1
test_init-init[2.9.5] (0001_test_in) 343.0951 (1.09) 399.3880 (1.04) 361.7528 (1.08) 11.9184 (1.0) 358.2719 (1.07) 15.0017 (1.33) 29;3 2.7643 (0.93) 100 1
test_init-init[main] (0001_test_in) 322.2169 (1.02) 383.2093 (1.0) 354.5880 (1.06) 11.9774 (1.00) 353.2274 (1.06) 15.0292 (1.34) 25;1 2.8202 (0.94) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_list-list': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.10.2] (0001_test_li) 2.7852 (4.23) 2.7852 (4.23) 2.7852 (4.23) 0.0000 (1.0) 2.7852 (4.23) 0.0000 (1.0) 0;0 0.3590 (0.24) 1 1
test_list-list[2.11.0] (0001_test_li) 2.5632 (3.90) 2.5632 (3.90) 2.5632 (3.90) 0.0000 (1.0) 2.5632 (3.90) 0.0000 (1.0) 0;0 0.3901 (0.26) 1 1
test_list-list[2.6.3] (0001_test_li) 0.6577 (1.0) 0.6577 (1.0) 0.6577 (1.0) 0.0000 (1.0) 0.6577 (1.0) 0.0000 (1.0) 0;0 1.5204 (1.0) 1 1
test_list-list[2.9.5] (0001_test_li) 2.8811 (4.38) 2.8811 (4.38) 2.8811 (4.38) 0.0000 (1.0) 2.8811 (4.38) 0.0000 (1.0) 0;0 0.3471 (0.23) 1 1
test_list-list[main] (0001_test_li) 2.4788 (3.77) 2.4788 (3.77) 2.4788 (3.77) 0.0000 (1.0) 2.4788 (3.77) 0.0000 (1.0) 0;0 0.4034 (0.27) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_list-list-dvc-only': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.10.2] (0001_test_li) 2.7265 (4.41) 2.7265 (4.41) 2.7265 (4.41) 0.0000 (1.0) 2.7265 (4.41) 0.0000 (1.0) 0;0 0.3668 (0.23) 1 1
test_list-list-dvc-only[2.11.0] (0001_test_li) 2.6910 (4.36) 2.6910 (4.36) 2.6910 (4.36) 0.0000 (1.0) 2.6910 (4.36) 0.0000 (1.0) 0;0 0.3716 (0.23) 1 1
test_list-list-dvc-only[2.6.3] (0001_test_li) 0.6178 (1.0) 0.6178 (1.0) 0.6178 (1.0) 0.0000 (1.0) 0.6178 (1.0) 0.0000 (1.0) 0;0 1.6188 (1.0) 1 1
test_list-list-dvc-only[2.9.5] (0001_test_li) 2.8485 (4.61) 2.8485 (4.61) 2.8485 (4.61) 0.0000 (1.0) 2.8485 (4.61) 0.0000 (1.0) 0;0 0.3511 (0.22) 1 1
test_list-list-dvc-only[main] (0001_test_li) 2.4984 (4.04) 2.4984 (4.04) 2.4984 (4.04) 0.0000 (1.0) 2.4984 (4.04) 0.0000 (1.0) 0;0 0.4003 (0.25) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------- benchmark 'test_plots_show-plots': 5 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots_show-plots[2.10.2] (0001_test_pl) 762.0324 (1.36) 762.0324 (1.36) 762.0324 (1.36) 0.0000 (1.0) 762.0324 (1.36) 0.0000 (1.0) 0;0 1.3123 (0.74) 1 1
test_plots_show-plots[2.11.0] (0001_test_pl) 923.0646 (1.65) 923.0646 (1.65) 923.0646 (1.65) 0.0000 (1.0) 923.0646 (1.65) 0.0000 (1.0) 0;0 1.0833 (0.61) 1 1
test_plots_show-plots[2.6.3] (0001_test_pl) 560.1587 (1.0) 560.1587 (1.0) 560.1587 (1.0) 0.0000 (1.0) 560.1587 (1.0) 0.0000 (1.0) 0;0 1.7852 (1.0) 1 1
test_plots_show-plots[2.9.5] (0001_test_pl) 1,022.1338 (1.82) 1,022.1338 (1.82) 1,022.1338 (1.82) 0.0000 (1.0) 1,022.1338 (1.82) 0.0000 (1.0) 0;0 0.9783 (0.55) 1 1
test_plots_show-plots[main] (0001_test_pl) 1,429.7562 (2.55) 1,429.7562 (2.55) 1,429.7562 (2.55) 0.0000 (1.0) 1,429.7562 (2.55) 0.0000 (1.0) 0;0 0.6994 (0.39) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_pull-pull': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.10.2] (0001_test_pu) 357.2911 (1.31) 357.2911 (1.31) 357.2911 (1.31) 0.0000 (1.0) 357.2911 (1.31) 0.0000 (1.0) 0;0 0.0028 (0.77) 1 1
test_pull-pull[2.11.0] (0001_test_pu) 354.0280 (1.30) 354.0280 (1.30) 354.0280 (1.30) 0.0000 (1.0) 354.0280 (1.30) 0.0000 (1.0) 0;0 0.0028 (0.77) 1 1
test_pull-pull[2.6.3] (0001_test_pu) 273.3775 (1.0) 273.3775 (1.0) 273.3775 (1.0) 0.0000 (1.0) 273.3775 (1.0) 0.0000 (1.0) 0;0 0.0037 (1.0) 1 1
test_pull-pull[2.9.5] (0001_test_pu) 392.5643 (1.44) 392.5643 (1.44) 392.5643 (1.44) 0.0000 (1.0) 392.5643 (1.44) 0.0000 (1.0) 0;0 0.0025 (0.70) 1 1
test_pull-pull[main] (0001_test_pu) 351.0877 (1.28) 351.0877 (1.28) 351.0877 (1.28) 0.0000 (1.0) 351.0877 (1.28) 0.0000 (1.0) 0;0 0.0028 (0.78) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_push-push': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.10.2] (0001_test_pu) 162.6468 (1.23) 162.6468 (1.23) 162.6468 (1.23) 0.0000 (1.0) 162.6468 (1.23) 0.0000 (1.0) 0;0 0.0061 (0.81) 1 1
test_push-push[2.11.0] (0001_test_pu) 163.0847 (1.23) 163.0847 (1.23) 163.0847 (1.23) 0.0000 (1.0) 163.0847 (1.23) 0.0000 (1.0) 0;0 0.0061 (0.81) 1 1
test_push-push[2.6.3] (0001_test_pu) 132.2008 (1.0) 132.2008 (1.0) 132.2008 (1.0) 0.0000 (1.0) 132.2008 (1.0) 0.0000 (1.0) 0;0 0.0076 (1.0) 1 1
test_push-push[2.9.5] (0001_test_pu) 183.4071 (1.39) 183.4071 (1.39) 183.4071 (1.39) 0.0000 (1.0) 183.4071 (1.39) 0.0000 (1.0) 0;0 0.0055 (0.72) 1 1
test_push-push[main] (0001_test_pu) 160.4629 (1.21) 160.4629 (1.21) 160.4629 (1.21) 0.0000 (1.0) 160.4629 (1.21) 0.0000 (1.0) 0;0 0.0062 (0.82) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_sharing-add': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.10.2] (0001_test_sh) 255.4200 (1.13) 255.4200 (1.13) 255.4200 (1.13) 0.0000 (1.0) 255.4200 (1.13) 0.0000 (1.0) 0;0 0.0039 (0.88) 1 1
test_sharing-add[2.11.0] (0001_test_sh) 252.1283 (1.12) 252.1283 (1.12) 252.1283 (1.12) 0.0000 (1.0) 252.1283 (1.12) 0.0000 (1.0) 0;0 0.0040 (0.89) 1 1
test_sharing-add[2.6.3] (0001_test_sh) 285.5190 (1.27) 285.5190 (1.27) 285.5190 (1.27) 0.0000 (1.0) 285.5190 (1.27) 0.0000 (1.0) 0;0 0.0035 (0.79) 1 1
test_sharing-add[2.9.5] (0001_test_sh) 308.6205 (1.37) 308.6205 (1.37) 308.6205 (1.37) 0.0000 (1.0) 308.6205 (1.37) 0.0000 (1.0) 0;0 0.0032 (0.73) 1 1
test_sharing-add[main] (0001_test_sh) 225.3548 (1.0) 225.3548 (1.0) 225.3548 (1.0) 0.0000 (1.0) 225.3548 (1.0) 0.0000 (1.0) 0;0 0.0044 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.10.2] (0001_test_sh) 125.9841 (6.46) 125.9841 (6.46) 125.9841 (6.46) 0.0000 (1.0) 125.9841 (6.46) 0.0000 (1.0) 0;0 0.0079 (0.15) 1 1
test_sharing-add-noop[2.11.0] (0001_test_sh) 124.1985 (6.37) 124.1985 (6.37) 124.1985 (6.37) 0.0000 (1.0) 124.1985 (6.37) 0.0000 (1.0) 0;0 0.0081 (0.16) 1 1
test_sharing-add-noop[2.6.3] (0001_test_sh) 19.4914 (1.0) 19.4914 (1.0) 19.4914 (1.0) 0.0000 (1.0) 19.4914 (1.0) 0.0000 (1.0) 0;0 0.0513 (1.0) 1 1
test_sharing-add-noop[2.9.5] (0001_test_sh) 139.8307 (7.17) 139.8307 (7.17) 139.8307 (7.17) 0.0000 (1.0) 139.8307 (7.17) 0.0000 (1.0) 0;0 0.0072 (0.14) 1 1
test_sharing-add-noop[main] (0001_test_sh) 136.2068 (6.99) 136.2068 (6.99) 136.2068 (6.99) 0.0000 (1.0) 136.2068 (6.99) 0.0000 (1.0) 0;0 0.0073 (0.14) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.10.2] (0001_test_sh) 5.8115 (1.93) 5.8115 (1.93) 5.8115 (1.93) 0.0000 (1.0) 5.8115 (1.93) 0.0000 (1.0) 0;0 0.1721 (0.52) 1 1
test_sharing-checkout-noop[2.11.0] (0001_test_sh) 5.8801 (1.96) 5.8801 (1.96) 5.8801 (1.96) 0.0000 (1.0) 5.8801 (1.96) 0.0000 (1.0) 0;0 0.1701 (0.51) 1 1
test_sharing-checkout-noop[2.6.3] (0001_test_sh) 3.0050 (1.0) 3.0050 (1.0) 3.0050 (1.0) 0.0000 (1.0) 3.0050 (1.0) 0.0000 (1.0) 0;0 0.3328 (1.0) 1 1
test_sharing-checkout-noop[2.9.5] (0001_test_sh) 5.9422 (1.98) 5.9422 (1.98) 5.9422 (1.98) 0.0000 (1.0) 5.9422 (1.98) 0.0000 (1.0) 0;0 0.1683 (0.51) 1 1
test_sharing-checkout-noop[main] (0001_test_sh) 8.3765 (2.79) 8.3765 (2.79) 8.3765 (2.79) 0.0000 (1.0) 8.3765 (2.79) 0.0000 (1.0) 0;0 0.1194 (0.36) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-pull': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.10.2] (0001_test_sh) 291.4106 (1.26) 291.4106 (1.26) 291.4106 (1.26) 0.0000 (1.0) 291.4106 (1.26) 0.0000 (1.0) 0;0 0.0034 (0.79) 1 1
test_sharing-pull[2.11.0] (0001_test_sh) 284.6384 (1.23) 284.6384 (1.23) 284.6384 (1.23) 0.0000 (1.0) 284.6384 (1.23) 0.0000 (1.0) 0;0 0.0035 (0.81) 1 1
test_sharing-pull[2.6.3] (0001_test_sh) 230.4933 (1.0) 230.4933 (1.0) 230.4933 (1.0) 0.0000 (1.0) 230.4933 (1.0) 0.0000 (1.0) 0;0 0.0043 (1.0) 1 1
test_sharing-pull[2.9.5] (0001_test_sh) 331.5350 (1.44) 331.5350 (1.44) 331.5350 (1.44) 0.0000 (1.0) 331.5350 (1.44) 0.0000 (1.0) 0;0 0.0030 (0.70) 1 1
test_sharing-pull[main] (0001_test_sh) 295.8505 (1.28) 295.8505 (1.28) 295.8505 (1.28) 0.0000 (1.0) 295.8505 (1.28) 0.0000 (1.0) 0;0 0.0034 (0.78) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.10.2] (0001_test_sh) 6.7843 (1.75) 6.7843 (1.75) 6.7843 (1.75) 0.0000 (1.0) 6.7843 (1.75) 0.0000 (1.0) 0;0 0.1474 (0.57) 1 1
test_sharing-pull-noop[2.11.0] (0001_test_sh) 6.7645 (1.74) 6.7645 (1.74) 6.7645 (1.74) 0.0000 (1.0) 6.7645 (1.74) 0.0000 (1.0) 0;0 0.1478 (0.57) 1 1
test_sharing-pull-noop[2.6.3] (0001_test_sh) 3.8773 (1.0) 3.8773 (1.0) 3.8773 (1.0) 0.0000 (1.0) 3.8773 (1.0) 0.0000 (1.0) 0;0 0.2579 (1.0) 1 1
test_sharing-pull-noop[2.9.5] (0001_test_sh) 6.6993 (1.73) 6.6993 (1.73) 6.6993 (1.73) 0.0000 (1.0) 6.6993 (1.73) 0.0000 (1.0) 0;0 0.1493 (0.58) 1 1
test_sharing-pull-noop[main] (0001_test_sh) 9.4170 (2.43) 9.4170 (2.43) 9.4170 (2.43) 0.0000 (1.0) 9.4170 (2.43) 0.0000 (1.0) 0;0 0.1062 (0.41) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 5 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.10.2] (0001_test_sh) 159.3521 (1.24) 159.3521 (1.24) 159.3521 (1.24) 0.0000 (1.0) 159.3521 (1.24) 0.0000 (1.0) 0;0 0.0063 (0.81) 1 1
test_sharing-push[2.11.0] (0001_test_sh) 159.4276 (1.24) 159.4276 (1.24) 159.4276 (1.24) 0.0000 (1.0) 159.4276 (1.24) 0.0000 (1.0) 0;0 0.0063 (0.81) 1 1
test_sharing-push[2.6.3] (0001_test_sh) 128.4674 (1.0) 128.4674 (1.0) 128.4674 (1.0) 0.0000 (1.0) 128.4674 (1.0) 0.0000 (1.0) 0;0 0.0078 (1.0) 1 1
test_sharing-push[2.9.5] (0001_test_sh) 180.0783 (1.40) 180.0783 (1.40) 180.0783 (1.40) 0.0000 (1.0) 180.0783 (1.40) 0.0000 (1.0) 0;0 0.0056 (0.71) 1 1
test_sharing-push[main] (0001_test_sh) 158.0506 (1.23) 158.0506 (1.23) 158.0506 (1.23) 0.0000 (1.0) 158.0506 (1.23) 0.0000 (1.0) 0;0 0.0063 (0.81) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 5 tests --------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.10.2] (0001_test_sh) 2.9992 (1.0) 2.9992 (1.0) 2.9992 (1.0) 0.0000 (1.0) 2.9992 (1.0) 0.0000 (1.0) 0;0 0.3334 (1.0) 1 1
test_sharing-push-noop[2.11.0] (0001_test_sh) 3.1077 (1.04) 3.1077 (1.04) 3.1077 (1.04) 0.0000 (1.0) 3.1077 (1.04) 0.0000 (1.0) 0;0 0.3218 (0.97) 1 1
test_sharing-push-noop[2.6.3] (0001_test_sh) 3.5841 (1.20) 3.5841 (1.20) 3.5841 (1.20) 0.0000 (1.0) 3.5841 (1.20) 0.0000 (1.0) 0;0 0.2790 (0.84) 1 1
test_sharing-push-noop[2.9.5] (0001_test_sh) 3.0420 (1.01) 3.0420 (1.01) 3.0420 (1.01) 0.0000 (1.0) 3.0420 (1.01) 0.0000 (1.0) 0;0 0.3287 (0.99) 1 1
test_sharing-push-noop[main] (0001_test_sh) 3.1228 (1.04) 3.1228 (1.04) 3.1228 (1.04) 0.0000 (1.0) 3.1228 (1.04) 0.0000 (1.0) 0;0 0.3202 (0.96) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_status-status': 5 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.10.2] (0001_test_st) 76.5909 (2.03) 76.5909 (2.03) 76.5909 (2.03) 0.0000 (1.0) 76.5909 (2.03) 0.0000 (1.0) 0;0 0.0131 (0.49) 1 1
test_status-status[2.11.0] (0001_test_st) 69.7862 (1.85) 69.7862 (1.85) 69.7862 (1.85) 0.0000 (1.0) 69.7862 (1.85) 0.0000 (1.0) 0;0 0.0143 (0.54) 1 1
test_status-status[2.6.3] (0001_test_st) 126.1651 (3.34) 126.1651 (3.34) 126.1651 (3.34) 0.0000 (1.0) 126.1651 (3.34) 0.0000 (1.0) 0;0 0.0079 (0.30) 1 1
test_status-status[2.9.5] (0001_test_st) 119.2081 (3.16) 119.2081 (3.16) 119.2081 (3.16) 0.0000 (1.0) 119.2081 (3.16) 0.0000 (1.0) 0;0 0.0084 (0.32) 1 1
test_status-status[main] (0001_test_st) 37.7262 (1.0) 37.7262 (1.0) 37.7262 (1.0) 0.0000 (1.0) 37.7262 (1.0) 0.0000 (1.0) 0;0 0.0265 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_status-status-changed': 5 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.10.2] (0001_test_st) 52.7455 (6.18) 52.7455 (6.18) 52.7455 (6.18) 0.0000 (1.0) 52.7455 (6.18) 0.0000 (1.0) 0;0 0.0190 (0.16) 1 1
test_status-status-changed[2.11.0] (0001_test_st) 51.0950 (5.98) 51.0950 (5.98) 51.0950 (5.98) 0.0000 (1.0) 51.0950 (5.98) 0.0000 (1.0) 0;0 0.0196 (0.17) 1 1
test_status-status-changed[2.6.3] (0001_test_st) 99.8877 (11.70) 99.8877 (11.70) 99.8877 (11.70) 0.0000 (1.0) 99.8877 (11.70) 0.0000 (1.0) 0;0 0.0100 (0.09) 1 1
test_status-status-changed[2.9.5] (0001_test_st) 97.2277 (11.39) 97.2277 (11.39) 97.2277 (11.39) 0.0000 (1.0) 97.2277 (11.39) 0.0000 (1.0) 0;0 0.0103 (0.09) 1 1
test_status-status-changed[main] (0001_test_st) 8.5376 (1.0) 8.5376 (1.0) 8.5376 (1.0) 0.0000 (1.0) 8.5376 (1.0) 0.0000 (1.0) 0;0 0.1171 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 5 tests ---------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.10.2] (0001_test_st) 52.2289 (5.88) 52.2289 (5.88) 52.2289 (5.88) 0.0000 (1.0) 52.2289 (5.88) 0.0000 (1.0) 0;0 0.0191 (0.17) 1 1
test_status-status-changed-noop[2.11.0] (0001_test_st) 13.3469 (1.50) 13.3469 (1.50) 13.3469 (1.50) 0.0000 (1.0) 13.3469 (1.50) 0.0000 (1.0) 0;0 0.0749 (0.67) 1 1
test_status-status-changed-noop[2.6.3] (0001_test_st) 99.9927 (11.25) 99.9927 (11.25) 99.9927 (11.25) 0.0000 (1.0) 99.9927 (11.25) 0.0000 (1.0) 0;0 0.0100 (0.09) 1 1
test_status-status-changed-noop[2.9.5] (0001_test_st) 96.3908 (10.85) 96.3908 (10.85) 96.3908 (10.85) 0.0000 (1.0) 96.3908 (10.85) 0.0000 (1.0) 0;0 0.0104 (0.09) 1 1
test_status-status-changed-noop[main] (0001_test_st) 8.8844 (1.0) 8.8844 (1.0) 8.8844 (1.0) 0.0000 (1.0) 8.8844 (1.0) 0.0000 (1.0) 0;0 0.1126 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_status-status-noop': 5 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.10.2] (0001_test_st) 3.1134 (1.02) 3.1134 (1.02) 3.1134 (1.02) 0.0000 (1.0) 3.1134 (1.02) 0.0000 (1.0) 0;0 0.3212 (0.98) 1 1
test_status-status-noop[2.11.0] (0001_test_st) 3.1249 (1.03) 3.1249 (1.03) 3.1249 (1.03) 0.0000 (1.0) 3.1249 (1.03) 0.0000 (1.0) 0;0 0.3200 (0.97) 1 1
test_status-status-noop[2.6.3] (0001_test_st) 4.5037 (1.48) 4.5037 (1.48) 4.5037 (1.48) 0.0000 (1.0) 4.5037 (1.48) 0.0000 (1.0) 0;0 0.2220 (0.67) 1 1
test_status-status-noop[2.9.5] (0001_test_st) 3.0378 (1.0) 3.0378 (1.0) 3.0378 (1.0) 0.0000 (1.0) 3.0378 (1.0) 0.0000 (1.0) 0;0 0.3292 (1.0) 1 1
test_status-status-noop[main] (0001_test_st) 8.5286 (2.81) 8.5286 (2.81) 8.5286 (2.81) 0.0000 (1.0) 8.5286 (2.81) 0.0000 (1.0) 0;0 0.1173 (0.36) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```