search

iterative / dvc-bench

Benchmarks for DVC
http://bench.dvc.org/
Apache License 2.0
20 stars 10 forks source link

improve naming #339

Closed dtrifiro closed 2 years ago

dtrifiro commented 2 years ago

reverts to --group-by name when comparing

Addendum to #337, fixes issues reported in https://github.com/iterative/dvc-bench/pull/337#issuecomment-1085956117

github-actions[bot] commented 2 years ago 


--------------------------------------------------------------------------------- benchmark 'test_add-add': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add (test_add/0001_main) 89.6000 (1.10) 89.6000 (1.10) 89.6000 (1.10) 0.0000 (1.0) 89.6000 (1.10) 0.0000 (1.0) 0;0 0.0112 (0.91) 1 1
test_add-add (test_add/0002_2.10.0) 89.7910 (1.10) 89.7910 (1.10) 89.7910 (1.10) 0.0000 (1.0) 89.7910 (1.10) 0.0000 (1.0) 0;0 0.0111 (0.91) 1 1
test_add-add (test_add/0003_2.9.5) 89.1678 (1.10) 89.1678 (1.10) 89.1678 (1.10) 0.0000 (1.0) 89.1678 (1.10) 0.0000 (1.0) 0;0 0.0112 (0.91) 1 1
test_add-add (test_add/0004_2.8.3) 91.6625 (1.13) 91.6625 (1.13) 91.6625 (1.13) 0.0000 (1.0) 91.6625 (1.13) 0.0000 (1.0) 0;0 0.0109 (0.89) 1 1
test_add-add (test_add/0005_2.7.3) 83.2681 (1.02) 83.2681 (1.02) 83.2681 (1.02) 0.0000 (1.0) 83.2681 (1.02) 0.0000 (1.0) 0;0 0.0120 (0.98) 1 1
test_add-add (test_add/0006_2.6.3) 81.3592 (1.0) 81.3592 (1.0) 81.3592 (1.0) 0.0000 (1.0) 81.3592 (1.0) 0.0000 (1.0) 0;0 0.0123 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_checkout-checkout': 6 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout (test_checkout/0001_main) 47.7540 (1.51) 47.7540 (1.51) 47.7540 (1.51) 0.0000 (1.0) 47.7540 (1.51) 0.0000 (1.0) 0;0 0.0209 (0.66) 1 1
test_checkout-checkout (test_checkout/0002_2.10.0) 49.3737 (1.56) 49.3737 (1.56) 49.3737 (1.56) 0.0000 (1.0) 49.3737 (1.56) 0.0000 (1.0) 0;0 0.0203 (0.64) 1 1
test_checkout-checkout (test_checkout/0003_2.9.5) 47.8219 (1.51) 47.8219 (1.51) 47.8219 (1.51) 0.0000 (1.0) 47.8219 (1.51) 0.0000 (1.0) 0;0 0.0209 (0.66) 1 1
test_checkout-checkout (test_checkout/0004_2.8.3) 42.1355 (1.33) 42.1355 (1.33) 42.1355 (1.33) 0.0000 (1.0) 42.1355 (1.33) 0.0000 (1.0) 0;0 0.0237 (0.75) 1 1
test_checkout-checkout (test_checkout/0005_2.7.3) 32.2778 (1.02) 32.2778 (1.02) 32.2778 (1.02) 0.0000 (1.0) 32.2778 (1.02) 0.0000 (1.0) 0;0 0.0310 (0.98) 1 1
test_checkout-checkout (test_checkout/0006_2.6.3) 31.6901 (1.0) 31.6901 (1.0) 31.6901 (1.0) 0.0000 (1.0) 31.6901 (1.0) 0.0000 (1.0) 0;0 0.0316 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_checkout-checkout-noop': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-noop (test_checkout/0001_main) 2.6988 (1.91) 2.6988 (1.91) 2.6988 (1.91) 0.0000 (1.0) 2.6988 (1.91) 0.0000 (1.0) 0;0 0.3705 (0.52) 1 1
test_checkout-checkout-noop (test_checkout/0002_2.10.0) 2.5399 (1.80) 2.5399 (1.80) 2.5399 (1.80) 0.0000 (1.0) 2.5399 (1.80) 0.0000 (1.0) 0;0 0.3937 (0.56) 1 1
test_checkout-checkout-noop (test_checkout/0003_2.9.5) 2.5983 (1.84) 2.5983 (1.84) 2.5983 (1.84) 0.0000 (1.0) 2.5983 (1.84) 0.0000 (1.0) 0;0 0.3849 (0.54) 1 1
test_checkout-checkout-noop (test_checkout/0004_2.8.3) 3.0132 (2.13) 3.0132 (2.13) 3.0132 (2.13) 0.0000 (1.0) 3.0132 (2.13) 0.0000 (1.0) 0;0 0.3319 (0.47) 1 1
test_checkout-checkout-noop (test_checkout/0005_2.7.3) 2.2074 (1.56) 2.2074 (1.56) 2.2074 (1.56) 0.0000 (1.0) 2.2074 (1.56) 0.0000 (1.0) 0;0 0.4530 (0.64) 1 1
test_checkout-checkout-noop (test_checkout/0006_2.6.3) 1.4117 (1.0) 1.4117 (1.0) 1.4117 (1.0) 0.0000 (1.0) 1.4117 (1.0) 0.0000 (1.0) 0;0 0.7084 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_checkout-checkout-update': 6 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout-checkout-update (test_checkout/0001_main) 17.1107 (1.05) 17.1107 (1.05) 17.1107 (1.05) 0.0000 (1.0) 17.1107 (1.05) 0.0000 (1.0) 0;0 0.0584 (0.96) 1 1
test_checkout-checkout-update (test_checkout/0002_2.10.0) 16.3447 (1.0) 16.3447 (1.0) 16.3447 (1.0) 0.0000 (1.0) 16.3447 (1.0) 0.0000 (1.0) 0;0 0.0612 (1.0) 1 1
test_checkout-checkout-update (test_checkout/0003_2.9.5) 16.6311 (1.02) 16.6311 (1.02) 16.6311 (1.02) 0.0000 (1.0) 16.6311 (1.02) 0.0000 (1.0) 0;0 0.0601 (0.98) 1 1
test_checkout-checkout-update (test_checkout/0004_2.8.3) 18.1789 (1.11) 18.1789 (1.11) 18.1789 (1.11) 0.0000 (1.0) 18.1789 (1.11) 0.0000 (1.0) 0;0 0.0550 (0.90) 1 1
test_checkout-checkout-update (test_checkout/0005_2.7.3) 48.6495 (2.98) 48.6495 (2.98) 48.6495 (2.98) 0.0000 (1.0) 48.6495 (2.98) 0.0000 (1.0) 0;0 0.0206 (0.34) 1 1
test_checkout-checkout-update (test_checkout/0006_2.6.3) 45.8899 (2.81) 45.8899 (2.81) 45.8899 (2.81) 0.0000 (1.0) 45.8899 (2.81) 0.0000 (1.0) 0;0 0.0218 (0.36) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff (test_diff/0001_main) 22.4003 (1.02) 22.4003 (1.02) 22.4003 (1.02) 0.0000 (1.0) 22.4003 (1.02) 0.0000 (1.0) 0;0 0.0446 (0.99) 1 1
test_diff-diff (test_diff/0002_2.10.0) 22.0684 (1.0) 22.0684 (1.0) 22.0684 (1.0) 0.0000 (1.0) 22.0684 (1.0) 0.0000 (1.0) 0;0 0.0453 (1.0) 1 1
test_diff-diff (test_diff/0003_2.9.5) 22.0732 (1.00) 22.0732 (1.00) 22.0732 (1.00) 0.0000 (1.0) 22.0732 (1.00) 0.0000 (1.0) 0;0 0.0453 (1.00) 1 1
test_diff-diff (test_diff/0004_2.8.3) 24.6098 (1.12) 24.6098 (1.12) 24.6098 (1.12) 0.0000 (1.0) 24.6098 (1.12) 0.0000 (1.0) 0;0 0.0406 (0.90) 1 1
test_diff-diff (test_diff/0005_2.7.3) 26.8637 (1.22) 26.8637 (1.22) 26.8637 (1.22) 0.0000 (1.0) 26.8637 (1.22) 0.0000 (1.0) 0;0 0.0372 (0.82) 1 1
test_diff-diff (test_diff/0006_2.6.3) 39.9969 (1.81) 39.9969 (1.81) 39.9969 (1.81) 0.0000 (1.0) 39.9969 (1.81) 0.0000 (1.0) 0;0 0.0250 (0.55) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop (test_diff/0001_main) 1.4873 (1.03) 1.4873 (1.03) 1.4873 (1.03) 0.0000 (1.0) 1.4873 (1.03) 0.0000 (1.0) 0;0 0.6723 (0.97) 1 1
test_diff-diff-noop (test_diff/0002_2.10.0) 1.4788 (1.03) 1.4788 (1.03) 1.4788 (1.03) 0.0000 (1.0) 1.4788 (1.03) 0.0000 (1.0) 0;0 0.6762 (0.97) 1 1
test_diff-diff-noop (test_diff/0003_2.9.5) 1.4414 (1.0) 1.4414 (1.0) 1.4414 (1.0) 0.0000 (1.0) 1.4414 (1.0) 0.0000 (1.0) 0;0 0.6938 (1.0) 1 1
test_diff-diff-noop (test_diff/0004_2.8.3) 3.0265 (2.10) 3.0265 (2.10) 3.0265 (2.10) 0.0000 (1.0) 3.0265 (2.10) 0.0000 (1.0) 0;0 0.3304 (0.48) 1 1
test_diff-diff-noop (test_diff/0005_2.7.3) 4.0631 (2.82) 4.0631 (2.82) 4.0631 (2.82) 0.0000 (1.0) 4.0631 (2.82) 0.0000 (1.0) 0;0 0.2461 (0.35) 1 1
test_diff-diff-noop (test_diff/0006_2.6.3) 18.1654 (12.60) 18.1654 (12.60) 18.1654 (12.60) 0.0000 (1.0) 18.1654 (12.60) 0.0000 (1.0) 0;0 0.0550 (0.08) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------- benchmark 'test_gc-gc': 6 tests -------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc (test_gc/0001_main) 1.8607 (1.20) 1.8607 (1.20) 1.8607 (1.20) 0.0000 (1.0) 1.8607 (1.20) 0.0000 (1.0) 0;0 0.5374 (0.83) 1 1
test_gc-gc (test_gc/0002_2.10.0) 1.5488 (1.0) 1.5488 (1.0) 1.5488 (1.0) 0.0000 (1.0) 1.5488 (1.0) 0.0000 (1.0) 0;0 0.6456 (1.0) 1 1
test_gc-gc (test_gc/0003_2.9.5) 1.6460 (1.06) 1.6460 (1.06) 1.6460 (1.06) 0.0000 (1.0) 1.6460 (1.06) 0.0000 (1.0) 0;0 0.6075 (0.94) 1 1
test_gc-gc (test_gc/0004_2.8.3) 2.0227 (1.31) 2.0227 (1.31) 2.0227 (1.31) 0.0000 (1.0) 2.0227 (1.31) 0.0000 (1.0) 0;0 0.4944 (0.77) 1 1
test_gc-gc (test_gc/0005_2.7.3) 2.2988 (1.48) 2.2988 (1.48) 2.2988 (1.48) 0.0000 (1.0) 2.2988 (1.48) 0.0000 (1.0) 0;0 0.4350 (0.67) 1 1
test_gc-gc (test_gc/0006_2.6.3) 2.6984 (1.74) 2.6984 (1.74) 2.6984 (1.74) 0.0000 (1.0) 2.6984 (1.74) 0.0000 (1.0) 0;0 0.3706 (0.57) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_get-get': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get (test_get/0001_main) 43.7619 (1.06) 43.7619 (1.06) 43.7619 (1.06) 0.0000 (1.0) 43.7619 (1.06) 0.0000 (1.0) 0;0 0.0229 (0.95) 1 1
test_get-get (test_get/0002_2.10.0) 41.3752 (1.0) 41.3752 (1.0) 41.3752 (1.0) 0.0000 (1.0) 41.3752 (1.0) 0.0000 (1.0) 0;0 0.0242 (1.0) 1 1
test_get-get (test_get/0003_2.9.5) 41.6011 (1.01) 41.6011 (1.01) 41.6011 (1.01) 0.0000 (1.0) 41.6011 (1.01) 0.0000 (1.0) 0;0 0.0240 (0.99) 1 1
test_get-get (test_get/0004_2.8.3) 42.7629 (1.03) 42.7629 (1.03) 42.7629 (1.03) 0.0000 (1.0) 42.7629 (1.03) 0.0000 (1.0) 0;0 0.0234 (0.97) 1 1
test_get-get (test_get/0005_2.7.3) 53.4313 (1.29) 53.4313 (1.29) 53.4313 (1.29) 0.0000 (1.0) 53.4313 (1.29) 0.0000 (1.0) 0;0 0.0187 (0.77) 1 1
test_get-get (test_get/0006_2.6.3) 52.4224 (1.27) 52.4224 (1.27) 52.4224 (1.27) 0.0000 (1.0) 52.4224 (1.27) 0.0000 (1.0) 0;0 0.0191 (0.79) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------- benchmark 'test_help---help': 6 tests -------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help (test_help/0001_main) 158.1899 (1.0) 239.6704 (1.23) 170.1376 (1.02) 12.5915 (2.14) 164.8215 (1.0) 12.6823 (2.29) 15;5 5.8776 (0.98) 100 1
test_help---help (test_help/0002_2.10.0) 161.1724 (1.02) 200.6275 (1.03) 169.6396 (1.02) 8.9908 (1.53) 166.3764 (1.01) 5.6590 (1.02) 13;13 5.8949 (0.98) 100 1
test_help---help (test_help/0003_2.9.5) 159.0190 (1.01) 194.3297 (1.0) 166.5774 (1.0) 5.8889 (1.0) 165.2183 (1.00) 5.5308 (1.0) 14;5 6.0032 (1.0) 100 1
test_help---help (test_help/0004_2.8.3) 188.0105 (1.19) 239.4558 (1.23) 201.6273 (1.21) 13.4597 (2.29) 195.0463 (1.18) 17.5948 (3.18) 21;2 4.9596 (0.83) 100 1
test_help---help (test_help/0005_2.7.3) 204.9857 (1.30) 254.7575 (1.31) 218.8012 (1.31) 11.9683 (2.03) 213.2832 (1.29) 16.1840 (2.93) 19;1 4.5704 (0.76) 100 1
test_help---help (test_help/0006_2.6.3) 199.0166 (1.26) 257.1579 (1.32) 215.6094 (1.29) 13.2630 (2.25) 210.6182 (1.28) 15.9313 (2.88) 23;4 4.6380 (0.77) 100 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------- benchmark 'test_import-import': 6 tests -------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import (test_import/0001_main) 156.7059 (1.61) 156.7059 (1.61) 156.7059 (1.61) 0.0000 (1.0) 156.7059 (1.61) 0.0000 (1.0) 0;0 0.0064 (0.62) 1 1
test_import-import (test_import/0002_2.10.0) 152.6597 (1.57) 152.6597 (1.57) 152.6597 (1.57) 0.0000 (1.0) 152.6597 (1.57) 0.0000 (1.0) 0;0 0.0066 (0.64) 1 1
test_import-import (test_import/0003_2.9.5) 153.7752 (1.58) 153.7752 (1.58) 153.7752 (1.58) 0.0000 (1.0) 153.7752 (1.58) 0.0000 (1.0) 0;0 0.0065 (0.63) 1 1
test_import-import (test_import/0004_2.8.3) 119.4815 (1.23) 119.4815 (1.23) 119.4815 (1.23) 0.0000 (1.0) 119.4815 (1.23) 0.0000 (1.0) 0;0 0.0084 (0.81) 1 1
test_import-import (test_import/0005_2.7.3) 97.3410 (1.0) 97.3410 (1.0) 97.3410 (1.0) 0.0000 (1.0) 97.3410 (1.0) 0.0000 (1.0) 0;0 0.0103 (1.0) 1 1
test_import-import (test_import/0006_2.6.3) 102.8186 (1.06) 102.8186 (1.06) 102.8186 (1.06) 0.0000 (1.0) 102.8186 (1.06) 0.0000 (1.0) 0;0 0.0097 (0.95) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_init-init': 6 tests -------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init (test_init/0001_main) 339.6970 (1.05) 411.8236 (1.22) 349.6740 (1.06) 11.3415 (3.79) 347.5349 (1.06) 5.8066 (1.53) 5;5 2.8598 (0.94) 100 1
test_init-init (test_init/0002_2.10.0) 341.4685 (1.06) 358.3082 (1.06) 347.8552 (1.06) 3.7979 (1.27) 346.9254 (1.05) 4.4140 (1.17) 32;5 2.8748 (0.95) 100 1
test_init-init (test_init/0003_2.9.5) 324.6379 (1.01) 347.0944 (1.03) 330.8954 (1.01) 3.8423 (1.29) 330.0045 (1.00) 4.4792 (1.18) 29;4 3.0221 (0.99) 100 1
test_init-init (test_init/0004_2.8.3) 335.0873 (1.04) 586.5212 (1.74) 344.0488 (1.05) 24.8126 (8.30) 341.3228 (1.04) 4.9494 (1.31) 1;3 2.9066 (0.96) 100 1
test_init-init (test_init/0005_2.7.3) 852.3797 (2.65) 893.5793 (2.65) 864.2400 (2.63) 6.0028 (2.01) 863.3019 (2.62) 7.0571 (1.86) 27;2 1.1571 (0.38) 100 1
test_init-init (test_init/0006_2.6.3) 322.0385 (1.0) 336.7103 (1.0) 328.8009 (1.0) 2.9886 (1.0) 328.9195 (1.0) 3.7859 (1.0) 29;3 3.0414 (1.0) 100 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_pull-pull': 6 tests ------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull (test_pull/0001_main) 169.1346 (1.50) 169.1346 (1.50) 169.1346 (1.50) 0.0000 (1.0) 169.1346 (1.50) 0.0000 (1.0) 0;0 0.0059 (0.67) 1 1
test_pull-pull (test_pull/0002_2.10.0) 165.4433 (1.47) 165.4433 (1.47) 165.4433 (1.47) 0.0000 (1.0) 165.4433 (1.47) 0.0000 (1.0) 0;0 0.0060 (0.68) 1 1
test_pull-pull (test_pull/0003_2.9.5) 172.8433 (1.53) 172.8433 (1.53) 172.8433 (1.53) 0.0000 (1.0) 172.8433 (1.53) 0.0000 (1.0) 0;0 0.0058 (0.65) 1 1
test_pull-pull (test_pull/0004_2.8.3) 152.3425 (1.35) 152.3425 (1.35) 152.3425 (1.35) 0.0000 (1.0) 152.3425 (1.35) 0.0000 (1.0) 0;0 0.0066 (0.74) 1 1
test_pull-pull (test_pull/0005_2.7.3) 120.0878 (1.07) 120.0878 (1.07) 120.0878 (1.07) 0.0000 (1.0) 120.0878 (1.07) 0.0000 (1.0) 0;0 0.0083 (0.94) 1 1
test_pull-pull (test_pull/0006_2.6.3) 112.7004 (1.0) 112.7004 (1.0) 112.7004 (1.0) 0.0000 (1.0) 112.7004 (1.0) 0.0000 (1.0) 0;0 0.0089 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------- benchmark 'test_push-push': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push (test_push/0001_main) 54.2960 (1.39) 54.2960 (1.39) 54.2960 (1.39) 0.0000 (1.0) 54.2960 (1.39) 0.0000 (1.0) 0;0 0.0184 (0.72) 1 1
test_push-push (test_push/0002_2.10.0) 54.4840 (1.40) 54.4840 (1.40) 54.4840 (1.40) 0.0000 (1.0) 54.4840 (1.40) 0.0000 (1.0) 0;0 0.0184 (0.72) 1 1
test_push-push (test_push/0003_2.9.5) 54.7075 (1.40) 54.7075 (1.40) 54.7075 (1.40) 0.0000 (1.0) 54.7075 (1.40) 0.0000 (1.0) 0;0 0.0183 (0.71) 1 1
test_push-push (test_push/0004_2.8.3) 49.2712 (1.26) 49.2712 (1.26) 49.2712 (1.26) 0.0000 (1.0) 49.2712 (1.26) 0.0000 (1.0) 0;0 0.0203 (0.79) 1 1
test_push-push (test_push/0005_2.7.3) 39.4441 (1.01) 39.4441 (1.01) 39.4441 (1.01) 0.0000 (1.0) 39.4441 (1.01) 0.0000 (1.0) 0;0 0.0254 (0.99) 1 1
test_push-push (test_push/0006_2.6.3) 38.9855 (1.0) 38.9855 (1.0) 38.9855 (1.0) 0.0000 (1.0) 38.9855 (1.0) 0.0000 (1.0) 0;0 0.0257 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------- benchmark 'test_sharing-add': 6 tests --------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add (test_sharing/0001_main) 116.0482 (1.07) 116.0482 (1.07) 116.0482 (1.07) 0.0000 (1.0) 116.0482 (1.07) 0.0000 (1.0) 0;0 0.0086 (0.93) 1 1
test_sharing-add (test_sharing/0002_2.10.0) 117.7699 (1.09) 117.7699 (1.09) 117.7699 (1.09) 0.0000 (1.0) 117.7699 (1.09) 0.0000 (1.0) 0;0 0.0085 (0.92) 1 1
test_sharing-add (test_sharing/0003_2.9.5) 117.5343 (1.09) 117.5343 (1.09) 117.5343 (1.09) 0.0000 (1.0) 117.5343 (1.09) 0.0000 (1.0) 0;0 0.0085 (0.92) 1 1
test_sharing-add (test_sharing/0004_2.8.3) 115.2627 (1.07) 115.2627 (1.07) 115.2627 (1.07) 0.0000 (1.0) 115.2627 (1.07) 0.0000 (1.0) 0;0 0.0087 (0.94) 1 1
test_sharing-add (test_sharing/0005_2.7.3) 110.2882 (1.02) 110.2882 (1.02) 110.2882 (1.02) 0.0000 (1.0) 110.2882 (1.02) 0.0000 (1.0) 0;0 0.0091 (0.98) 1 1
test_sharing-add (test_sharing/0006_2.6.3) 107.9931 (1.0) 107.9931 (1.0) 107.9931 (1.0) 0.0000 (1.0) 107.9931 (1.0) 0.0000 (1.0) 0;0 0.0093 (1.0) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 6 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop (test_sharing/0001_main) 49.5474 (7.20) 49.5474 (7.20) 49.5474 (7.20) 0.0000 (1.0) 49.5474 (7.20) 0.0000 (1.0) 0;0 0.0202 (0.14) 1 1
test_sharing-add-noop (test_sharing/0002_2.10.0) 49.5178 (7.19) 49.5178 (7.19) 49.5178 (7.19) 0.0000 (1.0) 49.5178 (7.19) 0.0000 (1.0) 0;0 0.0202 (0.14) 1 1
test_sharing-add-noop (test_sharing/0003_2.9.5) 49.9209 (7.25) 49.9209 (7.25) 49.9209 (7.25) 0.0000 (1.0) 49.9209 (7.25) 0.0000 (1.0) 0;0 0.0200 (0.14) 1 1
test_sharing-add-noop (test_sharing/0004_2.8.3) 14.1552 (2.06) 14.1552 (2.06) 14.1552 (2.06) 0.0000 (1.0) 14.1552 (2.06) 0.0000 (1.0) 0;0 0.0706 (0.49) 1 1
test_sharing-add-noop (test_sharing/0005_2.7.3) 7.9300 (1.15) 7.9300 (1.15) 7.9300 (1.15) 0.0000 (1.0) 7.9300 (1.15) 0.0000 (1.0) 0;0 0.1261 (0.87) 1 1
test_sharing-add-noop (test_sharing/0006_2.6.3) 6.8841 (1.0) 6.8841 (1.0) 6.8841 (1.0) 0.0000 (1.0) 6.8841 (1.0) 0.0000 (1.0) 0;0 0.1453 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop (test_sharing/0001_main) 2.3613 (1.91) 2.3613 (1.91) 2.3613 (1.91) 0.0000 (1.0) 2.3613 (1.91) 0.0000 (1.0) 0;0 0.4235 (0.52) 1 1
test_sharing-checkout-noop (test_sharing/0002_2.10.0) 2.3651 (1.92) 2.3651 (1.92) 2.3651 (1.92) 0.0000 (1.0) 2.3651 (1.92) 0.0000 (1.0) 0;0 0.4228 (0.52) 1 1
test_sharing-checkout-noop (test_sharing/0003_2.9.5) 2.3310 (1.89) 2.3310 (1.89) 2.3310 (1.89) 0.0000 (1.0) 2.3310 (1.89) 0.0000 (1.0) 0;0 0.4290 (0.53) 1 1
test_sharing-checkout-noop (test_sharing/0004_2.8.3) 2.7399 (2.22) 2.7399 (2.22) 2.7399 (2.22) 0.0000 (1.0) 2.7399 (2.22) 0.0000 (1.0) 0;0 0.3650 (0.45) 1 1
test_sharing-checkout-noop (test_sharing/0005_2.7.3) 2.0211 (1.64) 2.0211 (1.64) 2.0211 (1.64) 0.0000 (1.0) 2.0211 (1.64) 0.0000 (1.0) 0;0 0.4948 (0.61) 1 1
test_sharing-checkout-noop (test_sharing/0006_2.6.3) 1.2331 (1.0) 1.2331 (1.0) 1.2331 (1.0) 0.0000 (1.0) 1.2331 (1.0) 0.0000 (1.0) 0;0 0.8110 (1.0) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------- benchmark 'test_sharing-pull': 6 tests -------------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull (test_sharing/0001_main) 116.0312 (1.42) 116.0312 (1.42) 116.0312 (1.42) 0.0000 (1.0) 116.0312 (1.42) 0.0000 (1.0) 0;0 0.0086 (0.70) 1 1
test_sharing-pull (test_sharing/0002_2.10.0) 115.6442 (1.42) 115.6442 (1.42) 115.6442 (1.42) 0.0000 (1.0) 115.6442 (1.42) 0.0000 (1.0) 0;0 0.0086 (0.71) 1 1
test_sharing-pull (test_sharing/0003_2.9.5) 117.1849 (1.44) 117.1849 (1.44) 117.1849 (1.44) 0.0000 (1.0) 117.1849 (1.44) 0.0000 (1.0) 0;0 0.0085 (0.70) 1 1
test_sharing-pull (test_sharing/0004_2.8.3) 105.0443 (1.29) 105.0443 (1.29) 105.0443 (1.29) 0.0000 (1.0) 105.0443 (1.29) 0.0000 (1.0) 0;0 0.0095 (0.78) 1 1
test_sharing-pull (test_sharing/0005_2.7.3) 81.7433 (1.00) 81.7433 (1.00) 81.7433 (1.00) 0.0000 (1.0) 81.7433 (1.00) 0.0000 (1.0) 0;0 0.0122 (1.00) 1 1
test_sharing-pull (test_sharing/0006_2.6.3) 81.6111 (1.0) 81.6111 (1.0) 81.6111 (1.0) 0.0000 (1.0) 81.6111 (1.0) 0.0000 (1.0) 0;0 0.0123 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop (test_sharing/0001_main) 3.0491 (1.90) 3.0491 (1.90) 3.0491 (1.90) 0.0000 (1.0) 3.0491 (1.90) 0.0000 (1.0) 0;0 0.3280 (0.53) 1 1
test_sharing-pull-noop (test_sharing/0002_2.10.0) 2.6605 (1.66) 2.6605 (1.66) 2.6605 (1.66) 0.0000 (1.0) 2.6605 (1.66) 0.0000 (1.0) 0;0 0.3759 (0.60) 1 1
test_sharing-pull-noop (test_sharing/0003_2.9.5) 2.5371 (1.58) 2.5371 (1.58) 2.5371 (1.58) 0.0000 (1.0) 2.5371 (1.58) 0.0000 (1.0) 0;0 0.3942 (0.63) 1 1
test_sharing-pull-noop (test_sharing/0004_2.8.3) 3.0980 (1.93) 3.0980 (1.93) 3.0980 (1.93) 0.0000 (1.0) 3.0980 (1.93) 0.0000 (1.0) 0;0 0.3228 (0.52) 1 1
test_sharing-pull-noop (test_sharing/0005_2.7.3) 2.3256 (1.45) 2.3256 (1.45) 2.3256 (1.45) 0.0000 (1.0) 2.3256 (1.45) 0.0000 (1.0) 0;0 0.4300 (0.69) 1 1
test_sharing-pull-noop (test_sharing/0006_2.6.3) 1.6057 (1.0) 1.6057 (1.0) 1.6057 (1.0) 0.0000 (1.0) 1.6057 (1.0) 0.0000 (1.0) 0;0 0.6228 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'test_sharing-push': 6 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push (test_sharing/0001_main) 59.8342 (1.33) 59.8342 (1.33) 59.8342 (1.33) 0.0000 (1.0) 59.8342 (1.33) 0.0000 (1.0) 0;0 0.0167 (0.75) 1 1
test_sharing-push (test_sharing/0002_2.10.0) 62.1509 (1.38) 62.1509 (1.38) 62.1509 (1.38) 0.0000 (1.0) 62.1509 (1.38) 0.0000 (1.0) 0;0 0.0161 (0.72) 1 1
test_sharing-push (test_sharing/0003_2.9.5) 61.5248 (1.37) 61.5248 (1.37) 61.5248 (1.37) 0.0000 (1.0) 61.5248 (1.37) 0.0000 (1.0) 0;0 0.0163 (0.73) 1 1
test_sharing-push (test_sharing/0004_2.8.3) 55.8461 (1.24) 55.8461 (1.24) 55.8461 (1.24) 0.0000 (1.0) 55.8461 (1.24) 0.0000 (1.0) 0;0 0.0179 (0.80) 1 1
test_sharing-push (test_sharing/0005_2.7.3) 44.9322 (1.0) 44.9322 (1.0) 44.9322 (1.0) 0.0000 (1.0) 44.9322 (1.0) 0.0000 (1.0) 0;0 0.0223 (1.0) 1 1
test_sharing-push (test_sharing/0006_2.6.3) 45.0298 (1.00) 45.0298 (1.00) 45.0298 (1.00) 0.0000 (1.0) 45.0298 (1.00) 0.0000 (1.0) 0;0 0.0222 (1.00) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 6 tests ----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop (test_sharing/0001_main) 1.3440 (1.06) 1.3440 (1.06) 1.3440 (1.06) 0.0000 (1.0) 1.3440 (1.06) 0.0000 (1.0) 0;0 0.7440 (0.95) 1 1
test_sharing-push-noop (test_sharing/0002_2.10.0) 1.2714 (1.0) 1.2714 (1.0) 1.2714 (1.0) 0.0000 (1.0) 1.2714 (1.0) 0.0000 (1.0) 0;0 0.7865 (1.0) 1 1
test_sharing-push-noop (test_sharing/0003_2.9.5) 1.3386 (1.05) 1.3386 (1.05) 1.3386 (1.05) 0.0000 (1.0) 1.3386 (1.05) 0.0000 (1.0) 0;0 0.7471 (0.95) 1 1
test_sharing-push-noop (test_sharing/0004_2.8.3) 1.3216 (1.04) 1.3216 (1.04) 1.3216 (1.04) 0.0000 (1.0) 1.3216 (1.04) 0.0000 (1.0) 0;0 0.7566 (0.96) 1 1
test_sharing-push-noop (test_sharing/0005_2.7.3) 2.1931 (1.72) 2.1931 (1.72) 2.1931 (1.72) 0.0000 (1.0) 2.1931 (1.72) 0.0000 (1.0) 0;0 0.4560 (0.58) 1 1
test_sharing-push-noop (test_sharing/0006_2.6.3) 1.3760 (1.08) 1.3760 (1.08) 1.3760 (1.08) 0.0000 (1.0) 1.3760 (1.08) 0.0000 (1.0) 0;0 0.7268 (0.92) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'test_status-status': 6 tests -----------------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status (test_status/0001_main) 44.2520 (1.0) 44.2520 (1.0) 44.2520 (1.0) 0.0000 (1.0) 44.2520 (1.0) 0.0000 (1.0) 0;0 0.0226 (1.0) 1 1
test_status-status (test_status/0002_2.10.0) 47.2328 (1.07) 47.2328 (1.07) 47.2328 (1.07) 0.0000 (1.0) 47.2328 (1.07) 0.0000 (1.0) 0;0 0.0212 (0.94) 1 1
test_status-status (test_status/0003_2.9.5) 44.8344 (1.01) 44.8344 (1.01) 44.8344 (1.01) 0.0000 (1.0) 44.8344 (1.01) 0.0000 (1.0) 0;0 0.0223 (0.99) 1 1
test_status-status (test_status/0004_2.8.3) 49.0417 (1.11) 49.0417 (1.11) 49.0417 (1.11) 0.0000 (1.0) 49.0417 (1.11) 0.0000 (1.0) 0;0 0.0204 (0.90) 1 1
test_status-status (test_status/0005_2.7.3) 48.8634 (1.10) 48.8634 (1.10) 48.8634 (1.10) 0.0000 (1.0) 48.8634 (1.10) 0.0000 (1.0) 0;0 0.0205 (0.91) 1 1
test_status-status (test_status/0006_2.6.3) 46.8951 (1.06) 46.8951 (1.06) 46.8951 (1.06) 0.0000 (1.0) 46.8951 (1.06) 0.0000 (1.0) 0;0 0.0213 (0.94) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
 Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
 OPS: Operations Per Second, computed as 1 / Mean\n```