github-actions[bot]
commented
1 year ago
Mon May 8 06:02:49 UTC 2023
dataset: small
project: example-get-started
------------------------------------------------------------------------ benchmark 'test_plots-plots-diff': 5 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-diff[2.39.0] 1.1098 (1.17) 1.1211 (1.16) 1.1141 (1.16) 0.0043 (1.45) 1.1129 (1.17) 0.0049 (1.28) 1;0 0.8976 (0.86) 5 3
test_plots-plots-diff[2.40.0] 1.1150 (1.18) 1.1223 (1.16) 1.1198 (1.17) 0.0030 (1.0) 1.1210 (1.18) 0.0038 (1.0) 1;0 0.8930 (0.85) 5 3
test_plots-plots-diff[2.41.1] 1.1119 (1.17) 1.1274 (1.17) 1.1192 (1.17) 0.0067 (2.28) 1.1218 (1.18) 0.0112 (2.92) 2;0 0.8935 (0.85) 5 3
test_plots-plots-diff[2.45.0] 1.0130 (1.07) 1.0447 (1.08) 1.0303 (1.08) 0.0149 (5.04) 1.0389 (1.09) 0.0260 (6.79) 1;0 0.9706 (0.93) 5 3
test_plots-plots-diff[main] 0.9489 (1.0) 0.9666 (1.0) 0.9563 (1.0) 0.0084 (2.85) 0.9519 (1.0) 0.0151 (3.95) 1;0 1.0457 (1.0) 5 3
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_plots-plots-diff-json': 5 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-diff-json[2.39.0] 2.0485 (1.63) 2.0987 (1.65) 2.0639 (1.64) 0.0201 (6.60) 2.0576 (1.63) 0.0195 (3.46) 1;0 0.4845 (0.61) 5 3
test_plots-plots-diff-json[2.40.0] 2.0538 (1.64) 2.1087 (1.65) 2.0705 (1.64) 0.0227 (7.45) 2.0639 (1.64) 0.0276 (4.88) 1;0 0.4830 (0.61) 5 3
test_plots-plots-diff-json[2.41.1] 2.0532 (1.64) 2.0599 (1.62) 2.0564 (1.63) 0.0030 (1.0) 2.0555 (1.63) 0.0056 (1.0) 2;0 0.4863 (0.61) 5 3
test_plots-plots-diff-json[2.45.0] 1.8417 (1.47) 1.8548 (1.46) 1.8466 (1.46) 0.0051 (1.67) 1.8463 (1.46) 0.0066 (1.17) 1;0 0.5415 (0.68) 5 3
test_plots-plots-diff-json[main] 1.2544 (1.0) 1.2742 (1.0) 1.2622 (1.0) 0.0075 (2.45) 1.2603 (1.0) 0.0084 (1.49) 2;0 0.7923 (1.0) 5 3
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_plots-plots-show': 5 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-show[2.39.0] 745.9604 (1.05) 752.2003 (1.04) 749.3645 (1.05) 2.3642 (1.18) 749.9804 (1.04) 3.1852 (1.0) 2;0 1.3345 (0.96) 5 3
test_plots-plots-show[2.40.0] 747.0282 (1.05) 758.8131 (1.05) 754.1858 (1.05) 5.2927 (2.64) 756.2233 (1.05) 9.2334 (2.90) 1;0 1.3259 (0.95) 5 3
test_plots-plots-show[2.41.1] 749.5479 (1.05) 753.8572 (1.05) 751.3333 (1.05) 2.0035 (1.0) 750.0910 (1.04) 3.4080 (1.07) 1;0 1.3310 (0.95) 5 3
test_plots-plots-show[2.45.0] 712.4364 (1.0) 720.3907 (1.0) 717.0693 (1.0) 3.1943 (1.59) 718.3482 (1.0) 4.7306 (1.49) 2;0 1.3946 (1.0) 5 3
test_plots-plots-show[main] 770.2437 (1.08) 798.0461 (1.11) 784.8933 (1.09) 9.8918 (4.94) 785.4792 (1.09) 8.7961 (2.76) 2;0 1.2741 (0.91) 5 3
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_plots-plots-show-json': 5 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-show-json[2.39.0] 1.2494 (1.32) 1.2655 (1.32) 1.2558 (1.32) 0.0064 (1.74) 1.2531 (1.31) 0.0089 (2.49) 2;0 0.7963 (0.76) 5 3
test_plots-plots-show-json[2.40.0] 1.2499 (1.32) 1.2604 (1.32) 1.2551 (1.32) 0.0037 (1.02) 1.2547 (1.32) 0.0036 (1.0) 2;0 0.7967 (0.76) 5 3
test_plots-plots-show-json[2.41.1] 1.2458 (1.31) 1.2544 (1.31) 1.2517 (1.31) 0.0037 (1.0) 1.2537 (1.31) 0.0049 (1.39) 1;0 0.7989 (0.76) 5 3
test_plots-plots-show-json[2.45.0] 1.1625 (1.23) 1.1796 (1.23) 1.1723 (1.23) 0.0063 (1.71) 1.1738 (1.23) 0.0065 (1.83) 2;0 0.8530 (0.81) 5 3
test_plots-plots-show-json[main] 0.9476 (1.0) 0.9574 (1.0) 0.9534 (1.0) 0.0038 (1.02) 0.9536 (1.0) 0.0049 (1.37) 2;0 1.0489 (1.0) 5 3
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-changed': 6 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-changed[2.18.1] 1.3144 (1.0) 1.3144 (1.0) 1.3144 (1.0) 0.0000 (1.0) 1.3144 (1.0) 0.0000 (1.0) 0;0 0.7608 (1.0) 1 1
test_data_status_all_flags-data-changed[2.39.0] 1.4469 (1.10) 1.4469 (1.10) 1.4469 (1.10) 0.0000 (1.0) 1.4469 (1.10) 0.0000 (1.0) 0;0 0.6912 (0.91) 1 1
test_data_status_all_flags-data-changed[2.40.0] 1.4852 (1.13) 1.4852 (1.13) 1.4852 (1.13) 0.0000 (1.0) 1.4852 (1.13) 0.0000 (1.0) 0;0 0.6733 (0.89) 1 1
test_data_status_all_flags-data-changed[2.41.1] 1.4343 (1.09) 1.4343 (1.09) 1.4343 (1.09) 0.0000 (1.0) 1.4343 (1.09) 0.0000 (1.0) 0;0 0.6972 (0.92) 1 1
test_data_status_all_flags-data-changed[2.45.0] 1.3955 (1.06) 1.3955 (1.06) 1.3955 (1.06) 0.0000 (1.0) 1.3955 (1.06) 0.0000 (1.0) 0;0 0.7166 (0.94) 1 1
test_data_status_all_flags-data-changed[main] 1.3783 (1.05) 1.3783 (1.05) 1.3783 (1.05) 0.0000 (1.0) 1.3783 (1.05) 0.0000 (1.0) 0;0 0.7255 (0.95) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-changed-noop': 6 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-changed-noop[2.18.1] 1.3987 (1.02) 1.3987 (1.02) 1.3987 (1.02) 0.0000 (1.0) 1.3987 (1.02) 0.0000 (1.0) 0;0 0.7149 (0.98) 1 1
test_data_status_all_flags-data-changed-noop[2.39.0] 1.3777 (1.01) 1.3777 (1.01) 1.3777 (1.01) 0.0000 (1.0) 1.3777 (1.01) 0.0000 (1.0) 0;0 0.7258 (0.99) 1 1
test_data_status_all_flags-data-changed-noop[2.40.0] 1.4134 (1.03) 1.4134 (1.03) 1.4134 (1.03) 0.0000 (1.0) 1.4134 (1.03) 0.0000 (1.0) 0;0 0.7075 (0.97) 1 1
test_data_status_all_flags-data-changed-noop[2.41.1] 1.3706 (1.0) 1.3706 (1.0) 1.3706 (1.0) 0.0000 (1.0) 1.3706 (1.0) 0.0000 (1.0) 0;0 0.7296 (1.0) 1 1
test_data_status_all_flags-data-changed-noop[2.45.0] 1.4171 (1.03) 1.4171 (1.03) 1.4171 (1.03) 0.0000 (1.0) 1.4171 (1.03) 0.0000 (1.0) 0;0 0.7057 (0.97) 1 1
test_data_status_all_flags-data-changed-noop[main] 1.3867 (1.01) 1.3867 (1.01) 1.3867 (1.01) 0.0000 (1.0) 1.3867 (1.01) 0.0000 (1.0) 0;0 0.7211 (0.99) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-new': 6 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-new[2.18.1] 2.3863 (1.07) 2.3863 (1.07) 2.3863 (1.07) 0.0000 (1.0) 2.3863 (1.07) 0.0000 (1.0) 0;0 0.4191 (0.93) 1 1
test_data_status_all_flags-data-new[2.39.0] 2.4440 (1.10) 2.4440 (1.10) 2.4440 (1.10) 0.0000 (1.0) 2.4440 (1.10) 0.0000 (1.0) 0;0 0.4092 (0.91) 1 1
test_data_status_all_flags-data-new[2.40.0] 2.3410 (1.05) 2.3410 (1.05) 2.3410 (1.05) 0.0000 (1.0) 2.3410 (1.05) 0.0000 (1.0) 0;0 0.4272 (0.95) 1 1
test_data_status_all_flags-data-new[2.41.1] 2.3547 (1.06) 2.3547 (1.06) 2.3547 (1.06) 0.0000 (1.0) 2.3547 (1.06) 0.0000 (1.0) 0;0 0.4247 (0.95) 1 1
test_data_status_all_flags-data-new[2.45.0] 2.2292 (1.0) 2.2292 (1.0) 2.2292 (1.0) 0.0000 (1.0) 2.2292 (1.0) 0.0000 (1.0) 0;0 0.4486 (1.0) 1 1
test_data_status_all_flags-data-new[main] 2.3267 (1.04) 2.3267 (1.04) 2.3267 (1.04) 0.0000 (1.0) 2.3267 (1.04) 0.0000 (1.0) 0;0 0.4298 (0.96) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-noop': 6 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-noop[2.18.1] 1.3226 (1.0) 1.3226 (1.0) 1.3226 (1.0) 0.0000 (1.0) 1.3226 (1.0) 0.0000 (1.0) 0;0 0.7561 (1.0) 1 1
test_data_status_all_flags-data-noop[2.39.0] 1.3370 (1.01) 1.3370 (1.01) 1.3370 (1.01) 0.0000 (1.0) 1.3370 (1.01) 0.0000 (1.0) 0;0 0.7479 (0.99) 1 1
test_data_status_all_flags-data-noop[2.40.0] 1.3786 (1.04) 1.3786 (1.04) 1.3786 (1.04) 0.0000 (1.0) 1.3786 (1.04) 0.0000 (1.0) 0;0 0.7254 (0.96) 1 1
test_data_status_all_flags-data-noop[2.41.1] 1.4151 (1.07) 1.4151 (1.07) 1.4151 (1.07) 0.0000 (1.0) 1.4151 (1.07) 0.0000 (1.0) 0;0 0.7067 (0.93) 1 1
test_data_status_all_flags-data-noop[2.45.0] 1.3771 (1.04) 1.3771 (1.04) 1.3771 (1.04) 0.0000 (1.0) 1.3771 (1.04) 0.0000 (1.0) 0;0 0.7262 (0.96) 1 1
test_data_status_all_flags-data-noop[main] 1.3483 (1.02) 1.3483 (1.02) 1.3483 (1.02) 0.0000 (1.0) 1.3483 (1.02) 0.0000 (1.0) 0;0 0.7417 (0.98) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_sharing-add': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.11.0] 6.8774 (1.76) 6.8774 (1.76) 6.8774 (1.76) 0.0000 (1.0) 6.8774 (1.76) 0.0000 (1.0) 0;0 0.1454 (0.57) 1 1
test_sharing-add[2.18.1] 6.1779 (1.58) 6.1779 (1.58) 6.1779 (1.58) 0.0000 (1.0) 6.1779 (1.58) 0.0000 (1.0) 0;0 0.1619 (0.63) 1 1
test_sharing-add[2.39.0] 5.9500 (1.53) 5.9500 (1.53) 5.9500 (1.53) 0.0000 (1.0) 5.9500 (1.53) 0.0000 (1.0) 0;0 0.1681 (0.66) 1 1
test_sharing-add[2.40.0] 6.0791 (1.56) 6.0791 (1.56) 6.0791 (1.56) 0.0000 (1.0) 6.0791 (1.56) 0.0000 (1.0) 0;0 0.1645 (0.64) 1 1
test_sharing-add[2.41.1] 5.9243 (1.52) 5.9243 (1.52) 5.9243 (1.52) 0.0000 (1.0) 5.9243 (1.52) 0.0000 (1.0) 0;0 0.1688 (0.66) 1 1
test_sharing-add[2.45.0] 4.1075 (1.05) 4.1075 (1.05) 4.1075 (1.05) 0.0000 (1.0) 4.1075 (1.05) 0.0000 (1.0) 0;0 0.2435 (0.95) 1 1
test_sharing-add[main] 3.8983 (1.0) 3.8983 (1.0) 3.8983 (1.0) 0.0000 (1.0) 3.8983 (1.0) 0.0000 (1.0) 0;0 0.2565 (1.0) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.11.0] 3.4111 (1.40) 3.4111 (1.40) 3.4111 (1.40) 0.0000 (1.0) 3.4111 (1.40) 0.0000 (1.0) 0;0 0.2932 (0.71) 1 1
test_sharing-add-noop[2.18.1] 3.6218 (1.49) 3.6218 (1.49) 3.6218 (1.49) 0.0000 (1.0) 3.6218 (1.49) 0.0000 (1.0) 0;0 0.2761 (0.67) 1 1
test_sharing-add-noop[2.39.0] 2.4334 (1.0) 2.4334 (1.0) 2.4334 (1.0) 0.0000 (1.0) 2.4334 (1.0) 0.0000 (1.0) 0;0 0.4109 (1.0) 1 1
test_sharing-add-noop[2.40.0] 2.4414 (1.00) 2.4414 (1.00) 2.4414 (1.00) 0.0000 (1.0) 2.4414 (1.00) 0.0000 (1.0) 0;0 0.4096 (1.00) 1 1
test_sharing-add-noop[2.41.1] 2.4367 (1.00) 2.4367 (1.00) 2.4367 (1.00) 0.0000 (1.0) 2.4367 (1.00) 0.0000 (1.0) 0;0 0.4104 (1.00) 1 1
test_sharing-add-noop[2.45.0] 2.5025 (1.03) 2.5025 (1.03) 2.5025 (1.03) 0.0000 (1.0) 2.5025 (1.03) 0.0000 (1.0) 0;0 0.3996 (0.97) 1 1
test_sharing-add-noop[main] 2.5483 (1.05) 2.5483 (1.05) 2.5483 (1.05) 0.0000 (1.0) 2.5483 (1.05) 0.0000 (1.0) 0;0 0.3924 (0.95) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.11.0] 479.4488 (1.0) 479.4488 (1.0) 479.4488 (1.0) 0.0000 (1.0) 479.4488 (1.0) 0.0000 (1.0) 0;0 2.0857 (1.0) 1 1
test_sharing-checkout-noop[2.18.1] 546.7946 (1.14) 546.7946 (1.14) 546.7946 (1.14) 0.0000 (1.0) 546.7946 (1.14) 0.0000 (1.0) 0;0 1.8288 (0.88) 1 1
test_sharing-checkout-noop[2.39.0] 576.2208 (1.20) 576.2208 (1.20) 576.2208 (1.20) 0.0000 (1.0) 576.2208 (1.20) 0.0000 (1.0) 0;0 1.7354 (0.83) 1 1
test_sharing-checkout-noop[2.40.0] 572.9504 (1.20) 572.9504 (1.20) 572.9504 (1.20) 0.0000 (1.0) 572.9504 (1.20) 0.0000 (1.0) 0;0 1.7454 (0.84) 1 1
test_sharing-checkout-noop[2.41.1] 568.2655 (1.19) 568.2655 (1.19) 568.2655 (1.19) 0.0000 (1.0) 568.2655 (1.19) 0.0000 (1.0) 0;0 1.7597 (0.84) 1 1
test_sharing-checkout-noop[2.45.0] 592.9626 (1.24) 592.9626 (1.24) 592.9626 (1.24) 0.0000 (1.0) 592.9626 (1.24) 0.0000 (1.0) 0;0 1.6864 (0.81) 1 1
test_sharing-checkout-noop[main] 664.1057 (1.39) 664.1057 (1.39) 664.1057 (1.39) 0.0000 (1.0) 664.1057 (1.39) 0.0000 (1.0) 0;0 1.5058 (0.72) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_sharing-pull': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull[2.11.0] 7.0712 (1.05) 7.0712 (1.05) 7.0712 (1.05) 0.0000 (1.0) 7.0712 (1.05) 0.0000 (1.0) 0;0 0.1414 (0.95) 1 1
test_sharing-pull[2.18.1] 7.0887 (1.05) 7.0887 (1.05) 7.0887 (1.05) 0.0000 (1.0) 7.0887 (1.05) 0.0000 (1.0) 0;0 0.1411 (0.95) 1 1
test_sharing-pull[2.39.0] 8.4398 (1.25) 8.4398 (1.25) 8.4398 (1.25) 0.0000 (1.0) 8.4398 (1.25) 0.0000 (1.0) 0;0 0.1185 (0.80) 1 1
test_sharing-pull[2.40.0] 8.4828 (1.26) 8.4828 (1.26) 8.4828 (1.26) 0.0000 (1.0) 8.4828 (1.26) 0.0000 (1.0) 0;0 0.1179 (0.79) 1 1
test_sharing-pull[2.41.1] 8.4255 (1.25) 8.4255 (1.25) 8.4255 (1.25) 0.0000 (1.0) 8.4255 (1.25) 0.0000 (1.0) 0;0 0.1187 (0.80) 1 1
test_sharing-pull[2.45.0] 6.7378 (1.0) 6.7378 (1.0) 6.7378 (1.0) 0.0000 (1.0) 6.7378 (1.0) 0.0000 (1.0) 0;0 0.1484 (1.0) 1 1
test_sharing-pull[main] 6.7560 (1.00) 6.7560 (1.00) 6.7560 (1.00) 0.0000 (1.0) 6.7560 (1.00) 0.0000 (1.0) 0;0 0.1480 (1.00) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_sharing-pull-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-pull-noop[2.11.0] 502.2896 (1.0) 502.2896 (1.0) 502.2896 (1.0) 0.0000 (1.0) 502.2896 (1.0) 0.0000 (1.0) 0;0 1.9909 (1.0) 1 1
test_sharing-pull-noop[2.18.1] 577.1773 (1.15) 577.1773 (1.15) 577.1773 (1.15) 0.0000 (1.0) 577.1773 (1.15) 0.0000 (1.0) 0;0 1.7326 (0.87) 1 1
test_sharing-pull-noop[2.39.0] 602.1933 (1.20) 602.1933 (1.20) 602.1933 (1.20) 0.0000 (1.0) 602.1933 (1.20) 0.0000 (1.0) 0;0 1.6606 (0.83) 1 1
test_sharing-pull-noop[2.40.0] 604.8556 (1.20) 604.8556 (1.20) 604.8556 (1.20) 0.0000 (1.0) 604.8556 (1.20) 0.0000 (1.0) 0;0 1.6533 (0.83) 1 1
test_sharing-pull-noop[2.41.1] 606.8325 (1.21) 606.8325 (1.21) 606.8325 (1.21) 0.0000 (1.0) 606.8325 (1.21) 0.0000 (1.0) 0;0 1.6479 (0.83) 1 1
test_sharing-pull-noop[2.45.0] 642.6952 (1.28) 642.6952 (1.28) 642.6952 (1.28) 0.0000 (1.0) 642.6952 (1.28) 0.0000 (1.0) 0;0 1.5559 (0.78) 1 1
test_sharing-pull-noop[main] 710.6372 (1.41) 710.6372 (1.41) 710.6372 (1.41) 0.0000 (1.0) 710.6372 (1.41) 0.0000 (1.0) 0;0 1.4072 (0.71) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_sharing-push': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.11.0] 3.8839 (1.15) 3.8839 (1.15) 3.8839 (1.15) 0.0000 (1.0) 3.8839 (1.15) 0.0000 (1.0) 0;0 0.2575 (0.87) 1 1
test_sharing-push[2.18.1] 3.8565 (1.15) 3.8565 (1.15) 3.8565 (1.15) 0.0000 (1.0) 3.8565 (1.15) 0.0000 (1.0) 0;0 0.2593 (0.87) 1 1
test_sharing-push[2.39.0] 4.9512 (1.47) 4.9512 (1.47) 4.9512 (1.47) 0.0000 (1.0) 4.9512 (1.47) 0.0000 (1.0) 0;0 0.2020 (0.68) 1 1
test_sharing-push[2.40.0] 4.9294 (1.47) 4.9294 (1.47) 4.9294 (1.47) 0.0000 (1.0) 4.9294 (1.47) 0.0000 (1.0) 0;0 0.2029 (0.68) 1 1
test_sharing-push[2.41.1] 4.9412 (1.47) 4.9412 (1.47) 4.9412 (1.47) 0.0000 (1.0) 4.9412 (1.47) 0.0000 (1.0) 0;0 0.2024 (0.68) 1 1
test_sharing-push[2.45.0] 3.3630 (1.0) 3.3630 (1.0) 3.3630 (1.0) 0.0000 (1.0) 3.3630 (1.0) 0.0000 (1.0) 0;0 0.2974 (1.0) 1 1
test_sharing-push[main] 3.4550 (1.03) 3.4550 (1.03) 3.4550 (1.03) 0.0000 (1.0) 3.4550 (1.03) 0.0000 (1.0) 0;0 0.2894 (0.97) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.11.0] 418.0701 (1.0) 418.0701 (1.0) 418.0701 (1.0) 0.0000 (1.0) 418.0701 (1.0) 0.0000 (1.0) 0;0 2.3919 (1.0) 1 1
test_sharing-push-noop[2.18.1] 431.7895 (1.03) 431.7895 (1.03) 431.7895 (1.03) 0.0000 (1.0) 431.7895 (1.03) 0.0000 (1.0) 0;0 2.3159 (0.97) 1 1
test_sharing-push-noop[2.39.0] 464.4172 (1.11) 464.4172 (1.11) 464.4172 (1.11) 0.0000 (1.0) 464.4172 (1.11) 0.0000 (1.0) 0;0 2.1532 (0.90) 1 1
test_sharing-push-noop[2.40.0] 464.4525 (1.11) 464.4525 (1.11) 464.4525 (1.11) 0.0000 (1.0) 464.4525 (1.11) 0.0000 (1.0) 0;0 2.1531 (0.90) 1 1
test_sharing-push-noop[2.41.1] 466.5615 (1.12) 466.5615 (1.12) 466.5615 (1.12) 0.0000 (1.0) 466.5615 (1.12) 0.0000 (1.0) 0;0 2.1433 (0.90) 1 1
test_sharing-push-noop[2.45.0] 468.2803 (1.12) 468.2803 (1.12) 468.2803 (1.12) 0.0000 (1.0) 468.2803 (1.12) 0.0000 (1.0) 0;0 2.1355 (0.89) 1 1
test_sharing-push-noop[main] 547.3716 (1.31) 547.3716 (1.31) 547.3716 (1.31) 0.0000 (1.0) 547.3716 (1.31) 0.0000 (1.0) 0;0 1.8269 (0.76) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_get-get': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.11.0] 7.1911 (1.56) 7.1911 (1.56) 7.1911 (1.56) 0.0000 (1.0) 7.1911 (1.56) 0.0000 (1.0) 0;0 0.1391 (0.64) 1 1
test_get-get[2.18.1] 6.3253 (1.37) 6.3253 (1.37) 6.3253 (1.37) 0.0000 (1.0) 6.3253 (1.37) 0.0000 (1.0) 0;0 0.1581 (0.73) 1 1
test_get-get[2.39.0] 4.6875 (1.02) 4.6875 (1.02) 4.6875 (1.02) 0.0000 (1.0) 4.6875 (1.02) 0.0000 (1.0) 0;0 0.2133 (0.99) 1 1
test_get-get[2.40.0] 4.6178 (1.0) 4.6178 (1.0) 4.6178 (1.0) 0.0000 (1.0) 4.6178 (1.0) 0.0000 (1.0) 0;0 0.2166 (1.0) 1 1
test_get-get[2.41.1] 4.6617 (1.01) 4.6617 (1.01) 4.6617 (1.01) 0.0000 (1.0) 4.6617 (1.01) 0.0000 (1.0) 0;0 0.2145 (0.99) 1 1
test_get-get[2.45.0] 4.6406 (1.00) 4.6406 (1.00) 4.6406 (1.00) 0.0000 (1.0) 4.6406 (1.00) 0.0000 (1.0) 0;0 0.2155 (1.00) 1 1
test_get-get[main] 6.0771 (1.32) 6.0771 (1.32) 6.0771 (1.32) 0.0000 (1.0) 6.0771 (1.32) 0.0000 (1.0) 0;0 0.1646 (0.76) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_checkout_symlink-checkout': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_symlink-checkout[2.11.0] 1.3591 (1.0) 1.3591 (1.0) 1.3591 (1.0) 0.0000 (1.0) 1.3591 (1.0) 0.0000 (1.0) 0;0 0.7358 (1.0) 1 1
test_checkout_symlink-checkout[2.18.1] 1.6183 (1.19) 1.6183 (1.19) 1.6183 (1.19) 0.0000 (1.0) 1.6183 (1.19) 0.0000 (1.0) 0;0 0.6179 (0.84) 1 1
test_checkout_symlink-checkout[2.39.0] 1.4765 (1.09) 1.4765 (1.09) 1.4765 (1.09) 0.0000 (1.0) 1.4765 (1.09) 0.0000 (1.0) 0;0 0.6773 (0.92) 1 1
test_checkout_symlink-checkout[2.40.0] 1.4336 (1.05) 1.4336 (1.05) 1.4336 (1.05) 0.0000 (1.0) 1.4336 (1.05) 0.0000 (1.0) 0;0 0.6975 (0.95) 1 1
test_checkout_symlink-checkout[2.41.1] 1.8427 (1.36) 1.8427 (1.36) 1.8427 (1.36) 0.0000 (1.0) 1.8427 (1.36) 0.0000 (1.0) 0;0 0.5427 (0.74) 1 1
test_checkout_symlink-checkout[2.45.0] 1.6933 (1.25) 1.6933 (1.25) 1.6933 (1.25) 0.0000 (1.0) 1.6933 (1.25) 0.0000 (1.0) 0;0 0.5905 (0.80) 1 1
test_checkout_symlink-checkout[main] 1.7138 (1.26) 1.7138 (1.26) 1.7138 (1.26) 0.0000 (1.0) 1.7138 (1.26) 0.0000 (1.0) 0;0 0.5835 (0.79) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_checkout_symlink-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_symlink-checkout-noop[2.11.0] 490.0490 (1.0) 490.0490 (1.0) 490.0490 (1.0) 0.0000 (1.0) 490.0490 (1.0) 0.0000 (1.0) 0;0 2.0406 (1.0) 1 1
test_checkout_symlink-checkout-noop[2.18.1] 534.3332 (1.09) 534.3332 (1.09) 534.3332 (1.09) 0.0000 (1.0) 534.3332 (1.09) 0.0000 (1.0) 0;0 1.8715 (0.92) 1 1
test_checkout_symlink-checkout-noop[2.39.0] 850.0601 (1.73) 850.0601 (1.73) 850.0601 (1.73) 0.0000 (1.0) 850.0601 (1.73) 0.0000 (1.0) 0;0 1.1764 (0.58) 1 1
test_checkout_symlink-checkout-noop[2.40.0] 598.0047 (1.22) 598.0047 (1.22) 598.0047 (1.22) 0.0000 (1.0) 598.0047 (1.22) 0.0000 (1.0) 0;0 1.6722 (0.82) 1 1
test_checkout_symlink-checkout-noop[2.41.1] 575.5484 (1.17) 575.5484 (1.17) 575.5484 (1.17) 0.0000 (1.0) 575.5484 (1.17) 0.0000 (1.0) 0;0 1.7375 (0.85) 1 1
test_checkout_symlink-checkout-noop[2.45.0] 621.2164 (1.27) 621.2164 (1.27) 621.2164 (1.27) 0.0000 (1.0) 621.2164 (1.27) 0.0000 (1.0) 0;0 1.6097 (0.79) 1 1
test_checkout_symlink-checkout-noop[main] 668.5701 (1.36) 668.5701 (1.36) 668.5701 (1.36) 0.0000 (1.0) 668.5701 (1.36) 0.0000 (1.0) 0;0 1.4957 (0.73) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_checkout_symlink-checkout-update': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_symlink-checkout-update[2.11.0] 1.3892 (2.18) 1.3892 (2.18) 1.3892 (2.18) 0.0000 (1.0) 1.3892 (2.18) 0.0000 (1.0) 0;0 0.7198 (0.46) 1 1
test_checkout_symlink-checkout-update[2.18.1] 0.6410 (1.00) 0.6410 (1.00) 0.6410 (1.00) 0.0000 (1.0) 0.6410 (1.00) 0.0000 (1.0) 0;0 1.5602 (1.00) 1 1
test_checkout_symlink-checkout-update[2.39.0] 0.6682 (1.05) 0.6682 (1.05) 0.6682 (1.05) 0.0000 (1.0) 0.6682 (1.05) 0.0000 (1.0) 0;0 1.4965 (0.96) 1 1
test_checkout_symlink-checkout-update[2.40.0] 0.6382 (1.0) 0.6382 (1.0) 0.6382 (1.0) 0.0000 (1.0) 0.6382 (1.0) 0.0000 (1.0) 0;0 1.5668 (1.0) 1 1
test_checkout_symlink-checkout-update[2.41.1] 0.6518 (1.02) 0.6518 (1.02) 0.6518 (1.02) 0.0000 (1.0) 0.6518 (1.02) 0.0000 (1.0) 0;0 1.5342 (0.98) 1 1
test_checkout_symlink-checkout-update[2.45.0] 0.6724 (1.05) 0.6724 (1.05) 0.6724 (1.05) 0.0000 (1.0) 0.6724 (1.05) 0.0000 (1.0) 0;0 1.4872 (0.95) 1 1
test_checkout_symlink-checkout-update[main] 0.7522 (1.18) 0.7522 (1.18) 0.7522 (1.18) 0.0000 (1.0) 0.7522 (1.18) 0.0000 (1.0) 0;0 1.3295 (0.85) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
---------------------------------------------------------------------------- benchmark 'test_gc-gc': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.11.0] 453.2215 (1.0) 453.2215 (1.0) 453.2215 (1.0) 0.0000 (1.0) 453.2215 (1.0) 0.0000 (1.0) 0;0 2.2064 (1.0) 1 1
test_gc-gc[2.18.1] 456.7857 (1.01) 456.7857 (1.01) 456.7857 (1.01) 0.0000 (1.0) 456.7857 (1.01) 0.0000 (1.0) 0;0 2.1892 (0.99) 1 1
test_gc-gc[2.39.0] 495.1263 (1.09) 495.1263 (1.09) 495.1263 (1.09) 0.0000 (1.0) 495.1263 (1.09) 0.0000 (1.0) 0;0 2.0197 (0.92) 1 1
test_gc-gc[2.40.0] 496.9246 (1.10) 496.9246 (1.10) 496.9246 (1.10) 0.0000 (1.0) 496.9246 (1.10) 0.0000 (1.0) 0;0 2.0124 (0.91) 1 1
test_gc-gc[2.41.1] 496.6032 (1.10) 496.6032 (1.10) 496.6032 (1.10) 0.0000 (1.0) 496.6032 (1.10) 0.0000 (1.0) 0;0 2.0137 (0.91) 1 1
test_gc-gc[2.45.0] 506.1922 (1.12) 506.1922 (1.12) 506.1922 (1.12) 0.0000 (1.0) 506.1922 (1.12) 0.0000 (1.0) 0;0 1.9755 (0.90) 1 1
test_gc-gc[main] 543.9033 (1.20) 543.9033 (1.20) 543.9033 (1.20) 0.0000 (1.0) 543.9033 (1.20) 0.0000 (1.0) 0;0 1.8386 (0.83) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_get_url-get-url': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get_url-get-url[2.11.0] 4.0589 (1.00) 4.0589 (1.00) 4.0589 (1.00) 0.0000 (1.0) 4.0589 (1.00) 0.0000 (1.0) 0;0 0.2464 (1.00) 1 1
test_get_url-get-url[2.18.1] 4.2751 (1.06) 4.2751 (1.06) 4.2751 (1.06) 0.0000 (1.0) 4.2751 (1.06) 0.0000 (1.0) 0;0 0.2339 (0.95) 1 1
test_get_url-get-url[2.39.0] 4.3521 (1.08) 4.3521 (1.08) 4.3521 (1.08) 0.0000 (1.0) 4.3521 (1.08) 0.0000 (1.0) 0;0 0.2298 (0.93) 1 1
test_get_url-get-url[2.40.0] 4.0422 (1.0) 4.0422 (1.0) 4.0422 (1.0) 0.0000 (1.0) 4.0422 (1.0) 0.0000 (1.0) 0;0 0.2474 (1.0) 1 1
test_get_url-get-url[2.41.1] 4.1320 (1.02) 4.1320 (1.02) 4.1320 (1.02) 0.0000 (1.0) 4.1320 (1.02) 0.0000 (1.0) 0;0 0.2420 (0.98) 1 1
test_get_url-get-url[2.45.0] 4.1786 (1.03) 4.1786 (1.03) 4.1786 (1.03) 0.0000 (1.0) 4.1786 (1.03) 0.0000 (1.0) 0;0 0.2393 (0.97) 1 1
test_get_url-get-url[main] 4.1729 (1.03) 4.1729 (1.03) 4.1729 (1.03) 0.0000 (1.0) 4.1729 (1.03) 0.0000 (1.0) 0;0 0.2396 (0.97) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_sharing_s3-add': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add[2.11.0] 7.3783 (1.90) 7.3783 (1.90) 7.3783 (1.90) 0.0000 (1.0) 7.3783 (1.90) 0.0000 (1.0) 0;0 0.1355 (0.53) 1 1
test_sharing_s3-add[2.18.1] 6.5506 (1.69) 6.5506 (1.69) 6.5506 (1.69) 0.0000 (1.0) 6.5506 (1.69) 0.0000 (1.0) 0;0 0.1527 (0.59) 1 1
test_sharing_s3-add[2.39.0] 6.2626 (1.61) 6.2626 (1.61) 6.2626 (1.61) 0.0000 (1.0) 6.2626 (1.61) 0.0000 (1.0) 0;0 0.1597 (0.62) 1 1
test_sharing_s3-add[2.40.0] 6.1464 (1.58) 6.1464 (1.58) 6.1464 (1.58) 0.0000 (1.0) 6.1464 (1.58) 0.0000 (1.0) 0;0 0.1627 (0.63) 1 1
test_sharing_s3-add[2.41.1] 5.9714 (1.54) 5.9714 (1.54) 5.9714 (1.54) 0.0000 (1.0) 5.9714 (1.54) 0.0000 (1.0) 0;0 0.1675 (0.65) 1 1
test_sharing_s3-add[2.45.0] 3.8854 (1.0) 3.8854 (1.0) 3.8854 (1.0) 0.0000 (1.0) 3.8854 (1.0) 0.0000 (1.0) 0;0 0.2574 (1.0) 1 1
test_sharing_s3-add[main] 3.9844 (1.03) 3.9844 (1.03) 3.9844 (1.03) 0.0000 (1.0) 3.9844 (1.03) 0.0000 (1.0) 0;0 0.2510 (0.98) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_sharing_s3-add-noop': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-add-noop[2.11.0] 3.4389 (1.38) 3.4389 (1.38) 3.4389 (1.38) 0.0000 (1.0) 3.4389 (1.38) 0.0000 (1.0) 0;0 0.2908 (0.72) 1 1
test_sharing_s3-add-noop[2.18.1] 3.7554 (1.51) 3.7554 (1.51) 3.7554 (1.51) 0.0000 (1.0) 3.7554 (1.51) 0.0000 (1.0) 0;0 0.2663 (0.66) 1 1
test_sharing_s3-add-noop[2.39.0] 2.5295 (1.02) 2.5295 (1.02) 2.5295 (1.02) 0.0000 (1.0) 2.5295 (1.02) 0.0000 (1.0) 0;0 0.3953 (0.98) 1 1
test_sharing_s3-add-noop[2.40.0] 2.5160 (1.01) 2.5160 (1.01) 2.5160 (1.01) 0.0000 (1.0) 2.5160 (1.01) 0.0000 (1.0) 0;0 0.3975 (0.99) 1 1
test_sharing_s3-add-noop[2.41.1] 2.4847 (1.0) 2.4847 (1.0) 2.4847 (1.0) 0.0000 (1.0) 2.4847 (1.0) 0.0000 (1.0) 0;0 0.4025 (1.0) 1 1
test_sharing_s3-add-noop[2.45.0] 2.5253 (1.02) 2.5253 (1.02) 2.5253 (1.02) 0.0000 (1.0) 2.5253 (1.02) 0.0000 (1.0) 0;0 0.3960 (0.98) 1 1
test_sharing_s3-add-noop[main] 2.6094 (1.05) 2.6094 (1.05) 2.6094 (1.05) 0.0000 (1.0) 2.6094 (1.05) 0.0000 (1.0) 0;0 0.3832 (0.95) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_sharing_s3-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-checkout-noop[2.11.0] 488.0705 (1.0) 488.0705 (1.0) 488.0705 (1.0) 0.0000 (1.0) 488.0705 (1.0) 0.0000 (1.0) 0;0 2.0489 (1.0) 1 1
test_sharing_s3-checkout-noop[2.18.1] 560.0827 (1.15) 560.0827 (1.15) 560.0827 (1.15) 0.0000 (1.0) 560.0827 (1.15) 0.0000 (1.0) 0;0 1.7855 (0.87) 1 1
test_sharing_s3-checkout-noop[2.39.0] 623.2170 (1.28) 623.2170 (1.28) 623.2170 (1.28) 0.0000 (1.0) 623.2170 (1.28) 0.0000 (1.0) 0;0 1.6046 (0.78) 1 1
test_sharing_s3-checkout-noop[2.40.0] 616.7271 (1.26) 616.7271 (1.26) 616.7271 (1.26) 0.0000 (1.0) 616.7271 (1.26) 0.0000 (1.0) 0;0 1.6215 (0.79) 1 1
test_sharing_s3-checkout-noop[2.41.1] 601.7862 (1.23) 601.7862 (1.23) 601.7862 (1.23) 0.0000 (1.0) 601.7862 (1.23) 0.0000 (1.0) 0;0 1.6617 (0.81) 1 1
test_sharing_s3-checkout-noop[2.45.0] 619.5331 (1.27) 619.5331 (1.27) 619.5331 (1.27) 0.0000 (1.0) 619.5331 (1.27) 0.0000 (1.0) 0;0 1.6141 (0.79) 1 1
test_sharing_s3-checkout-noop[main] 695.3172 (1.42) 695.3172 (1.42) 695.3172 (1.42) 0.0000 (1.0) 695.3172 (1.42) 0.0000 (1.0) 0;0 1.4382 (0.70) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull': 7 tests --------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull[2.11.0] 15.3276 (1.19) 15.3276 (1.19) 15.3276 (1.19) 0.0000 (1.0) 15.3276 (1.19) 0.0000 (1.0) 0;0 0.0652 (0.84) 1 1
test_sharing_s3-pull[2.18.1] 16.2324 (1.26) 16.2324 (1.26) 16.2324 (1.26) 0.0000 (1.0) 16.2324 (1.26) 0.0000 (1.0) 0;0 0.0616 (0.80) 1 1
test_sharing_s3-pull[2.39.0] 16.9418 (1.31) 16.9418 (1.31) 16.9418 (1.31) 0.0000 (1.0) 16.9418 (1.31) 0.0000 (1.0) 0;0 0.0590 (0.76) 1 1
test_sharing_s3-pull[2.40.0] 16.8823 (1.31) 16.8823 (1.31) 16.8823 (1.31) 0.0000 (1.0) 16.8823 (1.31) 0.0000 (1.0) 0;0 0.0592 (0.76) 1 1
test_sharing_s3-pull[2.41.1] 15.6694 (1.21) 15.6694 (1.21) 15.6694 (1.21) 0.0000 (1.0) 15.6694 (1.21) 0.0000 (1.0) 0;0 0.0638 (0.82) 1 1
test_sharing_s3-pull[2.45.0] 13.0528 (1.01) 13.0528 (1.01) 13.0528 (1.01) 0.0000 (1.0) 13.0528 (1.01) 0.0000 (1.0) 0;0 0.0766 (0.99) 1 1
test_sharing_s3-pull[main] 12.9069 (1.0) 12.9069 (1.0) 12.9069 (1.0) 0.0000 (1.0) 12.9069 (1.0) 0.0000 (1.0) 0;0 0.0775 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_sharing_s3-pull-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-pull-noop[2.11.0] 748.8870 (1.0) 748.8870 (1.0) 748.8870 (1.0) 0.0000 (1.0) 748.8870 (1.0) 0.0000 (1.0) 0;0 1.3353 (1.0) 1 1
test_sharing_s3-pull-noop[2.18.1] 788.6401 (1.05) 788.6401 (1.05) 788.6401 (1.05) 0.0000 (1.0) 788.6401 (1.05) 0.0000 (1.0) 0;0 1.2680 (0.95) 1 1
test_sharing_s3-pull-noop[2.39.0] 875.9894 (1.17) 875.9894 (1.17) 875.9894 (1.17) 0.0000 (1.0) 875.9894 (1.17) 0.0000 (1.0) 0;0 1.1416 (0.85) 1 1
test_sharing_s3-pull-noop[2.40.0] 872.7511 (1.17) 872.7511 (1.17) 872.7511 (1.17) 0.0000 (1.0) 872.7511 (1.17) 0.0000 (1.0) 0;0 1.1458 (0.86) 1 1
test_sharing_s3-pull-noop[2.41.1] 818.7208 (1.09) 818.7208 (1.09) 818.7208 (1.09) 0.0000 (1.0) 818.7208 (1.09) 0.0000 (1.0) 0;0 1.2214 (0.91) 1 1
test_sharing_s3-pull-noop[2.45.0] 876.6323 (1.17) 876.6323 (1.17) 876.6323 (1.17) 0.0000 (1.0) 876.6323 (1.17) 0.0000 (1.0) 0;0 1.1407 (0.85) 1 1
test_sharing_s3-pull-noop[main] 911.9369 (1.22) 911.9369 (1.22) 911.9369 (1.22) 0.0000 (1.0) 911.9369 (1.22) 0.0000 (1.0) 0;0 1.0966 (0.82) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------- benchmark 'test_sharing_s3-push': 7 tests --------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push[2.11.0] 20.8482 (2.30) 20.8482 (2.30) 20.8482 (2.30) 0.0000 (1.0) 20.8482 (2.30) 0.0000 (1.0) 0;0 0.0480 (0.43) 1 1
test_sharing_s3-push[2.18.1] 23.8079 (2.63) 23.8079 (2.63) 23.8079 (2.63) 0.0000 (1.0) 23.8079 (2.63) 0.0000 (1.0) 0;0 0.0420 (0.38) 1 1
test_sharing_s3-push[2.39.0] 24.8598 (2.75) 24.8598 (2.75) 24.8598 (2.75) 0.0000 (1.0) 24.8598 (2.75) 0.0000 (1.0) 0;0 0.0402 (0.36) 1 1
test_sharing_s3-push[2.40.0] 25.3740 (2.80) 25.3740 (2.80) 25.3740 (2.80) 0.0000 (1.0) 25.3740 (2.80) 0.0000 (1.0) 0;0 0.0394 (0.36) 1 1
test_sharing_s3-push[2.41.1] 12.2314 (1.35) 12.2314 (1.35) 12.2314 (1.35) 0.0000 (1.0) 12.2314 (1.35) 0.0000 (1.0) 0;0 0.0818 (0.74) 1 1
test_sharing_s3-push[2.45.0] 9.0526 (1.0) 9.0526 (1.0) 9.0526 (1.0) 0.0000 (1.0) 9.0526 (1.0) 0.0000 (1.0) 0;0 0.1105 (1.0) 1 1
test_sharing_s3-push[main] 9.2495 (1.02) 9.2495 (1.02) 9.2495 (1.02) 0.0000 (1.0) 9.2495 (1.02) 0.0000 (1.0) 0;0 0.1081 (0.98) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'test_sharing_s3-push-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing_s3-push-noop[2.11.0] 958.4071 (1.0) 958.4071 (1.0) 958.4071 (1.0) 0.0000 (1.0) 958.4071 (1.0) 0.0000 (1.0) 0;0 1.0434 (1.0) 1 1
test_sharing_s3-push-noop[2.18.1] 1,141.4930 (1.19) 1,141.4930 (1.19) 1,141.4930 (1.19) 0.0000 (1.0) 1,141.4930 (1.19) 0.0000 (1.0) 0;0 0.8760 (0.84) 1 1
test_sharing_s3-push-noop[2.39.0] 1,170.2557 (1.22) 1,170.2557 (1.22) 1,170.2557 (1.22) 0.0000 (1.0) 1,170.2557 (1.22) 0.0000 (1.0) 0;0 0.8545 (0.82) 1 1
test_sharing_s3-push-noop[2.40.0] 1,061.9165 (1.11) 1,061.9165 (1.11) 1,061.9165 (1.11) 0.0000 (1.0) 1,061.9165 (1.11) 0.0000 (1.0) 0;0 0.9417 (0.90) 1 1
test_sharing_s3-push-noop[2.41.1] 1,007.4258 (1.05) 1,007.4258 (1.05) 1,007.4258 (1.05) 0.0000 (1.0) 1,007.4258 (1.05) 0.0000 (1.0) 0;0 0.9926 (0.95) 1 1
test_sharing_s3-push-noop[2.45.0] 992.3677 (1.04) 992.3677 (1.04) 992.3677 (1.04) 0.0000 (1.0) 992.3677 (1.04) 0.0000 (1.0) 0;0 1.0077 (0.97) 1 1
test_sharing_s3-push-noop[main] 1,106.9142 (1.15) 1,106.9142 (1.15) 1,106.9142 (1.15) 0.0000 (1.0) 1,106.9142 (1.15) 0.0000 (1.0) 0;0 0.9034 (0.87) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
-------------------------------------------------------------------------- benchmark 'test_import_url-import-url': 7 tests --------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import_url-import-url[2.11.0] 15.9849 (1.40) 15.9849 (1.40) 15.9849 (1.40) 0.0000 (1.0) 15.9849 (1.40) 0.0000 (1.0) 0;0 0.0626 (0.72) 1 1
test_import_url-import-url[2.18.1] 14.5088 (1.27) 14.5088 (1.27) 14.5088 (1.27) 0.0000 (1.0) 14.5088 (1.27) 0.0000 (1.0) 0;0 0.0689 (0.79) 1 1
test_import_url-import-url[2.39.0] 13.8651 (1.21) 13.8651 (1.21) 13.8651 (1.21) 0.0000 (1.0) 13.8651 (1.21) 0.0000 (1.0) 0;0 0.0721 (0.83) 1 1
test_import_url-import-url[2.40.0] 14.4426 (1.26) 14.4426 (1.26) 14.4426 (1.26) 0.0000 (1.0) 14.4426 (1.26) 0.0000 (1.0) 0;0 0.0692 (0.79) 1 1
test_import_url-import-url[2.41.1] 14.5448 (1.27) 14.5448 (1.27) 14.5448 (1.27) 0.0000 (1.0) 14.5448 (1.27) 0.0000 (1.0) 0;0 0.0688 (0.79) 1 1
test_import_url-import-url[2.45.0] 11.4570 (1.0) 11.4570 (1.0) 11.4570 (1.0) 0.0000 (1.0) 11.4570 (1.0) 0.0000 (1.0) 0;0 0.0873 (1.0) 1 1
test_import_url-import-url[main] 11.9888 (1.05) 11.9888 (1.05) 11.9888 (1.05) 0.0000 (1.0) 11.9888 (1.05) 0.0000 (1.0) 0;0 0.0834 (0.96) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_status-status': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.11.0] 2.1768 (1.61) 2.1768 (1.61) 2.1768 (1.61) 0.0000 (1.0) 2.1768 (1.61) 0.0000 (1.0) 0;0 0.4594 (0.62) 1 1
test_status-status[2.18.1] 1.3518 (1.0) 1.3518 (1.0) 1.3518 (1.0) 0.0000 (1.0) 1.3518 (1.0) 0.0000 (1.0) 0;0 0.7398 (1.0) 1 1
test_status-status[2.39.0] 1.3975 (1.03) 1.3975 (1.03) 1.3975 (1.03) 0.0000 (1.0) 1.3975 (1.03) 0.0000 (1.0) 0;0 0.7155 (0.97) 1 1
test_status-status[2.40.0] 1.3844 (1.02) 1.3844 (1.02) 1.3844 (1.02) 0.0000 (1.0) 1.3844 (1.02) 0.0000 (1.0) 0;0 0.7223 (0.98) 1 1
test_status-status[2.41.1] 1.3858 (1.03) 1.3858 (1.03) 1.3858 (1.03) 0.0000 (1.0) 1.3858 (1.03) 0.0000 (1.0) 0;0 0.7216 (0.98) 1 1
test_status-status[2.45.0] 1.3923 (1.03) 1.3923 (1.03) 1.3923 (1.03) 0.0000 (1.0) 1.3923 (1.03) 0.0000 (1.0) 0;0 0.7182 (0.97) 1 1
test_status-status[main] 1.4202 (1.05) 1.4202 (1.05) 1.4202 (1.05) 0.0000 (1.0) 1.4202 (1.05) 0.0000 (1.0) 0;0 0.7042 (0.95) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_status-status-changed': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.11.0] 1.5380 (3.06) 1.5380 (3.06) 1.5380 (3.06) 0.0000 (1.0) 1.5380 (3.06) 0.0000 (1.0) 0;0 0.6502 (0.33) 1 1
test_status-status-changed[2.18.1] 0.5019 (1.0) 0.5019 (1.0) 0.5019 (1.0) 0.0000 (1.0) 0.5019 (1.0) 0.0000 (1.0) 0;0 1.9925 (1.0) 1 1
test_status-status-changed[2.39.0] 0.5360 (1.07) 0.5360 (1.07) 0.5360 (1.07) 0.0000 (1.0) 0.5360 (1.07) 0.0000 (1.0) 0;0 1.8658 (0.94) 1 1
test_status-status-changed[2.40.0] 0.5372 (1.07) 0.5372 (1.07) 0.5372 (1.07) 0.0000 (1.0) 0.5372 (1.07) 0.0000 (1.0) 0;0 1.8614 (0.93) 1 1
test_status-status-changed[2.41.1] 0.5453 (1.09) 0.5453 (1.09) 0.5453 (1.09) 0.0000 (1.0) 0.5453 (1.09) 0.0000 (1.0) 0;0 1.8338 (0.92) 1 1
test_status-status-changed[2.45.0] 0.5606 (1.12) 0.5606 (1.12) 0.5606 (1.12) 0.0000 (1.0) 0.5606 (1.12) 0.0000 (1.0) 0;0 1.7838 (0.90) 1 1
test_status-status-changed[main] 0.6351 (1.27) 0.6351 (1.27) 0.6351 (1.27) 0.0000 (1.0) 0.6351 (1.27) 0.0000 (1.0) 0;0 1.5745 (0.79) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.11.0] 663.1995 (1.32) 663.1995 (1.32) 663.1995 (1.32) 0.0000 (1.0) 663.1995 (1.32) 0.0000 (1.0) 0;0 1.5078 (0.76) 1 1
test_status-status-changed-noop[2.18.1] 503.7678 (1.0) 503.7678 (1.0) 503.7678 (1.0) 0.0000 (1.0) 503.7678 (1.0) 0.0000 (1.0) 0;0 1.9850 (1.0) 1 1
test_status-status-changed-noop[2.39.0] 530.7807 (1.05) 530.7807 (1.05) 530.7807 (1.05) 0.0000 (1.0) 530.7807 (1.05) 0.0000 (1.0) 0;0 1.8840 (0.95) 1 1
test_status-status-changed-noop[2.40.0] 536.1153 (1.06) 536.1153 (1.06) 536.1153 (1.06) 0.0000 (1.0) 536.1153 (1.06) 0.0000 (1.0) 0;0 1.8653 (0.94) 1 1
test_status-status-changed-noop[2.41.1] 537.7930 (1.07) 537.7930 (1.07) 537.7930 (1.07) 0.0000 (1.0) 537.7930 (1.07) 0.0000 (1.0) 0;0 1.8595 (0.94) 1 1
test_status-status-changed-noop[2.45.0] 565.6762 (1.12) 565.6762 (1.12) 565.6762 (1.12) 0.0000 (1.0) 565.6762 (1.12) 0.0000 (1.0) 0;0 1.7678 (0.89) 1 1
test_status-status-changed-noop[main] 625.5877 (1.24) 625.5877 (1.24) 625.5877 (1.24) 0.0000 (1.0) 625.5877 (1.24) 0.0000 (1.0) 0;0 1.5985 (0.81) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_status-status-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.11.0] 419.1669 (1.0) 419.1669 (1.0) 419.1669 (1.0) 0.0000 (1.0) 419.1669 (1.0) 0.0000 (1.0) 0;0 2.3857 (1.0) 1 1
test_status-status-noop[2.18.1] 502.2906 (1.20) 502.2906 (1.20) 502.2906 (1.20) 0.0000 (1.0) 502.2906 (1.20) 0.0000 (1.0) 0;0 1.9909 (0.83) 1 1
test_status-status-noop[2.39.0] 534.9059 (1.28) 534.9059 (1.28) 534.9059 (1.28) 0.0000 (1.0) 534.9059 (1.28) 0.0000 (1.0) 0;0 1.8695 (0.78) 1 1
test_status-status-noop[2.40.0] 533.1741 (1.27) 533.1741 (1.27) 533.1741 (1.27) 0.0000 (1.0) 533.1741 (1.27) 0.0000 (1.0) 0;0 1.8756 (0.79) 1 1
test_status-status-noop[2.41.1] 537.7998 (1.28) 537.7998 (1.28) 537.7998 (1.28) 0.0000 (1.0) 537.7998 (1.28) 0.0000 (1.0) 0;0 1.8594 (0.78) 1 1
test_status-status-noop[2.45.0] 560.2748 (1.34) 560.2748 (1.34) 560.2748 (1.34) 0.0000 (1.0) 560.2748 (1.34) 0.0000 (1.0) 0;0 1.7848 (0.75) 1 1
test_status-status-noop[main] 625.7854 (1.49) 625.7854 (1.49) 625.7854 (1.49) 0.0000 (1.0) 625.7854 (1.49) 0.0000 (1.0) 0;0 1.5980 (0.67) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_pull-pull': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pull-pull[2.11.0] 7.1077 (1.04) 7.1077 (1.04) 7.1077 (1.04) 0.0000 (1.0) 7.1077 (1.04) 0.0000 (1.0) 0;0 0.1407 (0.96) 1 1
test_pull-pull[2.18.1] 7.1355 (1.05) 7.1355 (1.05) 7.1355 (1.05) 0.0000 (1.0) 7.1355 (1.05) 0.0000 (1.0) 0;0 0.1401 (0.95) 1 1
test_pull-pull[2.39.0] 8.4301 (1.24) 8.4301 (1.24) 8.4301 (1.24) 0.0000 (1.0) 8.4301 (1.24) 0.0000 (1.0) 0;0 0.1186 (0.81) 1 1
test_pull-pull[2.40.0] 8.4150 (1.24) 8.4150 (1.24) 8.4150 (1.24) 0.0000 (1.0) 8.4150 (1.24) 0.0000 (1.0) 0;0 0.1188 (0.81) 1 1
test_pull-pull[2.41.1] 8.4993 (1.25) 8.4993 (1.25) 8.4993 (1.25) 0.0000 (1.0) 8.4993 (1.25) 0.0000 (1.0) 0;0 0.1177 (0.80) 1 1
test_pull-pull[2.45.0] 6.9189 (1.02) 6.9189 (1.02) 6.9189 (1.02) 0.0000 (1.0) 6.9189 (1.02) 0.0000 (1.0) 0;0 0.1445 (0.98) 1 1
test_pull-pull[main] 6.8123 (1.0) 6.8123 (1.0) 6.8123 (1.0) 0.0000 (1.0) 6.8123 (1.0) 0.0000 (1.0) 0;0 0.1468 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_add-add': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.11.0] 8.9633 (1.91) 8.9633 (1.91) 8.9633 (1.91) 0.0000 (1.0) 8.9633 (1.91) 0.0000 (1.0) 0;0 0.1116 (0.52) 1 1
test_add-add[2.18.1] 7.8337 (1.67) 7.8337 (1.67) 7.8337 (1.67) 0.0000 (1.0) 7.8337 (1.67) 0.0000 (1.0) 0;0 0.1277 (0.60) 1 1
test_add-add[2.39.0] 7.0812 (1.51) 7.0812 (1.51) 7.0812 (1.51) 0.0000 (1.0) 7.0812 (1.51) 0.0000 (1.0) 0;0 0.1412 (0.66) 1 1
test_add-add[2.40.0] 7.9349 (1.69) 7.9349 (1.69) 7.9349 (1.69) 0.0000 (1.0) 7.9349 (1.69) 0.0000 (1.0) 0;0 0.1260 (0.59) 1 1
test_add-add[2.41.1] 7.1771 (1.53) 7.1771 (1.53) 7.1771 (1.53) 0.0000 (1.0) 7.1771 (1.53) 0.0000 (1.0) 0;0 0.1393 (0.65) 1 1
test_add-add[2.45.0] 4.8181 (1.03) 4.8181 (1.03) 4.8181 (1.03) 0.0000 (1.0) 4.8181 (1.03) 0.0000 (1.0) 0;0 0.2075 (0.97) 1 1
test_add-add[main] 4.6976 (1.0) 4.6976 (1.0) 4.6976 (1.0) 0.0000 (1.0) 4.6976 (1.0) 0.0000 (1.0) 0;0 0.2129 (1.0) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
----------------------------------------------------------------------------- benchmark 'test_init-init': 7 tests -----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.11.0] 321.7279 (1.0) 377.2235 (1.0) 329.2311 (1.0) 6.1490 (1.83) 327.9632 (1.0) 4.0142 (1.0) 8;6 3.0374 (1.0) 100 1
test_init-init[2.18.1] 330.2270 (1.03) 460.1591 (1.22) 341.0009 (1.04) 12.6980 (3.77) 339.3554 (1.03) 5.2172 (1.30) 1;2 2.9325 (0.97) 100 1
test_init-init[2.39.0] 381.6520 (1.19) 448.6850 (1.19) 394.2164 (1.20) 9.1007 (2.70) 392.1376 (1.20) 10.9053 (2.72) 16;2 2.5367 (0.84) 100 1
test_init-init[2.40.0] 384.2718 (1.19) 417.1928 (1.11) 395.6525 (1.20) 7.8829 (2.34) 394.2918 (1.20) 12.4518 (3.10) 38;0 2.5275 (0.83) 100 1
test_init-init[2.41.1] 385.2404 (1.20) 408.7115 (1.08) 391.2745 (1.19) 4.1387 (1.23) 390.5538 (1.19) 4.3462 (1.08) 19;6 2.5558 (0.84) 100 1
test_init-init[2.45.0] 385.2935 (1.20) 403.9383 (1.07) 391.3632 (1.19) 3.3657 (1.0) 390.6446 (1.19) 4.0889 (1.02) 24;3 2.5552 (0.84) 100 1
test_init-init[main] 475.5048 (1.48) 550.5217 (1.46) 483.3544 (1.47) 7.9698 (2.37) 482.2794 (1.47) 5.8678 (1.46) 5;1 2.0689 (0.68) 100 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_push-push': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.11.0] 3.5831 (1.17) 3.5831 (1.17) 3.5831 (1.17) 0.0000 (1.0) 3.5831 (1.17) 0.0000 (1.0) 0;0 0.2791 (0.86) 1 1
test_push-push[2.18.1] 3.5411 (1.15) 3.5411 (1.15) 3.5411 (1.15) 0.0000 (1.0) 3.5411 (1.15) 0.0000 (1.0) 0;0 0.2824 (0.87) 1 1
test_push-push[2.39.0] 5.1706 (1.68) 5.1706 (1.68) 5.1706 (1.68) 0.0000 (1.0) 5.1706 (1.68) 0.0000 (1.0) 0;0 0.1934 (0.59) 1 1
test_push-push[2.40.0] 5.1297 (1.67) 5.1297 (1.67) 5.1297 (1.67) 0.0000 (1.0) 5.1297 (1.67) 0.0000 (1.0) 0;0 0.1949 (0.60) 1 1
test_push-push[2.41.1] 4.5330 (1.47) 4.5330 (1.47) 4.5330 (1.47) 0.0000 (1.0) 4.5330 (1.47) 0.0000 (1.0) 0;0 0.2206 (0.68) 1 1
test_push-push[2.45.0] 3.0746 (1.0) 3.0746 (1.0) 3.0746 (1.0) 0.0000 (1.0) 3.0746 (1.0) 0.0000 (1.0) 0;0 0.3252 (1.0) 1 1
test_push-push[main] 3.1392 (1.02) 3.1392 (1.02) 3.1392 (1.02) 0.0000 (1.0) 3.1392 (1.02) 0.0000 (1.0) 0;0 0.3185 (0.98) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_checkout_hardlink-checkout': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_hardlink-checkout[2.11.0] 1.2323 (1.00) 1.2323 (1.00) 1.2323 (1.00) 0.0000 (1.0) 1.2323 (1.00) 0.0000 (1.0) 0;0 0.8115 (1.00) 1 1
test_checkout_hardlink-checkout[2.18.1] 1.2363 (1.01) 1.2363 (1.01) 1.2363 (1.01) 0.0000 (1.0) 1.2363 (1.01) 0.0000 (1.0) 0;0 0.8089 (0.99) 1 1
test_checkout_hardlink-checkout[2.39.0] 1.6723 (1.36) 1.6723 (1.36) 1.6723 (1.36) 0.0000 (1.0) 1.6723 (1.36) 0.0000 (1.0) 0;0 0.5980 (0.73) 1 1
test_checkout_hardlink-checkout[2.40.0] 1.2595 (1.03) 1.2595 (1.03) 1.2595 (1.03) 0.0000 (1.0) 1.2595 (1.03) 0.0000 (1.0) 0;0 0.7939 (0.98) 1 1
test_checkout_hardlink-checkout[2.41.1] 1.2282 (1.0) 1.2282 (1.0) 1.2282 (1.0) 0.0000 (1.0) 1.2282 (1.0) 0.0000 (1.0) 0;0 0.8142 (1.0) 1 1
test_checkout_hardlink-checkout[2.45.0] 2.0999 (1.71) 2.0999 (1.71) 2.0999 (1.71) 0.0000 (1.0) 2.0999 (1.71) 0.0000 (1.0) 0;0 0.4762 (0.58) 1 1
test_checkout_hardlink-checkout[main] 1.6113 (1.31) 1.6113 (1.31) 1.6113 (1.31) 0.0000 (1.0) 1.6113 (1.31) 0.0000 (1.0) 0;0 0.6206 (0.76) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_checkout_hardlink-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_hardlink-checkout-noop[2.11.0] 482.4815 (1.0) 482.4815 (1.0) 482.4815 (1.0) 0.0000 (1.0) 482.4815 (1.0) 0.0000 (1.0) 0;0 2.0726 (1.0) 1 1
test_checkout_hardlink-checkout-noop[2.18.1] 552.7674 (1.15) 552.7674 (1.15) 552.7674 (1.15) 0.0000 (1.0) 552.7674 (1.15) 0.0000 (1.0) 0;0 1.8091 (0.87) 1 1
test_checkout_hardlink-checkout-noop[2.39.0] 588.9815 (1.22) 588.9815 (1.22) 588.9815 (1.22) 0.0000 (1.0) 588.9815 (1.22) 0.0000 (1.0) 0;0 1.6978 (0.82) 1 1
test_checkout_hardlink-checkout-noop[2.40.0] 593.3728 (1.23) 593.3728 (1.23) 593.3728 (1.23) 0.0000 (1.0) 593.3728 (1.23) 0.0000 (1.0) 0;0 1.6853 (0.81) 1 1
test_checkout_hardlink-checkout-noop[2.41.1] 583.7195 (1.21) 583.7195 (1.21) 583.7195 (1.21) 0.0000 (1.0) 583.7195 (1.21) 0.0000 (1.0) 0;0 1.7132 (0.83) 1 1
test_checkout_hardlink-checkout-noop[2.45.0] 598.2728 (1.24) 598.2728 (1.24) 598.2728 (1.24) 0.0000 (1.0) 598.2728 (1.24) 0.0000 (1.0) 0;0 1.6715 (0.81) 1 1
test_checkout_hardlink-checkout-noop[main] 665.2357 (1.38) 665.2357 (1.38) 665.2357 (1.38) 0.0000 (1.0) 665.2357 (1.38) 0.0000 (1.0) 0;0 1.5032 (0.73) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_checkout_hardlink-checkout-update': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_hardlink-checkout-update[2.11.0] 1.2534 (2.00) 1.2534 (2.00) 1.2534 (2.00) 0.0000 (1.0) 1.2534 (2.00) 0.0000 (1.0) 0;0 0.7978 (0.50) 1 1
test_checkout_hardlink-checkout-update[2.18.1] 0.6265 (1.0) 0.6265 (1.0) 0.6265 (1.0) 0.0000 (1.0) 0.6265 (1.0) 0.0000 (1.0) 0;0 1.5961 (1.0) 1 1
test_checkout_hardlink-checkout-update[2.39.0] 0.6623 (1.06) 0.6623 (1.06) 0.6623 (1.06) 0.0000 (1.0) 0.6623 (1.06) 0.0000 (1.0) 0;0 1.5098 (0.95) 1 1
test_checkout_hardlink-checkout-update[2.40.0] 0.6628 (1.06) 0.6628 (1.06) 0.6628 (1.06) 0.0000 (1.0) 0.6628 (1.06) 0.0000 (1.0) 0;0 1.5088 (0.95) 1 1
test_checkout_hardlink-checkout-update[2.41.1] 0.6704 (1.07) 0.6704 (1.07) 0.6704 (1.07) 0.0000 (1.0) 0.6704 (1.07) 0.0000 (1.0) 0;0 1.4917 (0.93) 1 1
test_checkout_hardlink-checkout-update[2.45.0] 0.6666 (1.06) 0.6666 (1.06) 0.6666 (1.06) 0.0000 (1.0) 0.6666 (1.06) 0.0000 (1.0) 0;0 1.5002 (0.94) 1 1
test_checkout_hardlink-checkout-update[main] 0.7300 (1.17) 0.7300 (1.17) 0.7300 (1.17) 0.0000 (1.0) 0.7300 (1.17) 0.0000 (1.0) 0;0 1.3698 (0.86) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_update-update': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_update-update[2.11.0] 9.2823 (1.26) 9.2823 (1.26) 9.2823 (1.26) 0.0000 (1.0) 9.2823 (1.26) 0.0000 (1.0) 0;0 0.1077 (0.80) 1 1
test_update-update[2.18.1] 8.2768 (1.12) 8.2768 (1.12) 8.2768 (1.12) 0.0000 (1.0) 8.2768 (1.12) 0.0000 (1.0) 0;0 0.1208 (0.89) 1 1
test_update-update[2.39.0] 7.4265 (1.01) 7.4265 (1.01) 7.4265 (1.01) 0.0000 (1.0) 7.4265 (1.01) 0.0000 (1.0) 0;0 0.1347 (0.99) 1 1
test_update-update[2.40.0] 7.4555 (1.01) 7.4555 (1.01) 7.4555 (1.01) 0.0000 (1.0) 7.4555 (1.01) 0.0000 (1.0) 0;0 0.1341 (0.99) 1 1
test_update-update[2.41.1] 7.3881 (1.0) 7.3881 (1.0) 7.3881 (1.0) 0.0000 (1.0) 7.3881 (1.0) 0.0000 (1.0) 0;0 0.1354 (1.0) 1 1
test_update-update[2.45.0] 7.5161 (1.02) 7.5161 (1.02) 7.5161 (1.02) 0.0000 (1.0) 7.5161 (1.02) 0.0000 (1.0) 0;0 0.1330 (0.98) 1 1
test_update-update[main] 7.8208 (1.06) 7.8208 (1.06) 7.8208 (1.06) 0.0000 (1.0) 7.8208 (1.06) 0.0000 (1.0) 0;0 0.1279 (0.94) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'test_update-update-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_update-update-noop[2.11.0] 497.8569 (1.0) 497.8569 (1.0) 497.8569 (1.0) 0.0000 (1.0) 497.8569 (1.0) 0.0000 (1.0) 0;0 2.0086 (1.0) 1 1
test_update-update-noop[2.18.1] 647.0759 (1.30) 647.0759 (1.30) 647.0759 (1.30) 0.0000 (1.0) 647.0759 (1.30) 0.0000 (1.0) 0;0 1.5454 (0.77) 1 1
test_update-update-noop[2.39.0] 950.3605 (1.91) 950.3605 (1.91) 950.3605 (1.91) 0.0000 (1.0) 950.3605 (1.91) 0.0000 (1.0) 0;0 1.0522 (0.52) 1 1
test_update-update-noop[2.40.0] 942.4985 (1.89) 942.4985 (1.89) 942.4985 (1.89) 0.0000 (1.0) 942.4985 (1.89) 0.0000 (1.0) 0;0 1.0610 (0.53) 1 1
test_update-update-noop[2.41.1] 950.5291 (1.91) 950.5291 (1.91) 950.5291 (1.91) 0.0000 (1.0) 950.5291 (1.91) 0.0000 (1.0) 0;0 1.0520 (0.52) 1 1
test_update-update-noop[2.45.0] 1,052.0630 (2.11) 1,052.0630 (2.11) 1,052.0630 (2.11) 0.0000 (1.0) 1,052.0630 (2.11) 0.0000 (1.0) 0;0 0.9505 (0.47) 1 1
test_update-update-noop[main] 1,125.5057 (2.26) 1,125.5057 (2.26) 1,125.5057 (2.26) 0.0000 (1.0) 1,125.5057 (2.26) 0.0000 (1.0) 0;0 0.8885 (0.44) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
-------------------------------------------------------------------------------- benchmark 'test_data_status-data-changed': 6 tests --------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-changed[2.18.1] 970.3188 (1.03) 970.3188 (1.03) 970.3188 (1.03) 0.0000 (1.0) 970.3188 (1.03) 0.0000 (1.0) 0;0 1.0306 (0.97) 1 1
test_data_status-data-changed[2.39.0] 989.9591 (1.05) 989.9591 (1.05) 989.9591 (1.05) 0.0000 (1.0) 989.9591 (1.05) 0.0000 (1.0) 0;0 1.0101 (0.95) 1 1
test_data_status-data-changed[2.40.0] 986.3695 (1.05) 986.3695 (1.05) 986.3695 (1.05) 0.0000 (1.0) 986.3695 (1.05) 0.0000 (1.0) 0;0 1.0138 (0.96) 1 1
test_data_status-data-changed[2.41.1] 978.3756 (1.04) 978.3756 (1.04) 978.3756 (1.04) 0.0000 (1.0) 978.3756 (1.04) 0.0000 (1.0) 0;0 1.0221 (0.96) 1 1
test_data_status-data-changed[2.45.0] 943.0480 (1.0) 943.0480 (1.0) 943.0480 (1.0) 0.0000 (1.0) 943.0480 (1.0) 0.0000 (1.0) 0;0 1.0604 (1.0) 1 1
test_data_status-data-changed[main] 1,023.2969 (1.09) 1,023.2969 (1.09) 1,023.2969 (1.09) 0.0000 (1.0) 1,023.2969 (1.09) 0.0000 (1.0) 0;0 0.9772 (0.92) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'test_data_status-data-changed-noop': 6 tests --------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-changed-noop[2.18.1] 989.2948 (1.04) 989.2948 (1.04) 989.2948 (1.04) 0.0000 (1.0) 989.2948 (1.04) 0.0000 (1.0) 0;0 1.0108 (0.96) 1 1
test_data_status-data-changed-noop[2.39.0] 981.7812 (1.03) 981.7812 (1.03) 981.7812 (1.03) 0.0000 (1.0) 981.7812 (1.03) 0.0000 (1.0) 0;0 1.0186 (0.97) 1 1
test_data_status-data-changed-noop[2.40.0] 987.1513 (1.04) 987.1513 (1.04) 987.1513 (1.04) 0.0000 (1.0) 987.1513 (1.04) 0.0000 (1.0) 0;0 1.0130 (0.96) 1 1
test_data_status-data-changed-noop[2.41.1] 1,036.2777 (1.09) 1,036.2777 (1.09) 1,036.2777 (1.09) 0.0000 (1.0) 1,036.2777 (1.09) 0.0000 (1.0) 0;0 0.9650 (0.92) 1 1
test_data_status-data-changed-noop[2.45.0] 948.7065 (1.0) 948.7065 (1.0) 948.7065 (1.0) 0.0000 (1.0) 948.7065 (1.0) 0.0000 (1.0) 0;0 1.0541 (1.0) 1 1
test_data_status-data-changed-noop[main] 1,009.4060 (1.06) 1,009.4060 (1.06) 1,009.4060 (1.06) 0.0000 (1.0) 1,009.4060 (1.06) 0.0000 (1.0) 0;0 0.9907 (0.94) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_data_status-data-new': 6 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-new[2.18.1] 2.2265 (1.08) 2.2265 (1.08) 2.2265 (1.08) 0.0000 (1.0) 2.2265 (1.08) 0.0000 (1.0) 0;0 0.4491 (0.92) 1 1
test_data_status-data-new[2.39.0] 2.2234 (1.08) 2.2234 (1.08) 2.2234 (1.08) 0.0000 (1.0) 2.2234 (1.08) 0.0000 (1.0) 0;0 0.4498 (0.92) 1 1
test_data_status-data-new[2.40.0] 2.1856 (1.06) 2.1856 (1.06) 2.1856 (1.06) 0.0000 (1.0) 2.1856 (1.06) 0.0000 (1.0) 0;0 0.4575 (0.94) 1 1
test_data_status-data-new[2.41.1] 2.2050 (1.07) 2.2050 (1.07) 2.2050 (1.07) 0.0000 (1.0) 2.2050 (1.07) 0.0000 (1.0) 0;0 0.4535 (0.93) 1 1
test_data_status-data-new[2.45.0] 2.0566 (1.0) 2.0566 (1.0) 2.0566 (1.0) 0.0000 (1.0) 2.0566 (1.0) 0.0000 (1.0) 0;0 0.4862 (1.0) 1 1
test_data_status-data-new[main] 2.1883 (1.06) 2.1883 (1.06) 2.1883 (1.06) 0.0000 (1.0) 2.1883 (1.06) 0.0000 (1.0) 0;0 0.4570 (0.94) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'test_data_status-data-noop': 6 tests --------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-noop[2.18.1] 960.1092 (1.02) 960.1092 (1.02) 960.1092 (1.02) 0.0000 (1.0) 960.1092 (1.02) 0.0000 (1.0) 0;0 1.0415 (0.98) 1 1
test_data_status-data-noop[2.39.0] 968.6676 (1.03) 968.6676 (1.03) 968.6676 (1.03) 0.0000 (1.0) 968.6676 (1.03) 0.0000 (1.0) 0;0 1.0323 (0.97) 1 1
test_data_status-data-noop[2.40.0] 990.4174 (1.06) 990.4174 (1.06) 990.4174 (1.06) 0.0000 (1.0) 990.4174 (1.06) 0.0000 (1.0) 0;0 1.0097 (0.95) 1 1
test_data_status-data-noop[2.41.1] 993.4122 (1.06) 993.4122 (1.06) 993.4122 (1.06) 0.0000 (1.0) 993.4122 (1.06) 0.0000 (1.0) 0;0 1.0066 (0.94) 1 1
test_data_status-data-noop[2.45.0] 937.5600 (1.0) 937.5600 (1.0) 937.5600 (1.0) 0.0000 (1.0) 937.5600 (1.0) 0.0000 (1.0) 0;0 1.0666 (1.0) 1 1
test_data_status-data-noop[main] 1,034.0477 (1.10) 1,034.0477 (1.10) 1,034.0477 (1.10) 0.0000 (1.0) 1,034.0477 (1.10) 0.0000 (1.0) 0;0 0.9671 (0.91) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_exp_show-exp': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.11.0] 4.4236 (1.11) 4.4236 (1.11) 4.4236 (1.11) 0.0000 (1.0) 4.4236 (1.11) 0.0000 (1.0) 0;0 0.2261 (0.90) 1 1
test_exp_show-exp[2.18.1] 4.6276 (1.16) 4.6276 (1.16) 4.6276 (1.16) 0.0000 (1.0) 4.6276 (1.16) 0.0000 (1.0) 0;0 0.2161 (0.86) 1 1
test_exp_show-exp[2.39.0] 4.1393 (1.04) 4.1393 (1.04) 4.1393 (1.04) 0.0000 (1.0) 4.1393 (1.04) 0.0000 (1.0) 0;0 0.2416 (0.96) 1 1
test_exp_show-exp[2.40.0] 4.0116 (1.01) 4.0116 (1.01) 4.0116 (1.01) 0.0000 (1.0) 4.0116 (1.01) 0.0000 (1.0) 0;0 0.2493 (0.99) 1 1
test_exp_show-exp[2.41.1] 3.9830 (1.0) 3.9830 (1.0) 3.9830 (1.0) 0.0000 (1.0) 3.9830 (1.0) 0.0000 (1.0) 0;0 0.2511 (1.0) 1 1
test_exp_show-exp[2.45.0] 4.4694 (1.12) 4.4694 (1.12) 4.4694 (1.12) 0.0000 (1.0) 4.4694 (1.12) 0.0000 (1.0) 0;0 0.2237 (0.89) 1 1
test_exp_show-exp[main] 5.4401 (1.37) 5.4401 (1.37) 5.4401 (1.37) 0.0000 (1.0) 5.4401 (1.37) 0.0000 (1.0) 0;0 0.1838 (0.73) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
---------------------------------------------------------------------------- benchmark 'test_list-list': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.11.0] 551.7096 (1.14) 551.7096 (1.14) 551.7096 (1.14) 0.0000 (1.0) 551.7096 (1.14) 0.0000 (1.0) 0;0 1.8125 (0.88) 1 1
test_list-list[2.18.1] 482.9626 (1.0) 482.9626 (1.0) 482.9626 (1.0) 0.0000 (1.0) 482.9626 (1.0) 0.0000 (1.0) 0;0 2.0706 (1.0) 1 1
test_list-list[2.39.0] 541.4399 (1.12) 541.4399 (1.12) 541.4399 (1.12) 0.0000 (1.0) 541.4399 (1.12) 0.0000 (1.0) 0;0 1.8469 (0.89) 1 1
test_list-list[2.40.0] 536.1374 (1.11) 536.1374 (1.11) 536.1374 (1.11) 0.0000 (1.0) 536.1374 (1.11) 0.0000 (1.0) 0;0 1.8652 (0.90) 1 1
test_list-list[2.41.1] 518.6946 (1.07) 518.6946 (1.07) 518.6946 (1.07) 0.0000 (1.0) 518.6946 (1.07) 0.0000 (1.0) 0;0 1.9279 (0.93) 1 1
test_list-list[2.45.0] 496.0335 (1.03) 496.0335 (1.03) 496.0335 (1.03) 0.0000 (1.0) 496.0335 (1.03) 0.0000 (1.0) 0;0 2.0160 (0.97) 1 1
test_list-list[main] 623.9065 (1.29) 623.9065 (1.29) 623.9065 (1.29) 0.0000 (1.0) 623.9065 (1.29) 0.0000 (1.0) 0;0 1.6028 (0.77) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_list-list-dvc-only': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.11.0] 508.8249 (1.11) 508.8249 (1.11) 508.8249 (1.11) 0.0000 (1.0) 508.8249 (1.11) 0.0000 (1.0) 0;0 1.9653 (0.90) 1 1
test_list-list-dvc-only[2.18.1] 458.3956 (1.0) 458.3956 (1.0) 458.3956 (1.0) 0.0000 (1.0) 458.3956 (1.0) 0.0000 (1.0) 0;0 2.1815 (1.0) 1 1
test_list-list-dvc-only[2.39.0] 495.2944 (1.08) 495.2944 (1.08) 495.2944 (1.08) 0.0000 (1.0) 495.2944 (1.08) 0.0000 (1.0) 0;0 2.0190 (0.93) 1 1
test_list-list-dvc-only[2.40.0] 500.8101 (1.09) 500.8101 (1.09) 500.8101 (1.09) 0.0000 (1.0) 500.8101 (1.09) 0.0000 (1.0) 0;0 1.9968 (0.92) 1 1
test_list-list-dvc-only[2.41.1] 502.1944 (1.10) 502.1944 (1.10) 502.1944 (1.10) 0.0000 (1.0) 502.1944 (1.10) 0.0000 (1.0) 0;0 1.9913 (0.91) 1 1
test_list-list-dvc-only[2.45.0] 480.2094 (1.05) 480.2094 (1.05) 480.2094 (1.05) 0.0000 (1.0) 480.2094 (1.05) 0.0000 (1.0) 0;0 2.0824 (0.95) 1 1
test_list-list-dvc-only[main] 594.5774 (1.30) 594.5774 (1.30) 594.5774 (1.30) 0.0000 (1.0) 594.5774 (1.30) 0.0000 (1.0) 0;0 1.6819 (0.77) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_list-list-recursive': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-recursive[2.11.0] 2.2726 (4.17) 2.2726 (4.17) 2.2726 (4.17) 0.0000 (1.0) 2.2726 (4.17) 0.0000 (1.0) 0;0 0.4400 (0.24) 1 1
test_list-list-recursive[2.18.1] 3.1613 (5.81) 3.1613 (5.81) 3.1613 (5.81) 0.0000 (1.0) 3.1613 (5.81) 0.0000 (1.0) 0;0 0.3163 (0.17) 1 1
test_list-list-recursive[2.39.0] 1.0104 (1.86) 1.0104 (1.86) 1.0104 (1.86) 0.0000 (1.0) 1.0104 (1.86) 0.0000 (1.0) 0;0 0.9897 (0.54) 1 1
test_list-list-recursive[2.40.0] 0.6195 (1.14) 0.6195 (1.14) 0.6195 (1.14) 0.0000 (1.0) 0.6195 (1.14) 0.0000 (1.0) 0;0 1.6143 (0.88) 1 1
test_list-list-recursive[2.41.1] 0.6331 (1.16) 0.6331 (1.16) 0.6331 (1.16) 0.0000 (1.0) 0.6331 (1.16) 0.0000 (1.0) 0;0 1.5794 (0.86) 1 1
test_list-list-recursive[2.45.0] 0.5444 (1.0) 0.5444 (1.0) 0.5444 (1.0) 0.0000 (1.0) 0.5444 (1.0) 0.0000 (1.0) 0;0 1.8370 (1.0) 1 1
test_list-list-recursive[main] 0.7310 (1.34) 0.7310 (1.34) 0.7310 (1.34) 0.0000 (1.0) 0.7310 (1.34) 0.0000 (1.0) 0;0 1.3680 (0.74) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_list-list-shallow': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-shallow[2.11.0] 518.1622 (1.01) 518.1622 (1.01) 518.1622 (1.01) 0.0000 (1.0) 518.1622 (1.01) 0.0000 (1.0) 0;0 1.9299 (0.99) 1 1
test_list-list-shallow[2.18.1] 528.9325 (1.04) 528.9325 (1.04) 528.9325 (1.04) 0.0000 (1.0) 528.9325 (1.04) 0.0000 (1.0) 0;0 1.8906 (0.97) 1 1
test_list-list-shallow[2.39.0] 527.8700 (1.03) 527.8700 (1.03) 527.8700 (1.03) 0.0000 (1.0) 527.8700 (1.03) 0.0000 (1.0) 0;0 1.8944 (0.97) 1 1
test_list-list-shallow[2.40.0] 529.2901 (1.04) 529.2901 (1.04) 529.2901 (1.04) 0.0000 (1.0) 529.2901 (1.04) 0.0000 (1.0) 0;0 1.8893 (0.97) 1 1
test_list-list-shallow[2.41.1] 510.7695 (1.0) 510.7695 (1.0) 510.7695 (1.0) 0.0000 (1.0) 510.7695 (1.0) 0.0000 (1.0) 0;0 1.9578 (1.0) 1 1
test_list-list-shallow[2.45.0] 532.5051 (1.04) 532.5051 (1.04) 532.5051 (1.04) 0.0000 (1.0) 532.5051 (1.04) 0.0000 (1.0) 0;0 1.8779 (0.96) 1 1
test_list-list-shallow[main] 598.0447 (1.17) 598.0447 (1.17) 598.0447 (1.17) 0.0000 (1.0) 598.0447 (1.17) 0.0000 (1.0) 0;0 1.6721 (0.85) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
---------------------------------------------------------------------------- benchmark 'test_help---help': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.11.0] 139.8357 (1.0) 142.9576 (1.0) 141.3801 (1.0) 0.7866 (1.28) 141.4010 (1.0) 1.2534 (2.15) 38;0 7.0731 (1.0) 100 1
test_help---help[2.18.1] 144.1368 (1.03) 147.7917 (1.03) 145.4247 (1.03) 0.6137 (1.0) 145.4165 (1.03) 0.5830 (1.0) 26;7 6.8764 (0.97) 100 1
test_help---help[2.39.0] 184.1788 (1.32) 188.5590 (1.32) 186.0669 (1.32) 0.7990 (1.30) 186.0238 (1.32) 1.0098 (1.73) 29;1 5.3744 (0.76) 100 1
test_help---help[2.40.0] 184.4778 (1.32) 190.6915 (1.33) 187.0247 (1.32) 1.1473 (1.87) 187.1182 (1.32) 1.2444 (2.13) 26;3 5.3469 (0.76) 100 1
test_help---help[2.41.1] 183.9836 (1.32) 187.9212 (1.31) 185.5688 (1.31) 0.8425 (1.37) 185.4094 (1.31) 1.2218 (2.10) 33;0 5.3888 (0.76) 100 1
test_help---help[2.45.0] 145.9405 (1.04) 154.6789 (1.08) 147.0725 (1.04) 1.0028 (1.63) 146.8884 (1.04) 0.6922 (1.19) 10;4 6.7994 (0.96) 100 1
test_help---help[main] 146.8541 (1.05) 153.2467 (1.07) 148.1672 (1.05) 0.8606 (1.40) 148.0914 (1.05) 1.0080 (1.73) 22;2 6.7491 (0.95) 100 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_diff-diff': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.11.0] 2.4925 (1.25) 2.4925 (1.25) 2.4925 (1.25) 0.0000 (1.0) 2.4925 (1.25) 0.0000 (1.0) 0;0 0.4012 (0.80) 1 1
test_diff-diff[2.18.1] 2.1975 (1.10) 2.1975 (1.10) 2.1975 (1.10) 0.0000 (1.0) 2.1975 (1.10) 0.0000 (1.0) 0;0 0.4551 (0.91) 1 1
test_diff-diff[2.39.0] 2.1887 (1.10) 2.1887 (1.10) 2.1887 (1.10) 0.0000 (1.0) 2.1887 (1.10) 0.0000 (1.0) 0;0 0.4569 (0.91) 1 1
test_diff-diff[2.40.0] 2.1512 (1.08) 2.1512 (1.08) 2.1512 (1.08) 0.0000 (1.0) 2.1512 (1.08) 0.0000 (1.0) 0;0 0.4649 (0.93) 1 1
test_diff-diff[2.41.1] 2.0984 (1.05) 2.0984 (1.05) 2.0984 (1.05) 0.0000 (1.0) 2.0984 (1.05) 0.0000 (1.0) 0;0 0.4765 (0.95) 1 1
test_diff-diff[2.45.0] 2.0631 (1.04) 2.0631 (1.04) 2.0631 (1.04) 0.0000 (1.0) 2.0631 (1.04) 0.0000 (1.0) 0;0 0.4847 (0.97) 1 1
test_diff-diff[main] 1.9930 (1.0) 1.9930 (1.0) 1.9930 (1.0) 0.0000 (1.0) 1.9930 (1.0) 0.0000 (1.0) 0;0 0.5017 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_diff-diff-changed': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.11.0] 1.5976 (1.67) 1.5976 (1.67) 1.5976 (1.67) 0.0000 (1.0) 1.5976 (1.67) 0.0000 (1.0) 0;0 0.6259 (0.60) 1 1
test_diff-diff-changed[2.18.1] 0.9624 (1.01) 0.9624 (1.01) 0.9624 (1.01) 0.0000 (1.0) 0.9624 (1.01) 0.0000 (1.0) 0;0 1.0390 (0.99) 1 1
test_diff-diff-changed[2.39.0] 0.9807 (1.03) 0.9807 (1.03) 0.9807 (1.03) 0.0000 (1.0) 0.9807 (1.03) 0.0000 (1.0) 0;0 1.0197 (0.98) 1 1
test_diff-diff-changed[2.40.0] 1.0026 (1.05) 1.0026 (1.05) 1.0026 (1.05) 0.0000 (1.0) 1.0026 (1.05) 0.0000 (1.0) 0;0 0.9974 (0.95) 1 1
test_diff-diff-changed[2.41.1] 0.9567 (1.0) 0.9567 (1.0) 0.9567 (1.0) 0.0000 (1.0) 0.9567 (1.0) 0.0000 (1.0) 0;0 1.0453 (1.0) 1 1
test_diff-diff-changed[2.45.0] 0.9892 (1.03) 0.9892 (1.03) 0.9892 (1.03) 0.0000 (1.0) 0.9892 (1.03) 0.0000 (1.0) 0;0 1.0109 (0.97) 1 1
test_diff-diff-changed[main] 1.1274 (1.18) 1.1274 (1.18) 1.1274 (1.18) 0.0000 (1.0) 1.1274 (1.18) 0.0000 (1.0) 0;0 0.8870 (0.85) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 7 tests --------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.11.0] 664.7508 (1.0) 664.7508 (1.0) 664.7508 (1.0) 0.0000 (1.0) 664.7508 (1.0) 0.0000 (1.0) 0;0 1.5043 (1.0) 1 1
test_diff-diff-changed-noop[2.18.1] 1,050.6990 (1.58) 1,050.6990 (1.58) 1,050.6990 (1.58) 0.0000 (1.0) 1,050.6990 (1.58) 0.0000 (1.0) 0;0 0.9517 (0.63) 1 1
test_diff-diff-changed-noop[2.39.0] 1,004.5120 (1.51) 1,004.5120 (1.51) 1,004.5120 (1.51) 0.0000 (1.0) 1,004.5120 (1.51) 0.0000 (1.0) 0;0 0.9955 (0.66) 1 1
test_diff-diff-changed-noop[2.40.0] 1,004.3881 (1.51) 1,004.3881 (1.51) 1,004.3881 (1.51) 0.0000 (1.0) 1,004.3881 (1.51) 0.0000 (1.0) 0;0 0.9956 (0.66) 1 1
test_diff-diff-changed-noop[2.41.1] 965.7691 (1.45) 965.7691 (1.45) 965.7691 (1.45) 0.0000 (1.0) 965.7691 (1.45) 0.0000 (1.0) 0;0 1.0354 (0.69) 1 1
test_diff-diff-changed-noop[2.45.0] 1,008.2296 (1.52) 1,008.2296 (1.52) 1,008.2296 (1.52) 0.0000 (1.0) 1,008.2296 (1.52) 0.0000 (1.0) 0;0 0.9918 (0.66) 1 1
test_diff-diff-changed-noop[main] 1,036.2767 (1.56) 1,036.2767 (1.56) 1,036.2767 (1.56) 0.0000 (1.0) 1,036.2767 (1.56) 0.0000 (1.0) 0;0 0.9650 (0.64) 1 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.11.0] 596.3634 (1.0) 596.3634 (1.0) 596.3634 (1.0) 0.0000 (1.0) 596.3634 (1.0) 0.0000 (1.0) 0;0 1.6768 (1.0) 1 1
test_diff-diff-noop[2.18.1] 902.6175 (1.51) 902.6175 (1.51) 902.6175 (1.51) 0.0000 (1.0) 902.6175 (1.51) 0.0000 (1.0) 0;0 1.1079 (0.66) 1 1
test_diff-diff-noop[2.39.0] 923.5446 (1.55) 923.5446 (1.55) 923.5446 (1.55) 0.0000 (1.0) 923.5446 (1.55) 0.0000 (1.0) 0;0 1.0828 (0.65) 1 1
test_diff-diff-noop[2.40.0] 981.8004 (1.65) 981.8004 (1.65) 981.8004 (1.65) 0.0000 (1.0) 981.8004 (1.65) 0.0000 (1.0) 0;0 1.0185 (0.61) 1 1
test_diff-diff-noop[2.41.1] 912.0100 (1.53) 912.0100 (1.53) 912.0100 (1.53) 0.0000 (1.0) 912.0100 (1.53) 0.0000 (1.0) 0;0 1.0965 (0.65) 1 1
test_diff-diff-noop[2.45.0] 952.6630 (1.60) 952.6630 (1.60) 952.6630 (1.60) 0.0000 (1.0) 952.6630 (1.60) 0.0000 (1.0) 0;0 1.0497 (0.63) 1 1
test_diff-diff-noop[main] 993.5702 (1.67) 993.5702 (1.67) 993.5702 (1.67) 0.0000 (1.0) 993.5702 (1.67) 0.0000 (1.0) 0;0 1.0065 (0.60) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_checkout_copy-checkout': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_copy-checkout[2.11.0] 3.1433 (1.01) 3.1433 (1.01) 3.1433 (1.01) 0.0000 (1.0) 3.1433 (1.01) 0.0000 (1.0) 0;0 0.3181 (0.99) 1 1
test_checkout_copy-checkout[2.18.1] 3.1191 (1.0) 3.1191 (1.0) 3.1191 (1.0) 0.0000 (1.0) 3.1191 (1.0) 0.0000 (1.0) 0;0 0.3206 (1.0) 1 1
test_checkout_copy-checkout[2.39.0] 3.2337 (1.04) 3.2337 (1.04) 3.2337 (1.04) 0.0000 (1.0) 3.2337 (1.04) 0.0000 (1.0) 0;0 0.3092 (0.96) 1 1
test_checkout_copy-checkout[2.40.0] 3.5765 (1.15) 3.5765 (1.15) 3.5765 (1.15) 0.0000 (1.0) 3.5765 (1.15) 0.0000 (1.0) 0;0 0.2796 (0.87) 1 1
test_checkout_copy-checkout[2.41.1] 3.2695 (1.05) 3.2695 (1.05) 3.2695 (1.05) 0.0000 (1.0) 3.2695 (1.05) 0.0000 (1.0) 0;0 0.3059 (0.95) 1 1
test_checkout_copy-checkout[2.45.0] 3.4805 (1.12) 3.4805 (1.12) 3.4805 (1.12) 0.0000 (1.0) 3.4805 (1.12) 0.0000 (1.0) 0;0 0.2873 (0.90) 1 1
test_checkout_copy-checkout[main] 3.5174 (1.13) 3.5174 (1.13) 3.5174 (1.13) 0.0000 (1.0) 3.5174 (1.13) 0.0000 (1.0) 0;0 0.2843 (0.89) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_checkout_copy-checkout-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_copy-checkout-noop[2.11.0] 493.6121 (1.0) 493.6121 (1.0) 493.6121 (1.0) 0.0000 (1.0) 493.6121 (1.0) 0.0000 (1.0) 0;0 2.0259 (1.0) 1 1
test_checkout_copy-checkout-noop[2.18.1] 559.3813 (1.13) 559.3813 (1.13) 559.3813 (1.13) 0.0000 (1.0) 559.3813 (1.13) 0.0000 (1.0) 0;0 1.7877 (0.88) 1 1
test_checkout_copy-checkout-noop[2.39.0] 849.9952 (1.72) 849.9952 (1.72) 849.9952 (1.72) 0.0000 (1.0) 849.9952 (1.72) 0.0000 (1.0) 0;0 1.1765 (0.58) 1 1
test_checkout_copy-checkout-noop[2.40.0] 574.3197 (1.16) 574.3197 (1.16) 574.3197 (1.16) 0.0000 (1.0) 574.3197 (1.16) 0.0000 (1.0) 0;0 1.7412 (0.86) 1 1
test_checkout_copy-checkout-noop[2.41.1] 590.4164 (1.20) 590.4164 (1.20) 590.4164 (1.20) 0.0000 (1.0) 590.4164 (1.20) 0.0000 (1.0) 0;0 1.6937 (0.84) 1 1
test_checkout_copy-checkout-noop[2.45.0] 968.0251 (1.96) 968.0251 (1.96) 968.0251 (1.96) 0.0000 (1.0) 968.0251 (1.96) 0.0000 (1.0) 0;0 1.0330 (0.51) 1 1
test_checkout_copy-checkout-noop[main] 691.1259 (1.40) 691.1259 (1.40) 691.1259 (1.40) 0.0000 (1.0) 691.1259 (1.40) 0.0000 (1.0) 0;0 1.4469 (0.71) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_checkout_copy-checkout-update': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_copy-checkout-update[2.11.0] 1.2948 (2.03) 1.2948 (2.03) 1.2948 (2.03) 0.0000 (1.0) 1.2948 (2.03) 0.0000 (1.0) 0;0 0.7723 (0.49) 1 1
test_checkout_copy-checkout-update[2.18.1] 0.6379 (1.0) 0.6379 (1.0) 0.6379 (1.0) 0.0000 (1.0) 0.6379 (1.0) 0.0000 (1.0) 0;0 1.5678 (1.0) 1 1
test_checkout_copy-checkout-update[2.39.0] 0.6510 (1.02) 0.6510 (1.02) 0.6510 (1.02) 0.0000 (1.0) 0.6510 (1.02) 0.0000 (1.0) 0;0 1.5360 (0.98) 1 1
test_checkout_copy-checkout-update[2.40.0] 0.6506 (1.02) 0.6506 (1.02) 0.6506 (1.02) 0.0000 (1.0) 0.6506 (1.02) 0.0000 (1.0) 0;0 1.5370 (0.98) 1 1
test_checkout_copy-checkout-update[2.41.1] 0.6569 (1.03) 0.6569 (1.03) 0.6569 (1.03) 0.0000 (1.0) 0.6569 (1.03) 0.0000 (1.0) 0;0 1.5223 (0.97) 1 1
test_checkout_copy-checkout-update[2.45.0] 0.6862 (1.08) 0.6862 (1.08) 0.6862 (1.08) 0.0000 (1.0) 0.6862 (1.08) 0.0000 (1.0) 0;0 1.4574 (0.93) 1 1
test_checkout_copy-checkout-update[main] 0.7505 (1.18) 0.7505 (1.18) 0.7505 (1.18) 0.0000 (1.0) 0.7505 (1.18) 0.0000 (1.0) 0;0 1.3324 (0.85) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
-------------------------------------------------------------------------- benchmark 'test_import-import': 7 tests --------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.11.0] 14.6830 (1.51) 14.6830 (1.51) 14.6830 (1.51) 0.0000 (1.0) 14.6830 (1.51) 0.0000 (1.0) 0;0 0.0681 (0.66) 1 1
test_import-import[2.18.1] 13.6352 (1.40) 13.6352 (1.40) 13.6352 (1.40) 0.0000 (1.0) 13.6352 (1.40) 0.0000 (1.0) 0;0 0.0733 (0.71) 1 1
test_import-import[2.39.0] 11.7053 (1.20) 11.7053 (1.20) 11.7053 (1.20) 0.0000 (1.0) 11.7053 (1.20) 0.0000 (1.0) 0;0 0.0854 (0.83) 1 1
test_import-import[2.40.0] 12.1752 (1.25) 12.1752 (1.25) 12.1752 (1.25) 0.0000 (1.0) 12.1752 (1.25) 0.0000 (1.0) 0;0 0.0821 (0.80) 1 1
test_import-import[2.41.1] 12.0147 (1.23) 12.0147 (1.23) 12.0147 (1.23) 0.0000 (1.0) 12.0147 (1.23) 0.0000 (1.0) 0;0 0.0832 (0.81) 1 1
test_import-import[2.45.0] 9.7332 (1.0) 9.7332 (1.0) 9.7332 (1.0) 0.0000 (1.0) 9.7332 (1.0) 0.0000 (1.0) 0;0 0.1027 (1.0) 1 1
test_import-import[main] 10.5937 (1.09) 10.5937 (1.09) 10.5937 (1.09) 0.0000 (1.0) 10.5937 (1.09) 0.0000 (1.0) 0;0 0.0944 (0.92) 1 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_partial_remove-add': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-add[2.11.0] 3.0168 (1.87) 3.0168 (1.87) 3.0168 (1.87) 0.0000 (1.0) 3.0168 (1.87) 0.0000 (1.0) 0;0 0.3315 (0.54) 1 1
test_partial_remove-add[2.18.1] 2.5297 (1.57) 2.5297 (1.57) 2.5297 (1.57) 0.0000 (1.0) 2.5297 (1.57) 0.0000 (1.0) 0;0 0.3953 (0.64) 1 1
test_partial_remove-add[2.39.0] 3.2830 (2.03) 3.2830 (2.03) 3.2830 (2.03) 0.0000 (1.0) 3.2830 (2.03) 0.0000 (1.0) 0;0 0.3046 (0.49) 1 1
test_partial_remove-add[2.40.0] 1.9488 (1.21) 1.9488 (1.21) 1.9488 (1.21) 0.0000 (1.0) 1.9488 (1.21) 0.0000 (1.0) 0;0 0.5131 (0.83) 1 1
test_partial_remove-add[2.41.1] 1.8763 (1.16) 1.8763 (1.16) 1.8763 (1.16) 0.0000 (1.0) 1.8763 (1.16) 0.0000 (1.0) 0;0 0.5330 (0.86) 1 1
test_partial_remove-add[2.45.0] 3.3268 (2.06) 3.3268 (2.06) 3.3268 (2.06) 0.0000 (1.0) 3.3268 (2.06) 0.0000 (1.0) 0;0 0.3006 (0.49) 1 1
test_partial_remove-add[main] 1.6144 (1.0) 1.6144 (1.0) 1.6144 (1.0) 0.0000 (1.0) 1.6144 (1.0) 0.0000 (1.0) 0;0 0.6194 (1.0) 1 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_partial_remove-gc': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-gc[2.11.0] 430.4494 (1.0) 430.4494 (1.0) 430.4494 (1.0) 0.0000 (1.0) 430.4494 (1.0) 0.0000 (1.0) 0;0 2.3232 (1.0) 1 1
test_partial_remove-gc[2.18.1] 460.6572 (1.07) 460.6572 (1.07) 460.6572 (1.07) 0.0000 (1.0) 460.6572 (1.07) 0.0000 (1.0) 0;0 2.1708 (0.93) 1 1
test_partial_remove-gc[2.39.0] 484.4292 (1.13) 484.4292 (1.13) 484.4292 (1.13) 0.0000 (1.0) 484.4292 (1.13) 0.0000 (1.0) 0;0 2.0643 (0.89) 1 1
test_partial_remove-gc[2.40.0] 490.2897 (1.14) 490.2897 (1.14) 490.2897 (1.14) 0.0000 (1.0) 490.2897 (1.14) 0.0000 (1.0) 0;0 2.0396 (0.88) 1 1
test_partial_remove-gc[2.41.1] 489.4608 (1.14) 489.4608 (1.14) 489.4608 (1.14) 0.0000 (1.0) 489.4608 (1.14) 0.0000 (1.0) 0;0 2.0431 (0.88) 1 1
test_partial_remove-gc[2.45.0] 475.5581 (1.10) 475.5581 (1.10) 475.5581 (1.10) 0.0000 (1.0) 475.5581 (1.10) 0.0000 (1.0) 0;0 2.1028 (0.91) 1 1
test_partial_remove-gc[main] 555.4432 (1.29) 555.4432 (1.29) 555.4432 (1.29) 0.0000 (1.0) 555.4432 (1.29) 0.0000 (1.0) 0;0 1.8004 (0.77) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_partial_remove-gc-cloud': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-gc-cloud[2.11.0] 467.5526 (1.0) 467.5526 (1.0) 467.5526 (1.0) 0.0000 (1.0) 467.5526 (1.0) 0.0000 (1.0) 0;0 2.1388 (1.0) 1 1
test_partial_remove-gc-cloud[2.18.1] 500.8842 (1.07) 500.8842 (1.07) 500.8842 (1.07) 0.0000 (1.0) 500.8842 (1.07) 0.0000 (1.0) 0;0 1.9965 (0.93) 1 1
test_partial_remove-gc-cloud[2.39.0] 514.1390 (1.10) 514.1390 (1.10) 514.1390 (1.10) 0.0000 (1.0) 514.1390 (1.10) 0.0000 (1.0) 0;0 1.9450 (0.91) 1 1
test_partial_remove-gc-cloud[2.40.0] 542.7431 (1.16) 542.7431 (1.16) 542.7431 (1.16) 0.0000 (1.0) 542.7431 (1.16) 0.0000 (1.0) 0;0 1.8425 (0.86) 1 1
test_partial_remove-gc-cloud[2.41.1] 525.0107 (1.12) 525.0107 (1.12) 525.0107 (1.12) 0.0000 (1.0) 525.0107 (1.12) 0.0000 (1.0) 0;0 1.9047 (0.89) 1 1
test_partial_remove-gc-cloud[2.45.0] 505.8885 (1.08) 505.8885 (1.08) 505.8885 (1.08) 0.0000 (1.0) 505.8885 (1.08) 0.0000 (1.0) 0;0 1.9767 (0.92) 1 1
test_partial_remove-gc-cloud[main] 609.5283 (1.30) 609.5283 (1.30) 609.5283 (1.30) 0.0000 (1.0) 609.5283 (1.30) 0.0000 (1.0) 0;0 1.6406 (0.77) 1 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_partial_remove-push': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-push[2.11.0] 436.1086 (1.0) 436.1086 (1.0) 436.1086 (1.0) 0.0000 (1.0) 436.1086 (1.0) 0.0000 (1.0) 0;0 2.2930 (1.0) 1 1
test_partial_remove-push[2.18.1] 444.9656 (1.02) 444.9656 (1.02) 444.9656 (1.02) 0.0000 (1.0) 444.9656 (1.02) 0.0000 (1.0) 0;0 2.2474 (0.98) 1 1
test_partial_remove-push[2.39.0] 495.8387 (1.14) 495.8387 (1.14) 495.8387 (1.14) 0.0000 (1.0) 495.8387 (1.14) 0.0000 (1.0) 0;0 2.0168 (0.88) 1 1
test_partial_remove-push[2.40.0] 475.7971 (1.09) 475.7971 (1.09) 475.7971 (1.09) 0.0000 (1.0) 475.7971 (1.09) 0.0000 (1.0) 0;0 2.1017 (0.92) 1 1
test_partial_remove-push[2.41.1] 473.2685 (1.09) 473.2685 (1.09) 473.2685 (1.09) 0.0000 (1.0) 473.2685 (1.09) 0.0000 (1.0) 0;0 2.1130 (0.92) 1 1
test_partial_remove-push[2.45.0] 501.7115 (1.15) 501.7115 (1.15) 501.7115 (1.15) 0.0000 (1.0) 501.7115 (1.15) 0.0000 (1.0) 0;0 1.9932 (0.87) 1 1
test_partial_remove-push[main] 595.0350 (1.36) 595.0350 (1.36) 595.0350 (1.36) 0.0000 (1.0) 595.0350 (1.36) 0.0000 (1.0) 0;0 1.6806 (0.73) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
------------------------------------------------------------------------ benchmark 'test_partial_add-add-partial-add': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-add-partial-add[2.11.0] 7.3879 (1.94) 7.3879 (1.94) 7.3879 (1.94) 0.0000 (1.0) 7.3879 (1.94) 0.0000 (1.0) 0;0 0.1354 (0.51) 1 1
test_partial_add-add-partial-add[2.18.1] 6.5590 (1.73) 6.5590 (1.73) 6.5590 (1.73) 0.0000 (1.0) 6.5590 (1.73) 0.0000 (1.0) 0;0 0.1525 (0.58) 1 1
test_partial_add-add-partial-add[2.39.0] 5.4128 (1.42) 5.4128 (1.42) 5.4128 (1.42) 0.0000 (1.0) 5.4128 (1.42) 0.0000 (1.0) 0;0 0.1847 (0.70) 1 1
test_partial_add-add-partial-add[2.40.0] 5.5331 (1.46) 5.5331 (1.46) 5.5331 (1.46) 0.0000 (1.0) 5.5331 (1.46) 0.0000 (1.0) 0;0 0.1807 (0.69) 1 1
test_partial_add-add-partial-add[2.41.1] 5.5265 (1.45) 5.5265 (1.45) 5.5265 (1.45) 0.0000 (1.0) 5.5265 (1.45) 0.0000 (1.0) 0;0 0.1809 (0.69) 1 1
test_partial_add-add-partial-add[2.45.0] 3.9449 (1.04) 3.9449 (1.04) 3.9449 (1.04) 0.0000 (1.0) 3.9449 (1.04) 0.0000 (1.0) 0;0 0.2535 (0.96) 1 1
test_partial_add-add-partial-add[main] 3.8005 (1.0) 3.8005 (1.0) 3.8005 (1.0) 0.0000 (1.0) 3.8005 (1.0) 0.0000 (1.0) 0;0 0.2631 (1.0) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_partial_add-gc-cloud-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-gc-cloud-noop[2.11.0] 552.2735 (1.0) 552.2735 (1.0) 552.2735 (1.0) 0.0000 (1.0) 552.2735 (1.0) 0.0000 (1.0) 0;0 1.8107 (1.0) 1 1
test_partial_add-gc-cloud-noop[2.18.1] 611.3034 (1.11) 611.3034 (1.11) 611.3034 (1.11) 0.0000 (1.0) 611.3034 (1.11) 0.0000 (1.0) 0;0 1.6358 (0.90) 1 1
test_partial_add-gc-cloud-noop[2.39.0] 645.7618 (1.17) 645.7618 (1.17) 645.7618 (1.17) 0.0000 (1.0) 645.7618 (1.17) 0.0000 (1.0) 0;0 1.5486 (0.86) 1 1
test_partial_add-gc-cloud-noop[2.40.0] 653.6379 (1.18) 653.6379 (1.18) 653.6379 (1.18) 0.0000 (1.0) 653.6379 (1.18) 0.0000 (1.0) 0;0 1.5299 (0.84) 1 1
test_partial_add-gc-cloud-noop[2.41.1] 647.2986 (1.17) 647.2986 (1.17) 647.2986 (1.17) 0.0000 (1.0) 647.2986 (1.17) 0.0000 (1.0) 0;0 1.5449 (0.85) 1 1
test_partial_add-gc-cloud-noop[2.45.0] 597.2629 (1.08) 597.2629 (1.08) 597.2629 (1.08) 0.0000 (1.0) 597.2629 (1.08) 0.0000 (1.0) 0;0 1.6743 (0.92) 1 1
test_partial_add-gc-cloud-noop[main] 735.2905 (1.33) 735.2905 (1.33) 735.2905 (1.33) 0.0000 (1.0) 735.2905 (1.33) 0.0000 (1.0) 0;0 1.3600 (0.75) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------- benchmark 'test_partial_add-gc-noop': 7 tests ----------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-gc-noop[2.11.0] 495.1942 (1.0) 495.1942 (1.0) 495.1942 (1.0) 0.0000 (1.0) 495.1942 (1.0) 0.0000 (1.0) 0;0 2.0194 (1.0) 1 1
test_partial_add-gc-noop[2.18.1] 532.2792 (1.07) 532.2792 (1.07) 532.2792 (1.07) 0.0000 (1.0) 532.2792 (1.07) 0.0000 (1.0) 0;0 1.8787 (0.93) 1 1
test_partial_add-gc-noop[2.39.0] 575.3058 (1.16) 575.3058 (1.16) 575.3058 (1.16) 0.0000 (1.0) 575.3058 (1.16) 0.0000 (1.0) 0;0 1.7382 (0.86) 1 1
test_partial_add-gc-noop[2.40.0] 585.2291 (1.18) 585.2291 (1.18) 585.2291 (1.18) 0.0000 (1.0) 585.2291 (1.18) 0.0000 (1.0) 0;0 1.7087 (0.85) 1 1
test_partial_add-gc-noop[2.41.1] 573.2859 (1.16) 573.2859 (1.16) 573.2859 (1.16) 0.0000 (1.0) 573.2859 (1.16) 0.0000 (1.0) 0;0 1.7443 (0.86) 1 1
test_partial_add-gc-noop[2.45.0] 573.6146 (1.16) 573.6146 (1.16) 573.6146 (1.16) 0.0000 (1.0) 573.6146 (1.16) 0.0000 (1.0) 0;0 1.7433 (0.86) 1 1
test_partial_add-gc-noop[main] 654.9821 (1.32) 654.9821 (1.32) 654.9821 (1.32) 0.0000 (1.0) 654.9821 (1.32) 0.0000 (1.0) 0;0 1.5268 (0.76) 1 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------ benchmark 'test_partial_add-push-partial-add': 7 tests ------------------------------------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-push-partial-add[2.11.0] 2.6282 (4.47) 2.6282 (4.47) 2.6282 (4.47) 0.0000 (1.0) 2.6282 (4.47) 0.0000 (1.0) 0;0 0.3805 (0.22) 1 1
test_partial_add-push-partial-add[2.18.1] 2.6674 (4.53) 2.6674 (4.53) 2.6674 (4.53) 0.0000 (1.0) 2.6674 (4.53) 0.0000 (1.0) 0;0 0.3749 (0.22) 1 1
test_partial_add-push-partial-add[2.39.0] 3.2803 (5.57) 3.2803 (5.57) 3.2803 (5.57) 0.0000 (1.0) 3.2803 (5.57) 0.0000 (1.0) 0;0 0.3048 (0.18) 1 1
test_partial_add-push-partial-add[2.40.0] 3.2726 (5.56) 3.2726 (5.56) 3.2726 (5.56) 0.0000 (1.0) 3.2726 (5.56) 0.0000 (1.0) 0;0 0.3056 (0.18) 1 1
test_partial_add-push-partial-add[2.41.1] 3.2462 (5.52) 3.2462 (5.52) 3.2462 (5.52) 0.0000 (1.0) 3.2462 (5.52) 0.0000 (1.0) 0;0 0.3080 (0.18) 1 1
test_partial_add-push-partial-add[2.45.0] 0.5886 (1.0) 0.5886 (1.0) 0.5886 (1.0) 0.0000 (1.0) 0.5886 (1.0) 0.0000 (1.0) 0;0 1.6989 (1.0) 1 1
test_partial_add-push-partial-add[main] 2.4419 (4.15) 2.4419 (4.15) 2.4419 (4.15) 0.0000 (1.0) 2.4419 (4.15) 0.0000 (1.0) 0;0 0.4095 (0.24) 1 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
We may also need to just reduce the # of revs we test for S3, aws token timeout is currently 4 hours, not sure it is worth extending it