search

iterative / dvc-bench

Benchmarks for DVC
http://bench.dvc.org/
Apache License 2.0
20 stars 10 forks source link

pin pytest #514

Closed skshetry closed 7 months ago

skshetry commented 7 months ago

Newer release of pytest 8.1.0 is breaking flaky plugin. Pinning the pytest version temporarily.

Run echo "tests=$(./scripts/ci/list_tests.sh)" >> $GITHUB_OUTPUT
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.13/x64/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 195, in console_main
    code = main()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 153, in main
    config = _prepareconfig(args, plugins)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 335, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 138, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
    config = yield
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1141, in pytest_cmdline_parse
    self.parse(args)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1490, in parse
    self._preparse(args, addopts=addopts)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1377, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 414, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 178, in exec_module
    exec(co, module.__dict__)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/flaky/flaky_pytest_plugin.py", line 5, in <module>
    from _pytest.runner import call_runtest_hook  # pylint:disable=import-error
ImportError: cannot import name 'call_runtest_hook' from '_pytest.runner' (/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/_pytest/runner.py)
github-actions[bot] commented 7 months ago 

Mon Mar  4 04:00:56 UTC 2024
dataset: small
project: example-get-started

------------------------------------------------------------------------ benchmark 'test_add-add': 3 tests ------------------------------------------------------------------------
Name (time in s)            Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_add-add[2.58.2]     3.5147 (1.02)     3.5147 (1.02)     3.5147 (1.02)     0.0000 (1.0)      3.5147 (1.02)     0.0000 (1.0)           0;0  0.2845 (0.98)          1           1
test_add-add[3.10.0]     3.6151 (1.04)     3.6151 (1.04)     3.6151 (1.04)     0.0000 (1.0)      3.6151 (1.04)     0.0000 (1.0)           0;0  0.2766 (0.96)          1           1
test_add-add[main]       3.4608 (1.0)      3.4608 (1.0)      3.4608 (1.0)      0.0000 (1.0)      3.4608 (1.0)      0.0000 (1.0)           0;0  0.2890 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_checkout_copy-checkout': 3 tests ------------------------------------------------------------------------
Name (time in s)                           Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_copy-checkout[2.58.2]     1.8643 (1.21)     1.8643 (1.21)     1.8643 (1.21)     0.0000 (1.0)      1.8643 (1.21)     0.0000 (1.0)           0;0  0.5364 (0.83)          1           1
test_checkout_copy-checkout[3.10.0]     3.8188 (2.48)     3.8188 (2.48)     3.8188 (2.48)     0.0000 (1.0)      3.8188 (2.48)     0.0000 (1.0)           0;0  0.2619 (0.40)          1           1
test_checkout_copy-checkout[main]       1.5382 (1.0)      1.5382 (1.0)      1.5382 (1.0)      0.0000 (1.0)      1.5382 (1.0)      0.0000 (1.0)           0;0  0.6501 (1.0)           1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_checkout_copy-checkout-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                 Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_copy-checkout-noop[2.58.2]     551.5566 (1.00)     551.5566 (1.00)     551.5566 (1.00)     0.0000 (1.0)      551.5566 (1.00)     0.0000 (1.0)           0;0  1.8131 (1.00)          1           1
test_checkout_copy-checkout-noop[3.10.0]     567.2736 (1.03)     567.2736 (1.03)     567.2736 (1.03)     0.0000 (1.0)      567.2736 (1.03)     0.0000 (1.0)           0;0  1.7628 (0.97)          1           1
test_checkout_copy-checkout-noop[main]       551.0768 (1.0)      551.0768 (1.0)      551.0768 (1.0)      0.0000 (1.0)      551.0768 (1.0)      0.0000 (1.0)           0;0  1.8146 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_checkout_copy-checkout-update': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                   Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_copy-checkout-update[2.58.2]     685.2182 (1.10)     685.2182 (1.10)     685.2182 (1.10)     0.0000 (1.0)      685.2182 (1.10)     0.0000 (1.0)           0;0  1.4594 (0.91)          1           1
test_checkout_copy-checkout-update[3.10.0]     625.3044 (1.0)      625.3044 (1.0)      625.3044 (1.0)      0.0000 (1.0)      625.3044 (1.0)      0.0000 (1.0)           0;0  1.5992 (1.0)           1           1
test_checkout_copy-checkout-update[main]       644.5419 (1.03)     644.5419 (1.03)     644.5419 (1.03)     0.0000 (1.0)      644.5419 (1.03)     0.0000 (1.0)           0;0  1.5515 (0.97)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_checkout_hardlink-checkout': 3 tests ------------------------------------------------------------------------
Name (time in s)                               Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_hardlink-checkout[2.58.2]     1.1931 (1.16)     1.1931 (1.16)     1.1931 (1.16)     0.0000 (1.0)      1.1931 (1.16)     0.0000 (1.0)           0;0  0.8381 (0.86)          1           1
test_checkout_hardlink-checkout[3.10.0]     1.0652 (1.04)     1.0652 (1.04)     1.0652 (1.04)     0.0000 (1.0)      1.0652 (1.04)     0.0000 (1.0)           0;0  0.9388 (0.96)          1           1
test_checkout_hardlink-checkout[main]       1.0262 (1.0)      1.0262 (1.0)      1.0262 (1.0)      0.0000 (1.0)      1.0262 (1.0)      0.0000 (1.0)           0;0  0.9744 (1.0)           1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_checkout_hardlink-checkout-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                     Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_hardlink-checkout-noop[2.58.2]     545.5825 (1.00)     545.5825 (1.00)     545.5825 (1.00)     0.0000 (1.0)      545.5825 (1.00)     0.0000 (1.0)           0;0  1.8329 (1.00)          1           1
test_checkout_hardlink-checkout-noop[3.10.0]     574.2482 (1.05)     574.2482 (1.05)     574.2482 (1.05)     0.0000 (1.0)      574.2482 (1.05)     0.0000 (1.0)           0;0  1.7414 (0.95)          1           1
test_checkout_hardlink-checkout-noop[main]       544.8794 (1.0)      544.8794 (1.0)      544.8794 (1.0)      0.0000 (1.0)      544.8794 (1.0)      0.0000 (1.0)           0;0  1.8353 (1.0)           1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_checkout_hardlink-checkout-update': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                       Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_hardlink-checkout-update[2.58.2]     687.9312 (1.13)     687.9312 (1.13)     687.9312 (1.13)     0.0000 (1.0)      687.9312 (1.13)     0.0000 (1.0)           0;0  1.4536 (0.89)          1           1
test_checkout_hardlink-checkout-update[3.10.0]     624.1597 (1.02)     624.1597 (1.02)     624.1597 (1.02)     0.0000 (1.0)      624.1597 (1.02)     0.0000 (1.0)           0;0  1.6022 (0.98)          1           1
test_checkout_hardlink-checkout-update[main]       610.6018 (1.0)      610.6018 (1.0)      610.6018 (1.0)      0.0000 (1.0)      610.6018 (1.0)      0.0000 (1.0)           0;0  1.6377 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_checkout_symlink-checkout': 3 tests ------------------------------------------------------------------------
Name (time in s)                              Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_symlink-checkout[2.58.2]     1.3518 (1.09)     1.3518 (1.09)     1.3518 (1.09)     0.0000 (1.0)      1.3518 (1.09)     0.0000 (1.0)           0;0  0.7398 (0.91)          1           1
test_checkout_symlink-checkout[3.10.0]     1.2513 (1.01)     1.2513 (1.01)     1.2513 (1.01)     0.0000 (1.0)      1.2513 (1.01)     0.0000 (1.0)           0;0  0.7992 (0.99)          1           1
test_checkout_symlink-checkout[main]       1.2357 (1.0)      1.2357 (1.0)      1.2357 (1.0)      0.0000 (1.0)      1.2357 (1.0)      0.0000 (1.0)           0;0  0.8092 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_checkout_symlink-checkout-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                    Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_symlink-checkout-noop[2.58.2]     609.6523 (1.0)      609.6523 (1.0)      609.6523 (1.0)      0.0000 (1.0)      609.6523 (1.0)      0.0000 (1.0)           0;0  1.6403 (1.0)           1           1
test_checkout_symlink-checkout-noop[3.10.0]     651.6641 (1.07)     651.6641 (1.07)     651.6641 (1.07)     0.0000 (1.0)      651.6641 (1.07)     0.0000 (1.0)           0;0  1.5345 (0.94)          1           1
test_checkout_symlink-checkout-noop[main]       639.7648 (1.05)     639.7648 (1.05)     639.7648 (1.05)     0.0000 (1.0)      639.7648 (1.05)     0.0000 (1.0)           0;0  1.5631 (0.95)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_checkout_symlink-checkout-update': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                      Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_checkout_symlink-checkout-update[2.58.2]     806.6538 (1.11)     806.6538 (1.11)     806.6538 (1.11)     0.0000 (1.0)      806.6538 (1.11)     0.0000 (1.0)           0;0  1.2397 (0.90)          1           1
test_checkout_symlink-checkout-update[3.10.0]     728.7542 (1.0)      728.7542 (1.0)      728.7542 (1.0)      0.0000 (1.0)      728.7542 (1.0)      0.0000 (1.0)           0;0  1.3722 (1.0)           1           1
test_checkout_symlink-checkout-update[main]       749.7200 (1.03)     749.7200 (1.03)     749.7200 (1.03)     0.0000 (1.0)      749.7200 (1.03)     0.0000 (1.0)           0;0  1.3338 (0.97)          1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

---------------------------------------------------------------------------- benchmark 'test_data_status-data-changed': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                              Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-changed[2.58.2]     582.9678 (1.05)     582.9678 (1.05)     582.9678 (1.05)     0.0000 (1.0)      582.9678 (1.05)     0.0000 (1.0)           0;0  1.7154 (0.96)          1           1
test_data_status-data-changed[3.10.0]     604.6158 (1.09)     604.6158 (1.09)     604.6158 (1.09)     0.0000 (1.0)      604.6158 (1.09)     0.0000 (1.0)           0;0  1.6539 (0.92)          1           1
test_data_status-data-changed[main]       557.1454 (1.0)      557.1454 (1.0)      557.1454 (1.0)      0.0000 (1.0)      557.1454 (1.0)      0.0000 (1.0)           0;0  1.7949 (1.0)           1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_data_status-data-changed-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                   Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-changed-noop[2.58.2]     588.6495 (1.04)     588.6495 (1.04)     588.6495 (1.04)     0.0000 (1.0)      588.6495 (1.04)     0.0000 (1.0)           0;0  1.6988 (0.96)          1           1
test_data_status-data-changed-noop[3.10.0]     592.6565 (1.05)     592.6565 (1.05)     592.6565 (1.05)     0.0000 (1.0)      592.6565 (1.05)     0.0000 (1.0)           0;0  1.6873 (0.95)          1           1
test_data_status-data-changed-noop[main]       565.3804 (1.0)      565.3804 (1.0)      565.3804 (1.0)      0.0000 (1.0)      565.3804 (1.0)      0.0000 (1.0)           0;0  1.7687 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_data_status-data-new': 3 tests ------------------------------------------------------------------------
Name (time in s)                         Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-new[2.58.2]     1.2492 (1.02)     1.2492 (1.02)     1.2492 (1.02)     0.0000 (1.0)      1.2492 (1.02)     0.0000 (1.0)           0;0  0.8005 (0.98)          1           1
test_data_status-data-new[3.10.0]     1.2492 (1.02)     1.2492 (1.02)     1.2492 (1.02)     0.0000 (1.0)      1.2492 (1.02)     0.0000 (1.0)           0;0  0.8005 (0.98)          1           1
test_data_status-data-new[main]       1.2285 (1.0)      1.2285 (1.0)      1.2285 (1.0)      0.0000 (1.0)      1.2285 (1.0)      0.0000 (1.0)           0;0  0.8140 (1.0)           1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_data_status-data-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                           Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status-data-noop[2.58.2]     587.8928 (1.04)     587.8928 (1.04)     587.8928 (1.04)     0.0000 (1.0)      587.8928 (1.04)     0.0000 (1.0)           0;0  1.7010 (0.96)          1           1
test_data_status-data-noop[3.10.0]     597.2293 (1.06)     597.2293 (1.06)     597.2293 (1.06)     0.0000 (1.0)      597.2293 (1.06)     0.0000 (1.0)           0;0  1.6744 (0.95)          1           1
test_data_status-data-noop[main]       565.7298 (1.0)      565.7298 (1.0)      565.7298 (1.0)      0.0000 (1.0)      565.7298 (1.0)      0.0000 (1.0)           0;0  1.7676 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-changed': 3 tests ------------------------------------------------------------------------
Name (time in s)                                       Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-changed[2.58.2]     1.0300 (1.00)     1.0300 (1.00)     1.0300 (1.00)     0.0000 (1.0)      1.0300 (1.00)     0.0000 (1.0)           0;0  0.9709 (1.00)          1           1
test_data_status_all_flags-data-changed[3.10.0]     1.0266 (1.0)      1.0266 (1.0)      1.0266 (1.0)      0.0000 (1.0)      1.0266 (1.0)      0.0000 (1.0)           0;0  0.9741 (1.0)           1           1
test_data_status_all_flags-data-changed[main]       1.0997 (1.07)     1.0997 (1.07)     1.0997 (1.07)     0.0000 (1.0)      1.0997 (1.07)     0.0000 (1.0)           0;0  0.9094 (0.93)          1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-changed-noop': 3 tests ------------------------------------------------------------------------
Name (time in s)                                            Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-changed-noop[2.58.2]     1.0346 (1.02)     1.0346 (1.02)     1.0346 (1.02)     0.0000 (1.0)      1.0346 (1.02)     0.0000 (1.0)           0;0  0.9665 (0.98)          1           1
test_data_status_all_flags-data-changed-noop[3.10.0]     1.0231 (1.01)     1.0231 (1.01)     1.0231 (1.01)     0.0000 (1.0)      1.0231 (1.01)     0.0000 (1.0)           0;0  0.9774 (0.99)          1           1
test_data_status_all_flags-data-changed-noop[main]       1.0160 (1.0)      1.0160 (1.0)      1.0160 (1.0)      0.0000 (1.0)      1.0160 (1.0)      0.0000 (1.0)           0;0  0.9843 (1.0)           1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_data_status_all_flags-data-new': 3 tests ------------------------------------------------------------------------
Name (time in s)                                   Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-new[2.58.2]     1.6843 (1.00)     1.6843 (1.00)     1.6843 (1.00)     0.0000 (1.0)      1.6843 (1.00)     0.0000 (1.0)           0;0  0.5937 (1.00)          1           1
test_data_status_all_flags-data-new[3.10.0]     1.6798 (1.00)     1.6798 (1.00)     1.6798 (1.00)     0.0000 (1.0)      1.6798 (1.00)     0.0000 (1.0)           0;0  0.5953 (1.00)          1           1
test_data_status_all_flags-data-new[main]       1.6775 (1.0)      1.6775 (1.0)      1.6775 (1.0)      0.0000 (1.0)      1.6775 (1.0)      0.0000 (1.0)           0;0  0.5961 (1.0)           1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_data_status_all_flags-data-noop': 3 tests --------------------------------------------------------------------------------
Name (time in ms)                                       Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_data_status_all_flags-data-noop[2.58.2]       998.8613 (1.0)        998.8613 (1.0)        998.8613 (1.0)      0.0000 (1.0)        998.8613 (1.0)      0.0000 (1.0)           0;0  1.0011 (1.0)           1           1
test_data_status_all_flags-data-noop[3.10.0]     1,013.4184 (1.01)     1,013.4184 (1.01)     1,013.4184 (1.01)     0.0000 (1.0)      1,013.4184 (1.01)     0.0000 (1.0)           0;0  0.9868 (0.99)          1           1
test_data_status_all_flags-data-noop[main]         999.6327 (1.00)       999.6327 (1.00)       999.6327 (1.00)     0.0000 (1.0)        999.6327 (1.00)     0.0000 (1.0)           0;0  1.0004 (1.00)          1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_diff-diff': 3 tests ------------------------------------------------------------------------
Name (time in s)              Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff[2.58.2]     1.2074 (1.0)      1.2074 (1.0)      1.2074 (1.0)      0.0000 (1.0)      1.2074 (1.0)      0.0000 (1.0)           0;0  0.8282 (1.0)           1           1
test_diff-diff[3.10.0]     1.2466 (1.03)     1.2466 (1.03)     1.2466 (1.03)     0.0000 (1.0)      1.2466 (1.03)     0.0000 (1.0)           0;0  0.8022 (0.97)          1           1
test_diff-diff[main]       1.2900 (1.07)     1.2900 (1.07)     1.2900 (1.07)     0.0000 (1.0)      1.2900 (1.07)     0.0000 (1.0)           0;0  0.7752 (0.94)          1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_diff-diff-changed': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                       Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed[2.58.2]     634.8253 (1.00)     634.8253 (1.00)     634.8253 (1.00)     0.0000 (1.0)      634.8253 (1.00)     0.0000 (1.0)           0;0  1.5752 (1.00)          1           1
test_diff-diff-changed[3.10.0]     671.5001 (1.06)     671.5001 (1.06)     671.5001 (1.06)     0.0000 (1.0)      671.5001 (1.06)     0.0000 (1.0)           0;0  1.4892 (0.94)          1           1
test_diff-diff-changed[main]       633.3627 (1.0)      633.3627 (1.0)      633.3627 (1.0)      0.0000 (1.0)      633.3627 (1.0)      0.0000 (1.0)           0;0  1.5789 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_diff-diff-changed-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                            Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-changed-noop[2.58.2]     605.7728 (1.0)      605.7728 (1.0)      605.7728 (1.0)      0.0000 (1.0)      605.7728 (1.0)      0.0000 (1.0)           0;0  1.6508 (1.0)           1           1
test_diff-diff-changed-noop[3.10.0]     634.2167 (1.05)     634.2167 (1.05)     634.2167 (1.05)     0.0000 (1.0)      634.2167 (1.05)     0.0000 (1.0)           0;0  1.5767 (0.96)          1           1
test_diff-diff-changed-noop[main]       623.4004 (1.03)     623.4004 (1.03)     623.4004 (1.03)     0.0000 (1.0)      623.4004 (1.03)     0.0000 (1.0)           0;0  1.6041 (0.97)          1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_diff-diff-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                    Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_diff-diff-noop[2.58.2]     568.1910 (1.0)      568.1910 (1.0)      568.1910 (1.0)      0.0000 (1.0)      568.1910 (1.0)      0.0000 (1.0)           0;0  1.7600 (1.0)           1           1
test_diff-diff-noop[3.10.0]     584.0614 (1.03)     584.0614 (1.03)     584.0614 (1.03)     0.0000 (1.0)      584.0614 (1.03)     0.0000 (1.0)           0;0  1.7121 (0.97)          1           1
test_diff-diff-noop[main]       618.0935 (1.09)     618.0935 (1.09)     618.0935 (1.09)     0.0000 (1.0)      618.0935 (1.09)     0.0000 (1.0)           0;0  1.6179 (0.92)          1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

-------------------------------------------------------------------------------- benchmark 'test_exp_show-exp': 3 tests --------------------------------------------------------------------------------
Name (time in ms)                    Min                   Max                  Mean            StdDev                Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_exp_show-exp[2.58.2]       833.4103 (1.0)        833.4103 (1.0)        833.4103 (1.0)      0.0000 (1.0)        833.4103 (1.0)      0.0000 (1.0)           0;0  1.1999 (1.0)           1           1
test_exp_show-exp[3.10.0]     4,338.6892 (5.21)     4,338.6892 (5.21)     4,338.6892 (5.21)     0.0000 (1.0)      4,338.6892 (5.21)     0.0000 (1.0)           0;0  0.2305 (0.19)          1           1
test_exp_show-exp[main]       3,270.6552 (3.92)     3,270.6552 (3.92)     3,270.6552 (3.92)     0.0000 (1.0)      3,270.6552 (3.92)     0.0000 (1.0)           0;0  0.3057 (0.25)          1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_fetch-fetch': 3 tests ------------------------------------------------------------------------
Name (time in s)                Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fetch-fetch[2.58.2]     3.9406 (2.01)     3.9406 (2.01)     3.9406 (2.01)     0.0000 (1.0)      3.9406 (2.01)     0.0000 (1.0)           0;0  0.2538 (0.50)          1           1
test_fetch-fetch[3.10.0]     4.4007 (2.24)     4.4007 (2.24)     4.4007 (2.24)     0.0000 (1.0)      4.4007 (2.24)     0.0000 (1.0)           0;0  0.2272 (0.45)          1           1
test_fetch-fetch[main]       1.9614 (1.0)      1.9614 (1.0)      1.9614 (1.0)      0.0000 (1.0)      1.9614 (1.0)      0.0000 (1.0)           0;0  0.5098 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

---------------------------------------------------------------------------- benchmark 'test_gc-gc': 3 tests ----------------------------------------------------------------------------
Name (time in ms)           Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_gc-gc[2.58.2]     519.8703 (1.01)     519.8703 (1.01)     519.8703 (1.01)     0.0000 (1.0)      519.8703 (1.01)     0.0000 (1.0)           0;0  1.9236 (0.99)          1           1
test_gc-gc[3.10.0]     529.2081 (1.03)     529.2081 (1.03)     529.2081 (1.03)     0.0000 (1.0)      529.2081 (1.03)     0.0000 (1.0)           0;0  1.8896 (0.97)          1           1
test_gc-gc[main]       513.1452 (1.0)      513.1452 (1.0)      513.1452 (1.0)      0.0000 (1.0)      513.1452 (1.0)      0.0000 (1.0)           0;0  1.9488 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_get-get': 3 tests ------------------------------------------------------------------------
Name (time in s)            Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get-get[2.58.2]     5.9147 (1.12)     5.9147 (1.12)     5.9147 (1.12)     0.0000 (1.0)      5.9147 (1.12)     0.0000 (1.0)           0;0  0.1691 (0.89)          1           1
test_get-get[3.10.0]     7.0470 (1.33)     7.0470 (1.33)     7.0470 (1.33)     0.0000 (1.0)      7.0470 (1.33)     0.0000 (1.0)           0;0  0.1419 (0.75)          1           1
test_get-get[main]       5.2872 (1.0)      5.2872 (1.0)      5.2872 (1.0)      0.0000 (1.0)      5.2872 (1.0)      0.0000 (1.0)           0;0  0.1891 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_get_url-get-url': 3 tests ------------------------------------------------------------------------
Name (time in s)                    Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_get_url-get-url[2.58.2]     3.1222 (3.29)     3.1222 (3.29)     3.1222 (3.29)     0.0000 (1.0)      3.1222 (3.29)     0.0000 (1.0)           0;0  0.3203 (0.30)          1           1
test_get_url-get-url[3.10.0]     3.2514 (3.43)     3.2514 (3.43)     3.2514 (3.43)     0.0000 (1.0)      3.2514 (3.43)     0.0000 (1.0)           0;0  0.3076 (0.29)          1           1
test_get_url-get-url[main]       0.9480 (1.0)      0.9480 (1.0)      0.9480 (1.0)      0.0000 (1.0)      0.9480 (1.0)      0.0000 (1.0)           0;0  1.0548 (1.0)           1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

---------------------------------------------------------------------------- benchmark 'test_help---help': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                 Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_help---help[2.58.2]     132.3414 (1.04)     137.8181 (1.0)      134.8664 (1.02)     1.2349 (1.0)      134.8658 (1.02)     1.7406 (1.18)         33;0  7.4147 (0.98)        100           1
test_help---help[3.10.0]     127.6606 (1.0)      139.0963 (1.01)     132.2406 (1.0)      1.5060 (1.22)     132.1295 (1.0)      1.7192 (1.16)         23;5  7.5620 (1.0)         100           1
test_help---help[main]       131.2173 (1.03)     147.7534 (1.07)     133.9629 (1.01)     1.8215 (1.47)     133.6576 (1.01)     1.4758 (1.0)          14;2  7.4648 (0.99)        100           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

-------------------------------------------------------------------------- benchmark 'test_import-import': 3 tests --------------------------------------------------------------------------
Name (time in s)                   Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import-import[2.58.2]      9.1934 (1.11)      9.1934 (1.11)      9.1934 (1.11)     0.0000 (1.0)       9.1934 (1.11)     0.0000 (1.0)           0;0  0.1088 (0.90)          1           1
test_import-import[3.10.0]     10.3237 (1.24)     10.3237 (1.24)     10.3237 (1.24)     0.0000 (1.0)      10.3237 (1.24)     0.0000 (1.0)           0;0  0.0969 (0.80)          1           1
test_import-import[main]        8.3057 (1.0)       8.3057 (1.0)       8.3057 (1.0)      0.0000 (1.0)       8.3057 (1.0)      0.0000 (1.0)           0;0  0.1204 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_import_url-import-url': 3 tests ------------------------------------------------------------------------
Name (time in s)                          Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_import_url-import-url[2.58.2]     7.3306 (1.45)     7.3306 (1.45)     7.3306 (1.45)     0.0000 (1.0)      7.3306 (1.45)     0.0000 (1.0)           0;0  0.1364 (0.69)          1           1
test_import_url-import-url[3.10.0]     7.4681 (1.48)     7.4681 (1.48)     7.4681 (1.48)     0.0000 (1.0)      7.4681 (1.48)     0.0000 (1.0)           0;0  0.1339 (0.68)          1           1
test_import_url-import-url[main]       5.0468 (1.0)      5.0468 (1.0)      5.0468 (1.0)      0.0000 (1.0)      5.0468 (1.0)      0.0000 (1.0)           0;0  0.1981 (1.0)           1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

---------------------------------------------------------------------------- benchmark 'test_init-init': 3 tests ----------------------------------------------------------------------------
Name (time in ms)               Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_init-init[2.58.2]     370.9044 (1.00)     395.0418 (1.0)      380.6390 (1.0)      4.8128 (1.0)      380.0163 (1.0)      5.9699 (1.00)         31;2  2.6272 (1.0)         100           1
test_init-init[3.10.0]     375.1407 (1.01)     405.3947 (1.03)     385.4248 (1.01)     5.3625 (1.11)     385.1686 (1.01)     6.9805 (1.17)         32;2  2.5945 (0.99)        100           1
test_init-init[main]       370.4913 (1.0)      423.1710 (1.07)     382.6356 (1.01)     6.2606 (1.30)     382.1213 (1.01)     5.9682 (1.0)          17;3  2.6135 (0.99)        100           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

---------------------------------------------------------------------------- benchmark 'test_list-list': 3 tests ----------------------------------------------------------------------------
Name (time in ms)               Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list[2.58.2]     401.5392 (1.00)     401.5392 (1.00)     401.5392 (1.00)     0.0000 (1.0)      401.5392 (1.00)     0.0000 (1.0)           0;0  2.4904 (1.00)          1           1
test_list-list[3.10.0]     423.1191 (1.05)     423.1191 (1.05)     423.1191 (1.05)     0.0000 (1.0)      423.1191 (1.05)     0.0000 (1.0)           0;0  2.3634 (0.95)          1           1
test_list-list[main]       401.0845 (1.0)      401.0845 (1.0)      401.0845 (1.0)      0.0000 (1.0)      401.0845 (1.0)      0.0000 (1.0)           0;0  2.4932 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_list-list-dvc-only': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                        Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-dvc-only[2.58.2]     380.6467 (1.0)      380.6467 (1.0)      380.6467 (1.0)      0.0000 (1.0)      380.6467 (1.0)      0.0000 (1.0)           0;0  2.6271 (1.0)           1           1
test_list-list-dvc-only[3.10.0]     407.5920 (1.07)     407.5920 (1.07)     407.5920 (1.07)     0.0000 (1.0)      407.5920 (1.07)     0.0000 (1.0)           0;0  2.4534 (0.93)          1           1
test_list-list-dvc-only[main]       419.0412 (1.10)     419.0412 (1.10)     419.0412 (1.10)     0.0000 (1.0)      419.0412 (1.10)     0.0000 (1.0)           0;0  2.3864 (0.91)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_list-list-recursive': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-recursive[2.58.2]     463.7422 (1.0)      463.7422 (1.0)      463.7422 (1.0)      0.0000 (1.0)      463.7422 (1.0)      0.0000 (1.0)           0;0  2.1564 (1.0)           1           1
test_list-list-recursive[3.10.0]     476.0192 (1.03)     476.0192 (1.03)     476.0192 (1.03)     0.0000 (1.0)      476.0192 (1.03)     0.0000 (1.0)           0;0  2.1008 (0.97)          1           1
test_list-list-recursive[main]       474.4554 (1.02)     474.4554 (1.02)     474.4554 (1.02)     0.0000 (1.0)      474.4554 (1.02)     0.0000 (1.0)           0;0  2.1077 (0.98)          1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_list-list-shallow': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                       Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_list-list-shallow[2.58.2]     390.8286 (1.01)     390.8286 (1.01)     390.8286 (1.01)     0.0000 (1.0)      390.8286 (1.01)     0.0000 (1.0)           0;0  2.5587 (0.99)          1           1
test_list-list-shallow[3.10.0]     400.4318 (1.04)     400.4318 (1.04)     400.4318 (1.04)     0.0000 (1.0)      400.4318 (1.04)     0.0000 (1.0)           0;0  2.4973 (0.97)          1           1
test_list-list-shallow[main]       386.4867 (1.0)      386.4867 (1.0)      386.4867 (1.0)      0.0000 (1.0)      386.4867 (1.0)      0.0000 (1.0)           0;0  2.5874 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_partial_add-add': 3 tests ------------------------------------------------------------------------
Name (time in s)                    Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-add[2.58.2]     1.9706 (1.03)     1.9706 (1.03)     1.9706 (1.03)     0.0000 (1.0)      1.9706 (1.03)     0.0000 (1.0)           0;0  0.5075 (0.97)          1           1
test_partial_add-add[3.10.0]     1.9889 (1.04)     1.9889 (1.04)     1.9889 (1.04)     0.0000 (1.0)      1.9889 (1.04)     0.0000 (1.0)           0;0  0.5028 (0.96)          1           1
test_partial_add-add[main]       1.9061 (1.0)      1.9061 (1.0)      1.9061 (1.0)      0.0000 (1.0)      1.9061 (1.0)      0.0000 (1.0)           0;0  0.5246 (1.0)           1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_partial_add-add-partial': 3 tests ------------------------------------------------------------------------
Name (time in s)                            Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-add-partial[2.58.2]     2.9281 (1.03)     2.9281 (1.03)     2.9281 (1.03)     0.0000 (1.0)      2.9281 (1.03)     0.0000 (1.0)           0;0  0.3415 (0.97)          1           1
test_partial_add-add-partial[3.10.0]     2.9683 (1.04)     2.9683 (1.04)     2.9683 (1.04)     0.0000 (1.0)      2.9683 (1.04)     0.0000 (1.0)           0;0  0.3369 (0.96)          1           1
test_partial_add-add-partial[main]       2.8452 (1.0)      2.8452 (1.0)      2.8452 (1.0)      0.0000 (1.0)      2.8452 (1.0)      0.0000 (1.0)           0;0  0.3515 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_partial_add-gc-cloud-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                               Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-gc-cloud-noop[2.58.2]     552.2388 (1.0)      552.2388 (1.0)      552.2388 (1.0)      0.0000 (1.0)      552.2388 (1.0)      0.0000 (1.0)           0;0  1.8108 (1.0)           1           1
test_partial_add-gc-cloud-noop[3.10.0]     665.6549 (1.21)     665.6549 (1.21)     665.6549 (1.21)     0.0000 (1.0)      665.6549 (1.21)     0.0000 (1.0)           0;0  1.5023 (0.83)          1           1
test_partial_add-gc-cloud-noop[main]       668.6389 (1.21)     668.6389 (1.21)     668.6389 (1.21)     0.0000 (1.0)      668.6389 (1.21)     0.0000 (1.0)           0;0  1.4956 (0.83)          1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_partial_add-gc-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-gc-noop[2.58.2]     469.1639 (1.0)      469.1639 (1.0)      469.1639 (1.0)      0.0000 (1.0)      469.1639 (1.0)      0.0000 (1.0)           0;0  2.1315 (1.0)           1           1
test_partial_add-gc-noop[3.10.0]     549.2403 (1.17)     549.2403 (1.17)     549.2403 (1.17)     0.0000 (1.0)      549.2403 (1.17)     0.0000 (1.0)           0;0  1.8207 (0.85)          1           1
test_partial_add-gc-noop[main]       539.2337 (1.15)     539.2337 (1.15)     539.2337 (1.15)     0.0000 (1.0)      539.2337 (1.15)     0.0000 (1.0)           0;0  1.8545 (0.87)          1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_partial_add-push': 3 tests ------------------------------------------------------------------------
Name (time in s)                     Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-push[2.58.2]     1.9238 (1.82)     1.9238 (1.82)     1.9238 (1.82)     0.0000 (1.0)      1.9238 (1.82)     0.0000 (1.0)           0;0  0.5198 (0.55)          1           1
test_partial_add-push[3.10.0]     1.1959 (1.13)     1.1959 (1.13)     1.1959 (1.13)     0.0000 (1.0)      1.1959 (1.13)     0.0000 (1.0)           0;0  0.8362 (0.88)          1           1
test_partial_add-push[main]       1.0579 (1.0)      1.0579 (1.0)      1.0579 (1.0)      0.0000 (1.0)      1.0579 (1.0)      0.0000 (1.0)           0;0  0.9453 (1.0)           1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_partial_add-push-partial': 3 tests ------------------------------------------------------------------------
Name (time in s)                             Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_add-push-partial[2.58.2]     2.0155 (1.72)     2.0155 (1.72)     2.0155 (1.72)     0.0000 (1.0)      2.0155 (1.72)     0.0000 (1.0)           0;0  0.4962 (0.58)          1           1
test_partial_add-push-partial[3.10.0]     1.2155 (1.04)     1.2155 (1.04)     1.2155 (1.04)     0.0000 (1.0)      1.2155 (1.04)     0.0000 (1.0)           0;0  0.8227 (0.96)          1           1
test_partial_add-push-partial[main]       1.1725 (1.0)      1.1725 (1.0)      1.1725 (1.0)      0.0000 (1.0)      1.1725 (1.0)      0.0000 (1.0)           0;0  0.8529 (1.0)           1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_partial_remove-add': 3 tests ------------------------------------------------------------------------
Name (time in s)                       Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-add[2.58.2]     3.5507 (1.01)     3.5507 (1.01)     3.5507 (1.01)     0.0000 (1.0)      3.5507 (1.01)     0.0000 (1.0)           0;0  0.2816 (0.99)          1           1
test_partial_remove-add[3.10.0]     3.6806 (1.05)     3.6806 (1.05)     3.6806 (1.05)     0.0000 (1.0)      3.6806 (1.05)     0.0000 (1.0)           0;0  0.2717 (0.96)          1           1
test_partial_remove-add[main]       3.5210 (1.0)      3.5210 (1.0)      3.5210 (1.0)      0.0000 (1.0)      3.5210 (1.0)      0.0000 (1.0)           0;0  0.2840 (1.0)           1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_partial_remove-add-update': 3 tests ------------------------------------------------------------------------
Name (time in s)                              Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-add-update[2.58.2]     1.4753 (1.04)     1.4753 (1.04)     1.4753 (1.04)     0.0000 (1.0)      1.4753 (1.04)     0.0000 (1.0)           0;0  0.6778 (0.96)          1           1
test_partial_remove-add-update[3.10.0]     1.5470 (1.10)     1.5470 (1.10)     1.5470 (1.10)     0.0000 (1.0)      1.5470 (1.10)     0.0000 (1.0)           0;0  0.6464 (0.91)          1           1
test_partial_remove-add-update[main]       1.4128 (1.0)      1.4128 (1.0)      1.4128 (1.0)      0.0000 (1.0)      1.4128 (1.0)      0.0000 (1.0)           0;0  0.7078 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_partial_remove-gc': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                       Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-gc[2.58.2]     499.7758 (1.0)      499.7758 (1.0)      499.7758 (1.0)      0.0000 (1.0)      499.7758 (1.0)      0.0000 (1.0)           0;0  2.0009 (1.0)           1           1
test_partial_remove-gc[3.10.0]     560.6367 (1.12)     560.6367 (1.12)     560.6367 (1.12)     0.0000 (1.0)      560.6367 (1.12)     0.0000 (1.0)           0;0  1.7837 (0.89)          1           1
test_partial_remove-gc[main]       531.7372 (1.06)     531.7372 (1.06)     531.7372 (1.06)     0.0000 (1.0)      531.7372 (1.06)     0.0000 (1.0)           0;0  1.8806 (0.94)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_partial_remove-gc-cloud': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                             Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-gc-cloud[2.58.2]     556.2275 (1.0)      556.2275 (1.0)      556.2275 (1.0)      0.0000 (1.0)      556.2275 (1.0)      0.0000 (1.0)           0;0  1.7978 (1.0)           1           1
test_partial_remove-gc-cloud[3.10.0]     614.1273 (1.10)     614.1273 (1.10)     614.1273 (1.10)     0.0000 (1.0)      614.1273 (1.10)     0.0000 (1.0)           0;0  1.6283 (0.91)          1           1
test_partial_remove-gc-cloud[main]       623.1564 (1.12)     623.1564 (1.12)     623.1564 (1.12)     0.0000 (1.0)      623.1564 (1.12)     0.0000 (1.0)           0;0  1.6047 (0.89)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_partial_remove-push': 3 tests ------------------------------------------------------------------------
Name (time in s)                        Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-push[2.58.2]     3.4111 (2.36)     3.4111 (2.36)     3.4111 (2.36)     0.0000 (1.0)      3.4111 (2.36)     0.0000 (1.0)           0;0  0.2932 (0.42)          1           1
test_partial_remove-push[3.10.0]     1.9370 (1.34)     1.9370 (1.34)     1.9370 (1.34)     0.0000 (1.0)      1.9370 (1.34)     0.0000 (1.0)           0;0  0.5163 (0.75)          1           1
test_partial_remove-push[main]       1.4452 (1.0)      1.4452 (1.0)      1.4452 (1.0)      0.0000 (1.0)      1.4452 (1.0)      0.0000 (1.0)           0;0  0.6920 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_partial_remove-push-update': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_partial_remove-push-update[2.58.2]     502.1451 (1.03)     502.1451 (1.03)     502.1451 (1.03)     0.0000 (1.0)      502.1451 (1.03)     0.0000 (1.0)           0;0  1.9915 (0.97)          1           1
test_partial_remove-push-update[3.10.0]     489.4860 (1.0)      489.4860 (1.0)      489.4860 (1.0)      0.0000 (1.0)      489.4860 (1.0)      0.0000 (1.0)           0;0  2.0430 (1.0)           1           1
test_partial_remove-push-update[main]       798.7968 (1.63)     798.7968 (1.63)     798.7968 (1.63)     0.0000 (1.0)      798.7968 (1.63)     0.0000 (1.0)           0;0  1.2519 (0.61)          1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

----------------------------------------------------------------------------- benchmark 'test_plots-plots-diff': 3 tests -----------------------------------------------------------------------------
Name (time in ms)                      Min                 Max                Mean             StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-diff[2.58.2]     439.5933 (1.0)      460.8528 (1.0)      452.9287 (1.0)       9.1301 (1.0)      456.6835 (1.0)      14.6426 (1.0)           1;0  2.2079 (1.0)           5           3
test_plots-plots-diff[3.10.0]     801.8931 (1.82)     825.5366 (1.79)     813.5257 (1.80)     10.0114 (1.10)     812.1023 (1.78)     17.4696 (1.19)          2;0  1.2292 (0.56)          5           3
test_plots-plots-diff[main]       731.9990 (1.67)     755.1378 (1.64)     742.1319 (1.64)     10.4930 (1.15)     736.6110 (1.61)     18.0901 (1.24)          1;0  1.3475 (0.61)          5           3
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------- benchmark 'test_plots-plots-diff-json': 3 tests ---------------------------------------------------------------------------------
Name (time in ms)                             Min                   Max                  Mean            StdDev                Median                IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-diff-json[2.58.2]       460.0625 (1.0)        469.5061 (1.0)        463.5620 (1.0)      3.7337 (1.0)        462.8633 (1.0)       4.9960 (1.0)           1;0  2.1572 (1.0)           5           3
test_plots-plots-diff-json[3.10.0]     1,058.6842 (2.30)     1,076.4828 (2.29)     1,068.6667 (2.31)     6.6639 (1.78)     1,068.5134 (2.31)      8.4426 (1.69)          2;0  0.9357 (0.43)          5           3
test_plots-plots-diff-json[main]         809.0769 (1.76)       827.2183 (1.76)       818.2181 (1.77)     8.0783 (2.16)       819.9577 (1.77)     14.7179 (2.95)          2;0  1.2222 (0.57)          5           3
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'test_plots-plots-show': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                      Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-show[2.58.2]     407.1416 (1.0)      429.4013 (1.0)      417.0928 (1.0)      9.5002 (1.70)     419.6948 (1.0)      15.7063 (1.60)          2;0  2.3975 (1.0)           5           3
test_plots-plots-show[3.10.0]     672.4835 (1.65)     685.5009 (1.60)     678.7928 (1.63)     5.5803 (1.0)      679.7049 (1.62)      9.7980 (1.0)           2;0  1.4732 (0.61)          5           3
test_plots-plots-show[main]       661.6449 (1.63)     683.5411 (1.59)     673.1780 (1.61)     8.3771 (1.50)     675.8284 (1.61)     11.4907 (1.17)          2;0  1.4855 (0.62)          5           3
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------- benchmark 'test_plots-plots-show-json': 3 tests -----------------------------------------------------------------------------
Name (time in ms)                           Min                 Max                Mean             StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_plots-plots-show-json[2.58.2]     431.7842 (1.0)      452.6807 (1.0)      437.3701 (1.0)       8.6233 (1.0)      434.2619 (1.0)       5.7841 (1.0)           1;1  2.2864 (1.0)           5           3
test_plots-plots-show-json[3.10.0]     809.2985 (1.87)     849.9840 (1.88)     829.1145 (1.90)     18.8069 (2.18)     832.8365 (1.92)     35.3010 (6.10)          3;0  1.2061 (0.53)          5           3
test_plots-plots-show-json[main]       700.2441 (1.62)     722.4780 (1.60)     707.7680 (1.62)      8.8074 (1.02)     703.5470 (1.62)      9.6772 (1.67)          1;0  1.4129 (0.62)          5           3
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_push-push': 3 tests ------------------------------------------------------------------------
Name (time in s)              Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_push-push[2.58.2]     3.4711 (2.35)     3.4711 (2.35)     3.4711 (2.35)     0.0000 (1.0)      3.4711 (2.35)     0.0000 (1.0)           0;0  0.2881 (0.43)          1           1
test_push-push[3.10.0]     1.9266 (1.30)     1.9266 (1.30)     1.9266 (1.30)     0.0000 (1.0)      1.9266 (1.30)     0.0000 (1.0)           0;0  0.5191 (0.77)          1           1
test_push-push[main]       1.4776 (1.0)      1.4776 (1.0)      1.4776 (1.0)      0.0000 (1.0)      1.4776 (1.0)      0.0000 (1.0)           0;0  0.6768 (1.0)           1           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_sharing-add': 3 tests ------------------------------------------------------------------------
Name (time in s)                Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add[2.58.2]     3.5785 (1.03)     3.5785 (1.03)     3.5785 (1.03)     0.0000 (1.0)      3.5785 (1.03)     0.0000 (1.0)           0;0  0.2794 (0.97)          1           1
test_sharing-add[3.10.0]     3.6252 (1.04)     3.6252 (1.04)     3.6252 (1.04)     0.0000 (1.0)      3.6252 (1.04)     0.0000 (1.0)           0;0  0.2758 (0.96)          1           1
test_sharing-add[main]       3.4841 (1.0)      3.4841 (1.0)      3.4841 (1.0)      0.0000 (1.0)      3.4841 (1.0)      0.0000 (1.0)           0;0  0.2870 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing-add-noop': 3 tests ------------------------------------------------------------------------
Name (time in s)                     Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-add-noop[2.58.2]     2.4442 (1.04)     2.4442 (1.04)     2.4442 (1.04)     0.0000 (1.0)      2.4442 (1.04)     0.0000 (1.0)           0;0  0.4091 (0.97)          1           1
test_sharing-add-noop[3.10.0]     2.5133 (1.07)     2.5133 (1.07)     2.5133 (1.07)     0.0000 (1.0)      2.5133 (1.07)     0.0000 (1.0)           0;0  0.3979 (0.94)          1           1
test_sharing-add-noop[main]       2.3592 (1.0)      2.3592 (1.0)      2.3592 (1.0)      0.0000 (1.0)      2.3592 (1.0)      0.0000 (1.0)           0;0  0.4239 (1.0)           1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing-checkout': 3 tests ------------------------------------------------------------------------
Name (time in s)                     Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout[2.58.2]     1.7977 (1.28)     1.7977 (1.28)     1.7977 (1.28)     0.0000 (1.0)      1.7977 (1.28)     0.0000 (1.0)           0;0  0.5563 (0.78)          1           1
test_sharing-checkout[3.10.0]     3.7607 (2.68)     3.7607 (2.68)     3.7607 (2.68)     0.0000 (1.0)      3.7607 (2.68)     0.0000 (1.0)           0;0  0.2659 (0.37)          1           1
test_sharing-checkout[main]       1.4013 (1.0)      1.4013 (1.0)      1.4013 (1.0)      0.0000 (1.0)      1.4013 (1.0)      0.0000 (1.0)           0;0  0.7136 (1.0)           1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_sharing-checkout-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                           Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-checkout-noop[2.58.2]     632.8993 (1.07)     632.8993 (1.07)     632.8993 (1.07)     0.0000 (1.0)      632.8993 (1.07)     0.0000 (1.0)           0;0  1.5800 (0.94)          1           1
test_sharing-checkout-noop[3.10.0]     599.3414 (1.01)     599.3414 (1.01)     599.3414 (1.01)     0.0000 (1.0)      599.3414 (1.01)     0.0000 (1.0)           0;0  1.6685 (0.99)          1           1
test_sharing-checkout-noop[main]       593.5290 (1.0)      593.5290 (1.0)      593.5290 (1.0)      0.0000 (1.0)      593.5290 (1.0)      0.0000 (1.0)           0;0  1.6848 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing-fetch': 3 tests ------------------------------------------------------------------------
Name (time in s)                  Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-fetch[2.58.2]     3.8656 (2.09)     3.8656 (2.09)     3.8656 (2.09)     0.0000 (1.0)      3.8656 (2.09)     0.0000 (1.0)           0;0  0.2587 (0.48)          1           1
test_sharing-fetch[3.10.0]     4.3252 (2.34)     4.3252 (2.34)     4.3252 (2.34)     0.0000 (1.0)      4.3252 (2.34)     0.0000 (1.0)           0;0  0.2312 (0.43)          1           1
test_sharing-fetch[main]       1.8479 (1.0)      1.8479 (1.0)      1.8479 (1.0)      0.0000 (1.0)      1.8479 (1.0)      0.0000 (1.0)           0;0  0.5412 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_sharing-fetch-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                        Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-fetch-noop[2.58.2]     426.2530 (1.0)      426.2530 (1.0)      426.2530 (1.0)      0.0000 (1.0)      426.2530 (1.0)      0.0000 (1.0)           0;0  2.3460 (1.0)           1           1
test_sharing-fetch-noop[3.10.0]     515.3691 (1.21)     515.3691 (1.21)     515.3691 (1.21)     0.0000 (1.0)      515.3691 (1.21)     0.0000 (1.0)           0;0  1.9404 (0.83)          1           1
test_sharing-fetch-noop[main]       830.1326 (1.95)     830.1326 (1.95)     830.1326 (1.95)     0.0000 (1.0)      830.1326 (1.95)     0.0000 (1.0)           0;0  1.2046 (0.51)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_sharing-push': 3 tests ------------------------------------------------------------------------
Name (time in s)                 Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push[2.58.2]     3.4250 (2.30)     3.4250 (2.30)     3.4250 (2.30)     0.0000 (1.0)      3.4250 (2.30)     0.0000 (1.0)           0;0  0.2920 (0.43)          1           1
test_sharing-push[3.10.0]     1.9421 (1.31)     1.9421 (1.31)     1.9421 (1.31)     0.0000 (1.0)      1.9421 (1.31)     0.0000 (1.0)           0;0  0.5149 (0.77)          1           1
test_sharing-push[main]       1.4859 (1.0)      1.4859 (1.0)      1.4859 (1.0)      0.0000 (1.0)      1.4859 (1.0)      0.0000 (1.0)           0;0  0.6730 (1.0)           1           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_sharing-push-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                       Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_sharing-push-noop[2.58.2]     451.3283 (1.0)      451.3283 (1.0)      451.3283 (1.0)      0.0000 (1.0)      451.3283 (1.0)      0.0000 (1.0)           0;0  2.2157 (1.0)           1           1
test_sharing-push-noop[3.10.0]     456.5832 (1.01)     456.5832 (1.01)     456.5832 (1.01)     0.0000 (1.0)      456.5832 (1.01)     0.0000 (1.0)           0;0  2.1902 (0.99)          1           1
test_sharing-push-noop[main]       832.0753 (1.84)     832.0753 (1.84)     832.0753 (1.84)     0.0000 (1.0)      832.0753 (1.84)     0.0000 (1.0)           0;0  1.2018 (0.54)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_status-status': 3 tests ------------------------------------------------------------------------
Name (time in s)                  Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status[2.58.2]     1.1757 (1.0)      1.1757 (1.0)      1.1757 (1.0)      0.0000 (1.0)      1.1757 (1.0)      0.0000 (1.0)           0;0  0.8505 (1.0)           1           1
test_status-status[3.10.0]     1.2455 (1.06)     1.2455 (1.06)     1.2455 (1.06)     0.0000 (1.0)      1.2455 (1.06)     0.0000 (1.0)           0;0  0.8029 (0.94)          1           1
test_status-status[main]       1.2126 (1.03)     1.2126 (1.03)     1.2126 (1.03)     0.0000 (1.0)      1.2126 (1.03)     0.0000 (1.0)           0;0  0.8247 (0.97)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_status-status-changed': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                           Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed[2.58.2]     524.0504 (1.0)      524.0504 (1.0)      524.0504 (1.0)      0.0000 (1.0)      524.0504 (1.0)      0.0000 (1.0)           0;0  1.9082 (1.0)           1           1
test_status-status-changed[3.10.0]     556.6211 (1.06)     556.6211 (1.06)     556.6211 (1.06)     0.0000 (1.0)      556.6211 (1.06)     0.0000 (1.0)           0;0  1.7966 (0.94)          1           1
test_status-status-changed[main]       533.7926 (1.02)     533.7926 (1.02)     533.7926 (1.02)     0.0000 (1.0)      533.7926 (1.02)     0.0000 (1.0)           0;0  1.8734 (0.98)          1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_status-status-changed-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                                Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-changed-noop[2.58.2]     527.7007 (1.01)     527.7007 (1.01)     527.7007 (1.01)     0.0000 (1.0)      527.7007 (1.01)     0.0000 (1.0)           0;0  1.8950 (0.99)          1           1
test_status-status-changed-noop[3.10.0]     543.5695 (1.04)     543.5695 (1.04)     543.5695 (1.04)     0.0000 (1.0)      543.5695 (1.04)     0.0000 (1.0)           0;0  1.8397 (0.96)          1           1
test_status-status-changed-noop[main]       521.3415 (1.0)      521.3415 (1.0)      521.3415 (1.0)      0.0000 (1.0)      521.3415 (1.0)      0.0000 (1.0)           0;0  1.9181 (1.0)           1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_status-status-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                        Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_status-status-noop[2.58.2]     525.8183 (1.0)      525.8183 (1.0)      525.8183 (1.0)      0.0000 (1.0)      525.8183 (1.0)      0.0000 (1.0)           0;0  1.9018 (1.0)           1           1
test_status-status-noop[3.10.0]     542.9308 (1.03)     542.9308 (1.03)     542.9308 (1.03)     0.0000 (1.0)      542.9308 (1.03)     0.0000 (1.0)           0;0  1.8419 (0.97)          1           1
test_status-status-noop[main]       530.6451 (1.01)     530.6451 (1.01)     530.6451 (1.01)     0.0000 (1.0)      530.6451 (1.01)     0.0000 (1.0)           0;0  1.8845 (0.99)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

------------------------------------------------------------------------ benchmark 'test_update-import-url': 3 tests ------------------------------------------------------------------------
Name (time in s)                      Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_update-import-url[2.58.2]     7.2695 (1.45)     7.2695 (1.45)     7.2695 (1.45)     0.0000 (1.0)      7.2695 (1.45)     0.0000 (1.0)           0;0  0.1376 (0.69)          1           1
test_update-import-url[3.10.0]     7.4529 (1.49)     7.4529 (1.49)     7.4529 (1.49)     0.0000 (1.0)      7.4529 (1.49)     0.0000 (1.0)           0;0  0.1342 (0.67)          1           1
test_update-import-url[main]       4.9996 (1.0)      4.9996 (1.0)      4.9996 (1.0)      0.0000 (1.0)      4.9996 (1.0)      0.0000 (1.0)           0;0  0.2000 (1.0)           1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------ benchmark 'test_update-update': 3 tests ------------------------------------------------------------------------
Name (time in s)                  Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_update-update[2.58.2]     6.4723 (1.51)     6.4723 (1.51)     6.4723 (1.51)     0.0000 (1.0)      6.4723 (1.51)     0.0000 (1.0)           0;0  0.1545 (0.66)          1           1
test_update-update[3.10.0]     6.6660 (1.56)     6.6660 (1.56)     6.6660 (1.56)     0.0000 (1.0)      6.6660 (1.56)     0.0000 (1.0)           0;0  0.1500 (0.64)          1           1
test_update-update[main]       4.2840 (1.0)      4.2840 (1.0)      4.2840 (1.0)      0.0000 (1.0)      4.2840 (1.0)      0.0000 (1.0)           0;0  0.2334 (1.0)           1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------- benchmark 'test_update-update-noop': 3 tests ----------------------------------------------------------------------------
Name (time in ms)                        Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_update-update-noop[2.58.2]     954.3741 (1.0)      954.3741 (1.0)      954.3741 (1.0)      0.0000 (1.0)      954.3741 (1.0)      0.0000 (1.0)           0;0  1.0478 (1.0)           1           1
test_update-update-noop[3.10.0]     988.7640 (1.04)     988.7640 (1.04)     988.7640 (1.04)     0.0000 (1.0)      988.7640 (1.04)     0.0000 (1.0)           0;0  1.0114 (0.97)          1           1
test_update-update-noop[main]       963.0470 (1.01)     963.0470 (1.01)     963.0470 (1.01)     0.0000 (1.0)      963.0470 (1.01)     0.0000 (1.0)           0;0  1.0384 (0.99)          1           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean