Closed gabbard closed 5 years ago
------------------------------------------------------------------------------------------ benchmark 'Creating from big frozenset': 3 tests -----------------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
frozenset 41.9580 (1.0) 440.5594 (1.0) 47.1240 (1.0) 13.5645 (1.0) 41.9580 (1.0) 6.9930 (1.0) 3463;3463 21,220,615.7540 (1.0) 100000 143
set 51,000.0000 (>1000.0) 409,000.0000 (928.37) 56,053.7589 (>1000.0) 13,807.9584 (>1000.0) 52,000.0000 (>1000.0) 1,000.0000 (143.00) 800;2142 17,840.0168 (0.00) 9524 1
ImmutableSet 1,047,000.0000 (>1000.0) 2,001,000.0000 (>1000.0) 1,171,621.0996 (>1000.0) 135,563.1902 (>1000.0) 1,127,000.0000 (>1000.0) 163,250.0000 (>1000.0) 113;16 853.5183 (0.00) 673 1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------ benchmark 'Creating from big list': 6 tests ------------------------------------------------------------------------------
Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
list 26.0000 (1.0) 342.0000 (1.0) 29.3941 (1.0) 8.3899 (1.0) 28.0000 (1.0) 2.0000 (2.00) 1385;1640 34,020.4471 (1.0) 21277 1
tuple 27.0000 (1.04) 964.0000 (2.82) 30.4866 (1.04) 11.8144 (1.41) 28.0000 (1.00) 1.0000 (1.0) 1297;3650 32,801.3134 (0.96) 23256 1
ImmutableList 61.0000 (2.35) 788.0000 (2.30) 73.9014 (2.51) 26.8039 (3.19) 64.0000 (2.29) 8.0000 (8.00) 1010;1830 13,531.5434 (0.40) 10000 1
frozenset 191.0000 (7.35) 1,343.0000 (3.93) 226.6852 (7.71) 59.9255 (7.14) 200.0000 (7.14) 43.0000 (43.00) 450;371 4,411.4041 (0.13) 4066 1
set 191.0000 (7.35) 1,873.0000 (5.48) 232.9883 (7.93) 94.1754 (11.22) 199.0000 (7.11) 40.0000 (40.00) 327;479 4,292.0610 (0.13) 4017 1
ImmutableSet 1,193.0000 (45.88) 5,607.0000 (16.39) 1,540.8000 (52.42) 476.2356 (56.76) 1,363.5000 (48.70) 318.0000 (318.00) 85;82 649.0135 (0.02) 750 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------- benchmark 'Creating from big set': 3 tests ------------------------------------------------------------------------------
Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
frozenset 49.0000 (1.0) 405.0000 (1.32) 60.5523 (1.0) 17.0886 (1.0) 52.0000 (1.0) 11.0000 (1.0) 1742;1639 16,514.6527 (1.0) 13158 1
set 50.0000 (1.02) 307.0000 (1.0) 63.3455 (1.05) 19.5377 (1.14) 55.0000 (1.06) 19.0000 (1.73) 343;140 15,786.4505 (0.96) 2501 1
ImmutableSet 1,049.0000 (21.41) 2,389.0000 (7.78) 1,305.2515 (21.56) 273.7017 (16.02) 1,196.0000 (23.00) 311.0000 (28.27) 84;27 766.1359 (0.05) 497 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------- benchmark 'Creating from big tuple': 6 tests --------------------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuple 55.8659 (1.0) 2,256.9832 (1.0) 64.6637 (1.0) 17.0767 (1.0) 61.4525 (1.0) 5.5866 (1.0) 4722;4987 15,464,616.6978 (1.0) 100000 179
list 26,000.0000 (465.40) 396,000.0000 (175.46) 28,336.0852 (438.21) 9,242.3130 (541.22) 27,000.0000 (439.36) 2,000.0000 (358.00) 500;645 35,290.6900 (0.00) 10417 1
ImmutableList 61,000.0000 (>1000.0) 786,000.0000 (348.25) 68,057.6265 (>1000.0) 21,018.2130 (>1000.0) 63,000.0000 (>1000.0) 4,000.0000 (716.00) 340;648 14,693.4304 (0.00) 5848 1
frozenset 190,000.0000 (>1000.0) 996,000.0000 (441.30) 211,638.3372 (>1000.0) 41,120.5700 (>1000.0) 194,000.0000 (>1000.0) 22,000.0000 (>1000.0) 505;536 4,725.0418 (0.00) 4330 1
set 190,000.0000 (>1000.0) 536,000.0000 (237.49) 205,133.5006 (>1000.0) 29,371.3884 (>1000.0) 193,000.0000 (>1000.0) 8,000.0000 (>1000.0) 395;880 4,874.8742 (0.00) 3985 1
ImmutableSet 1,175,000.0000 (>1000.0) 11,467,000.0000 (>1000.0) 1,563,293.2227 (>1000.0) 662,502.3094 (>1000.0) 1,335,000.0000 (>1000.0) 417,750.0000 (>1000.0) 69;64 639.6753 (0.00) 723 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------ benchmark 'Creating from empty frozenset': 3 tests -----------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
frozenset 37.6569 (1.0) 439.3305 (1.0) 46.2467 (1.0) 14.5165 (1.0) 41.8410 (1.0) 4.1841 (1.0) 7662;8888 21,623.1798 (1.0) 90910 239
set 64.7482 (1.72) 2,697.8417 (6.14) 88.7194 (1.92) 35.8345 (2.47) 71.9424 (1.72) 14.3885 (3.44) 17046;22395 11,271.4918 (0.52) 100001 139
ImmutableSet 6,000.0000 (159.33) 88,000.0000 (200.30) 8,164.0147 (176.53) 4,265.4555 (293.83) 7,000.0000 (167.30) 1,000.0000 (239.00) 951;1435 122.4888 (0.01) 14706 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'Creating from empty list': 6 tests ---------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuple 60.0000 (1.0) 830.0000 (1.12) 72.6964 (1.0) 20.2048 (1.0) 70.0000 (1.0) 10.0000 (1.0) 2314;2314 13,755.8341 (1.0) 76924 100
list 80.0000 (1.33) 740.0000 (1.0) 94.6145 (1.30) 23.7537 (1.18) 90.0000 (1.29) 10.0000 (1.00) 3915;3915 10,569.2045 (0.77) 100000 100
frozenset 90.0000 (1.50) 1,190.0000 (1.61) 116.9546 (1.61) 42.7023 (2.11) 100.0000 (1.43) 20.0000 (2.00) 10384;11852 8,550.3231 (0.62) 58824 100
set 90.0000 (1.50) 4,790.0000 (6.47) 122.5634 (1.69) 47.7687 (2.36) 100.0000 (1.43) 60.0000 (6.00) 14988;2155 8,159.0443 (0.59) 100001 100
ImmutableList 4,000.0000 (66.67) 82,000.0000 (110.81) 5,577.7142 (76.73) 2,464.0386 (121.95) 5,000.0000 (71.43) 1,000.0000 (100.00) 583;705 179.2849 (0.01) 18182 1
ImmutableSet 6,000.0000 (100.00) 130,000.0000 (175.68) 8,149.2080 (112.10) 4,010.1985 (198.48) 7,000.0000 (100.00) 1,000.0000 (100.00) 891;1827 122.7113 (0.01) 13699 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------- benchmark 'Creating from empty set': 3 tests --------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
frozenset 62.9371 (1.0) 1,216.7832 (1.0) 77.0403 (1.03) 22.8540 (1.06) 69.9301 (1.0) 6.9930 (1.0) 5920;6708 12,980.2111 (0.97) 100000 143
set 64.7482 (1.03) 1,690.6475 (1.39) 74.5329 (1.0) 21.6329 (1.0) 71.9424 (1.03) 7.1942 (1.03) 4646;6274 13,416.9024 (1.0) 100001 139
ImmutableSet 6,000.0000 (95.33) 96,000.0000 (78.90) 8,367.8286 (112.27) 4,510.1651 (208.49) 7,000.0000 (100.10) 1,000.0000 (143.00) 896;1400 119.5053 (0.01) 11905 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------- benchmark 'Creating from empty tuple': 6 tests --------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuple 53.8922 (1.0) 425.1497 (1.0) 66.0407 (1.0) 16.2152 (1.0) 59.8802 (1.0) 5.9880 (1.0) 4689;6361 15,142.1764 (1.0) 90910 167
list 80.0000 (1.48) 500.0000 (1.18) 105.6030 (1.60) 27.6105 (1.70) 100.0000 (1.67) 20.0000 (3.34) 537;537 9,469.4236 (0.63) 9709 100
frozenset 90.0000 (1.67) 830.0000 (1.95) 104.9845 (1.59) 33.5786 (2.07) 100.0000 (1.67) 10.0000 (1.67) 3538;4144 9,525.2156 (0.63) 50001 100
set 90.0000 (1.67) 1,210.0000 (2.85) 110.5641 (1.67) 39.7027 (2.45) 100.0000 (1.67) 20.0000 (3.34) 11815;14122 9,044.5276 (0.60) 100001 100
ImmutableList 4,000.0000 (74.22) 85,000.0000 (199.93) 5,503.6959 (83.34) 2,414.3008 (148.89) 5,000.0000 (83.50) 1,000.0000 (167.00) 1044;1044 181.6961 (0.01) 30304 1
ImmutableSet 6,000.0000 (111.33) 164,000.0000 (385.75) 8,201.8215 (124.19) 4,415.8680 (272.33) 7,000.0000 (116.90) 1,000.0000 (167.00) 931;2040 121.9241 (0.01) 15152 1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------- benchmark 'Creating from small frozenset': 3 tests --------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
frozenset 80.3571 (1.0) 785.7143 (1.0) 95.2259 (1.0) 29.4567 (1.0) 89.2857 (1.0) 8.9286 (1.0) 8731;11716 10,501.3455 (1.0) 100000 112
set 83.3333 (1.04) 1,116.6667 (1.42) 107.2887 (1.13) 42.1724 (1.43) 100.0000 (1.12) 16.6667 (1.87) 11691;13442 9,320.6452 (0.89) 90910 60
ImmutableSet 13,000.0000 (161.78) 210,000.0000 (267.27) 16,594.3953 (174.26) 6,900.8404 (234.27) 15,000.0000 (168.00) 2,000.0000 (224.00) 1285;1731 60.2613 (0.01) 16950 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------- benchmark 'Creating from small list': 6 tests -----------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuple 72.0000 (1.0) 1,072.0000 (1.20) 86.6342 (1.0) 25.1236 (1.0) 80.0000 (1.0) 8.0000 (1.0) 5933;6232 11,542.7902 (1.0) 90910 125
list 100.0000 (1.39) 990.0000 (1.11) 114.2461 (1.32) 28.4773 (1.13) 110.0000 (1.38) 10.0000 (1.25) 4760;6132 8,753.0365 (0.76) 90910 100
frozenset 110.0000 (1.53) 890.0000 (1.0) 139.8081 (1.61) 45.4886 (1.81) 120.0000 (1.50) 10.0000 (1.25) 10852;14194 7,152.6603 (0.62) 83334 100
set 110.0000 (1.53) 11,360.0000 (12.76) 133.5510 (1.54) 64.4241 (2.56) 120.0000 (1.50) 10.0000 (1.25) 3553;4685 7,487.7746 (0.65) 47620 100
ImmutableList 5,000.0000 (69.44) 114,000.0000 (128.09) 7,478.4640 (86.32) 3,742.3074 (148.96) 7,000.0000 (87.50) 1,000.0000 (125.00) 1659;2457 133.7173 (0.01) 31250 1
ImmutableSet 13,000.0000 (180.56) 173,000.0000 (194.38) 17,962.3630 (207.34) 9,218.3517 (366.92) 15,000.0000 (187.50) 2,000.0000 (250.00) 1663;2603 55.6720 (0.00) 18519 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------- benchmark 'Creating from small set': 3 tests -----------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
frozenset 80.0000 (1.0) 1,230.0000 (1.64) 91.4462 (1.01) 24.1262 (1.0) 90.0000 (1.0) 10.0000 (1.0) 4255;4958 10,935.3915 (0.99) 100000 100
set 80.0000 (1.0) 750.0000 (1.0) 90.8813 (1.0) 26.9643 (1.12) 90.0000 (1.0) 10.0000 (1.0) 1870;2172 11,003.3670 (1.0) 41667 100
ImmutableSet 13,000.0000 (162.50) 145,000.0000 (193.33) 17,360.5310 (191.02) 8,260.1507 (342.37) 15,000.0000 (166.67) 2,000.0000 (200.00) 1687;2385 57.6019 (0.01) 16950 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------- benchmark 'Creating from small tuple': 6 tests ----------------------------------------------------------------------------------
Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuple 57.3248 (1.0) 394.9045 (1.0) 67.7698 (1.0) 18.0125 (1.0) 63.6943 (1.0) 12.7389 (1.27) 7259;7248 14,755.8333 (1.0) 100000 157
list 100.0000 (1.74) 920.0000 (2.33) 113.0101 (1.67) 25.9548 (1.44) 110.0000 (1.73) 10.0000 (1.0) 3489;5205 8,848.7651 (0.60) 90910 100
frozenset 110.0000 (1.92) 1,160.0000 (2.94) 132.2904 (1.95) 39.7962 (2.21) 120.0000 (1.88) 10.0000 (1.00) 7638;9008 7,559.1263 (0.51) 83334 100
set 110.0000 (1.92) 1,250.0000 (3.17) 129.8343 (1.92) 39.3428 (2.18) 120.0000 (1.88) 20.0000 (2.00) 6227;6589 7,702.1278 (0.52) 76924 100
ImmutableList 5,000.0000 (87.22) 94,000.0000 (238.03) 6,991.2502 (103.16) 2,641.3345 (146.64) 7,000.0000 (109.90) 1,000.0000 (100.00) 1013;1116 143.0359 (0.01) 28572 1
ImmutableSet 13,000.0000 (226.78) 168,000.0000 (425.42) 17,815.3248 (262.88) 8,571.2081 (475.85) 15,000.0000 (235.50) 2,000.0000 (200.00) 1255;2007 56.1314 (0.00) 12346 1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
=======================================================
Merging #22 into master will increase coverage by
0.15%
. The diff coverage isn/a
.
@@ Coverage Diff @@
## master #22 +/- ##
==========================================
+ Coverage 90.97% 91.13% +0.15%
==========================================
Files 7 7
Lines 576 575 -1
==========================================
Hits 524 524
+ Misses 52 51 -1
Impacted Files | Coverage Δ | |
---|---|---|
immutablecollections/immutableset.py | 87.04% <ø> (+0.44%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update a5984ce...a94a4fa. Read the comment docs.
Ready for re-review, be sure to note the last commit. I don't know why mypy didn't catch this before.
Partially addresses #15 , #14