search

openssl / project

Tracking of project related issues
2 stars 1 forks source link

Capture perf data for MacOSX 11 on x86_64 #586

Closed nhorman closed 5 months ago

nhorman commented 5 months ago
As per the description in openssl/project#577 record the requested perf programs output in the format specified: thread count number of iterations openssl 1.1.1 per operation avg usec openssl 3.0.0 avg usec master usec std deviation

And post results here and in openssl/project#577 

### Tasks
- [ ] https://github.com/openssl/private/issues/478
Sashan commented 5 months ago

make sure to use ./config (instead of ./Configure) script to set up your build. also make sure to always use clean source tree.

Sashan commented 5 months ago

evp_fetch

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 .107407 .002645 .106553 .002449 .106166 .002000
2 25 .210217 .006324 .208706 .005656 .212143 .005916
4 25 .498896 .003605 .500896 .004690 .499266 .003605
8 25 1.220927 .026608 1.234131 .016970 1.215973 .023345
16 25 2.851181 .097647 2.844786 .098203 2.862634 .084498
32 25 5.891213 .140758 5.853347 .176008 5.867714 .146512
64 25 12.161777 .292051 12.090883 .334683 12.072850 .323949

handshake

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 1068.805292 1.500326 1068.526872 3.207622 1068.262588 2.971279
2 25 1076.484148 4.226163 1075.637584 4.043849 1076.476668 4.036964
4 25 1139.331156 6.369512 1140.166872 7.638503 1136.561144 5.405924
8 25 1667.933140 30.174830 1674.693528 28.970226 1662.462744 26.468550
16 25 3116.548908 41.107117 3123.590668 44.643669 3117.735604 35.810539
32 25 6216.938933 45.891267 6206.209448 64.397754 6204.818961 57.056923
64 25 12413.494069 145.239455 12417.818634 136.238985 12399.904045 147.093044

newrawkey

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 2.058216 .015329 2.056107 .007348 2.057357 .008366
2 25 2.483357 .020784 2.491787 .022605 2.489193 .029899
4 25 2.964335 .046335 2.959571 .039974 2.960850 .029512
8 25 4.956714 .067134 4.967612 .065023 4.974907 .063984
16 25 10.703274 .253682 10.664812 .269807 10.714435 .287048
32 25 22.454479 .583642 22.249025 .603549 22.469316 .691770
64 25 49.855431 1.085600 50.219778 1.098067 49.981701 1.249850

providerdoall

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 .182718 .001732 .183732 .002236 .183244 .002236
2 25 .445169 .018138 .441974 .017578 .441099 .016852
4 25 .947395 .010954 .943890 .012328 .943401 .010723
8 25 2.186407 .088187 2.186842 .086017 2.152261 .081301
16 25 4.410106 .333830 4.464882 .358579 4.535078 .416592
32 25 8.986522 .776314 8.748686 .786027 8.491363 .701855
64 25 12.653564 .233863 12.758391 .283506 12.771754 .446709

randbytes

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 .431222 .002000 .431402 .003162 .430946 .001732
2 25 .483555 .005291 .481893 .003605 .483099 .004690
4 25 .492429 .002449 .493065 .006855 .497867 .016552
8 25 .920061 .015684 .947939 .136663 .921482 .014422
16 25 2.338150 .174155 2.361653 .166814 2.358720 .181703
32 25 4.511796 .389209 4.471037 .361959 4.587659 .337443
64 25 8.186506 .516370 8.648182 .727555 8.569789 .785143

rsasign

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 34.848489 .017691 34.860957 .049588 34.871458 .083060
2 25 35.275270 .121028 35.249907 .137709 35.229463 .106094
4 25 41.525833 .267110 41.438502 .229780 41.412760 .257732
8 25 60.702258 .511549 61.344629 .503497 60.981037 .631584
16 25 133.198815 .696065 133.096212 .695535 133.184608 .640207
32 25 278.158423 3.926719 273.739842 .892734 275.625687 3.724812
64 25 574.895573 10.085353 576.469054 7.634328 576.028017 7.475210

rwlocks

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 N/A N/A N/A N/A N/A N/A
2 25 3.306214 .022068 3.307544 .031575 3.278610 .040286
4 25 3.178639 .109256 3.181789 .151525 3.170664 .125443
8 25 3.204206 .078632 3.203022 .075159 3.192156 .089190
16 25 3.139111 .075670 3.180861 .084905 3.174309 .106240
32 25 2.935431 .064560 2.930543 .067889 2.931619 .072048
64 25 2.814059 .041267 2.833848 .033719 2.819064 .026210

sslnew

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 2.793753 .023916 2.804372 .034856 2.806729 .027928
2 25 3.144988 .047053 3.132254 .034481 3.133147 .035397
4 25 4.133564 .042213 4.146575 .055416 4.143362 .043726
8 25 9.024397 .197468 8.995041 .224942 8.963998 .207602
16 25 21.686208 .805274 21.750976 .757811 21.757871 .777348
32 25 42.666927 1.560820 43.429660 1.813306 43.576159 1.405094
64 25 85.679803 3.221943 86.210969 3.310796 85.961845 3.522217

x509storeissue

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 2.528112 .022338 2.532055 .030983 2.517468 .022781
2 25 2.945070 .024899 2.945448 .024145 2.951685 .020591
4 25 8.048242 1.018894 8.445787 .989864 8.108107 1.010857
8 25 48.227781 .223872 48.264002 .205652 48.233344 .188804
16 25 97.379154 .600104 97.353138 .522943 97.397330 .527343
32 25 195.829464 1.149861 195.840385 1.068193 195.853145 1.101411
64 25 405.942748 5.353241 410.256425 3.280184 408.204953 4.923107

pkeyread

der dh

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 1212.792560 9.156175 1219.442488 13.399631 1211.884420 8.177272
2 25 1217.922644 5.952862 1218.724992 4.875888 1219.402444 4.909603
4 25 1280.965112 12.467191 1280.950596 12.069471 1280.824612 12.580667
8 25 1884.623604 18.452448 1886.343260 15.224292 1888.911532 20.027561
16 25 3717.939544 50.103830 3705.822912 45.970864 3701.483436 48.262431
32 25 7543.902408 104.329414 7536.001713 98.252542 7553.607599 113.276942
64 25 15316.453422 312.248606 15252.859097 303.533147 15298.882360 269.581671

der dhx

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 249.478064 .185129 249.493824 .260541 249.484768 .222546
2 25 250.949100 .255043 250.864272 .278476 250.850752 .307175
4 25 272.102452 2.018856 270.589328 3.171979 271.146032 2.796996
8 25 405.434784 6.530930 404.789760 6.997359 401.442052 4.531645
16 25 827.496084 6.189461 827.705644 6.448666 828.558280 6.061743
32 25 1780.184500 5.146892 1774.315000 6.714493 1788.325618 18.015551
64 25 3768.671705 187.990744 3832.991789 178.017977 3771.000678 177.475819

der dsa

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 257.841904 .225816 257.838544 .212007 258.640520 3.420616
2 25 259.355412 .321003 259.417068 .368700 259.339048 .309523
4 25 280.488408 3.041096 280.907172 2.793014 279.022952 3.798488
8 25 417.871220 7.250622 418.520748 5.957799 417.708000 1.584601
16 25 857.702712 3.129422 852.726420 7.343061 856.966280 13.213474
32 25 1833.688051 19.844702 1820.789369 5.711306 1836.062212 23.638259
64 25 4072.679792 30.938349 4072.011777 13.759358 4107.308134 17.660038

der ec

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 31.877732 .247913 31.856044 .124374 31.897920 .288613
2 25 34.348968 .429681 34.195120 .498799 34.134868 .344193
4 25 41.590388 1.132532 41.691356 .636529 41.999568 1.145660
8 25 73.340996 2.365106 72.307336 1.701012 72.230692 1.281941
16 25 163.954024 4.407161 166.039776 6.226431 163.562132 3.792967
32 25 447.862667 9.828717 450.627591 14.242608 455.662915 17.795989
64 25 1429.743314 24.029160 1430.622859 20.439464 1444.511393 30.908126

der rsa

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 17.509648 .115199 17.569632 .114109 17.566044 .222326
2 25 19.851320 .248610 19.838144 .306929 19.894784 .228501
4 25 25.466220 .386894 25.449584 .440591 25.390448 .536180
8 25 46.911636 .842522 46.412444 .537363 46.363968 .500733
16 25 115.833416 1.246101 116.462176 1.395575 116.452612 1.250322
32 25 360.823126 4.015770 361.569181 2.736623 360.566301 2.721091
64 25 1242.620790 8.867752 1239.242935 9.080660 1244.424201 9.265838

der x25519

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 41.869712 .106084 41.837872 .134231 41.903592 .172921
2 25 43.428860 .149264 43.385848 .193005 43.372928 .137280
4 25 48.070532 .308209 48.087900 .292970 47.999052 .331131
8 25 76.242328 1.006695 76.482892 .850022 76.489208 1.073133
16 25 174.515112 .977781 172.531980 1.241615 174.742016 1.585073
32 25 467.630519 3.172632 467.436533 2.480146 465.773146 2.267565
64 25 1472.429373 32.386720 1483.883921 21.895496 1463.754708 32.706424

pem dh

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 1213.162796 13.088133 1207.571088 5.857143 1208.739016 5.941344
2 25 1223.025960 7.275827 1220.534288 10.193799 1222.509840 7.058377
4 25 1282.728224 11.605569 1287.041964 8.161448 1284.944496 8.817615
8 25 1915.552924 29.317597 1915.234688 31.767684 1912.779240 29.805739
16 25 3804.628200 50.487592 3792.518220 49.794299 3783.418660 53.047758
32 25 7723.734269 105.851221 7692.461821 92.797542 7690.586765 108.422629
64 25 15879.991621 219.999973 15890.128777 225.449240 15931.923366 307.283263

pem dhx

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 269.000288 .284332 268.921564 .294681 268.874280 .465359
2 25 270.442964 .354056 270.253252 .408891 270.272488 .340189
4 25 289.570512 .881292 289.275072 1.002019 289.824792 1.245473
8 25 435.572008 4.908348 435.654696 7.381037 435.760656 1.865680
16 25 891.575364 14.345909 891.349164 11.840187 888.417216 13.798211
32 25 1896.307563 17.662948 1895.247875 18.939956 1907.550039 14.947012
64 25 4321.619089 27.767859 4285.191653 45.272733 4300.288422 21.809156

pem dsa

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 288.093904 .754448 287.941408 .412502 287.926776 .593904
2 25 289.687092 .594952 289.620620 .605843 289.498588 .443134
4 25 310.280624 2.258125 311.599020 3.391487 311.024036 2.980072
8 25 461.076356 7.712280 462.808608 4.711920 458.103936 5.106642
16 25 926.321748 14.042247 922.726664 13.382998 924.539500 17.366860
32 25 1963.660463 17.870056 1963.699289 19.601704 1969.995814 17.459109
64 25 4376.355718 30.682724 4374.800830 19.699582 4397.392635 20.603442

pem ec

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 46.418612 .168139 46.603080 .472263 46.482904 .412336
2 25 49.773016 .444301 49.846924 .554808 50.055488 .501383
4 25 60.742800 1.423964 60.490336 1.477575 59.591620 1.157990
8 25 99.562260 1.249420 97.887856 1.376564 100.817728 2.125186
16 25 217.906620 5.005461 216.991732 4.317253 215.730684 2.654534
32 25 554.806301 12.895837 551.549764 10.736490 557.805339 14.751756
64 25 1598.446134 10.618536 1605.111589 13.911873 1599.471321 22.015448

pem rsa

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 44.755900 .168579 44.845740 .236603 44.752496 .189615
2 25 48.356204 .512784 48.234824 .551989 48.298804 .301181
4 25 56.573840 .645058 56.622816 .777463 56.437960 .766405
8 25 93.172372 1.126295 93.661980 1.238926 92.660244 1.221862
16 25 206.980112 2.539255 207.258220 2.118329 206.977872 2.581484
32 25 535.306853 4.016696 535.216665 6.005078 534.116856 5.593954
64 25 1587.475710 11.082581 1591.608206 9.253881 1582.712116 8.955417

pem x25519

thread count numder of iterations openssl 1.1.1 per operation avg usec 1.1,1 std dev openssl 3.0.0 avg usec 3.0 std dev master usec master std dev
1 25 52.809312 .134736 52.859304 .176045 52.840852 .137098
2 25 55.369980 .300501 55.436748 .213438 55.383984 .260161
4 25 62.023200 .378980 61.890228 .408620 62.030780 .675539
8 25 97.881584 1.006563 97.187248 .970719 97.435424 .769749
16 25 213.292764 2.243355 214.196628 1.730783 213.708064 2.131265
32 25 550.210111 5.554267 549.394357 4.505693 550.379628 4.366301
64 25 1604.462695 8.824898 1608.118051 8.058784 1604.473598 9.459760
t8m commented 5 months ago

@Sashan I think you must have some data from other runs in the 1.1.1 builds for evp_fetch and providerdoall tests as these cannot be built with 1.1.1. They exercise API calls that are not there.

t8m commented 5 months ago

The numbers also look too similar between the branches, so it might be that your script just measured the builds against the same library version 3 times.

nhorman commented 5 months ago

lets not reopen this, if it has to be redone, lets open a new issue for it:

New issue https://github.com/openssl/project/issues/669