openssl / perftools

Performance testing tools
Apache License 2.0
4 stars 4 forks source link

Add perf test for EVP_PKEY_derive_set_peer #4

Closed andrewkdinh closed 3 months ago

andrewkdinh commented 3 months ago

Adds a performance test profiling the run time of EVP_PKEY_derive_set_peer. This test is very similar to how pkeyread perf test is set up and run.

I created a new source/genkeys_setpeer.sh file that can be used to codegen a keys_setpeer.h header file with private keys for the test to use. The test will go through the key types (specified as a CLI option), runs EVP_PKEY_derive_set_peer 10k times split between however many threads is specified, and prints the results for each key type.

For now, it measures the performance on DH ffdhe2048, ECDH P-256, P-521, and X25519.

OpenSSL master branch:

$ ./evp_setpeer -k all 2
Average time per dh evp_set_peer call: 4191.861000us
Average time per ec256 evp_set_peer call: 125.292000us
Average time per ec521 evp_set_peer call: 760.357000us
Average time per x25519 evp_set_peer call: 0.957000us
$ ./evp_setpeer -k all -t 2
[dh] 4262.306000us
[ec256] 125.517000us
[ec521] 753.860000us
[x25519] 1.373000us
$ ./evp_setpeer -k x25519 -t 2
[x25519] 1.439600us

OpenSSL 1.1.1:

$ ./evp_setpeer -k all -t 2
[dh] 0.394000us
[ec256] 2.347000us
[ec521] 3.217000us
[x25519] 0.103000us

From some initial test results, it looks like DH is much slower than other key types. Also, OpenSSL 1.1.1 is much faster than OpenSSL 3.3.3, probably because it doesn't validate the peer as suggested in https://github.com/openssl/openssl/issues/21833.

Resolves issue: https://github.com/openssl/project/issues/646 Original PR: https://github.com/openssl/tools/pull/211

t8m commented 3 months ago

Did you test with an up to date master branch? I.e. one with e70e34d included?

andrewkdinh commented 3 months ago

@t8m I tested it on master branch from about a week ago, so let me test again with ToT master and post the results

andrewkdinh commented 3 months ago

@t8m I tested the results on ToT main, and these are the results. Looks about the same

$ ./evp_setpeer -k all -t 2
[dh] 4254.132400us
[ec256] 123.962700us
[ec521] 758.470000us
[x25519] 1.320900us
t8m commented 3 months ago

I believe you are actually using a system libcrypto or something else. The results do not correspond with what I see:

openssl-3.3 branch:

$ ./evp_setpeer -k all 1
Average time per dh evp_set_peer call: 1526.177200us
Average time per ec256 evp_set_peer call: 43.988700us
Average time per ec521 evp_set_peer call: 233.551700us
Average time per x25519 evp_set_peer call: 0.334500us

master branch:

$ ./evp_setpeer -k all 1
Average time per dh evp_set_peer call: 1.787400us
Average time per ec256 evp_set_peer call: 6.231100us
Average time per ec521 evp_set_peer call: 15.156100us
Average time per x25519 evp_set_peer call: 0.694100us

These numbers on master reflect the changes of the DH and ECDH key validation to use the quick test for parameters where it is sufficient (i.e., known safe primes such as the FFDHE primes and EC curves with cofactor 1).

So yeah, this call is not free as on 1.1.1 but IMO acceptable.

t8m commented 3 months ago

Anyway, approved as the tool code is OK.

t8m commented 3 months ago

@andrewkdinh I strongly suggest uncommenting this Makefile line https://github.com/openssl/perftools/blob/main/source/Makefile#L14 when you are building perftools against local builds of openssl.

andrewkdinh commented 3 months ago

@t8m Ah ok, yes it looks like it was because I missed uncommenting that line, and my results look more similar to your results now. Will make sure to do that for future perf results I gather. Thanks!

nhorman commented 3 months ago

merged, thank you!