Performance of OQS algorithms in TLS

[ibmo96]

I have setup an Nginx server running OQS-OpenSSL and with a certificate signed with dilithium2. I was running some performance tests to see the difference between a classical TLS sessions and post-quantum TLS session using the built in s_time client in the OQS-OpenSSL library. I connected to the server like so:

for OQS TLS session: /opt/openssl/apps/openssl s_time -connect host:port1 -curves kyber512

for classical TLS session: ' /opt/openssl/apps/openssl s_time -connect host:port2

Where port1 uses the dilithium2 signed certificate and port2 uses a classical RSA signed certificate.

These are the results when running s_time for 30 seconds per run:

These are the results from doing a localhost test where I start an openssl server like so:

/opt/openssl/apps/openssl s_server -cert <SIG_ALG>_srv.crt -key <SIG_ALG>_srv.key -www -tls1_3 -groups kyber512

I realize that localhost testing is a little more reliable in terms of network interference and such, but i do not really understand the 50-60% dip in performance seen with the Nginx performance testing vs the 20-25% drop in performance with the localhost testing.

I am running the Nginx server from within a DigitalOcean droplet.

[baentsch]

Hmm -- is it really necessary to run this within a VM rather than a local laptop? That already may cause "unexplicable" differences. Then, a localhost connection obviously optimizes data transport to the max, i.e., protocols with more packets get penalized less (as compared to transporting all packets over the wire): What about determining the amount of data and packets that gets transferred for each setup to better understand the differences? Maybe look at exchanges using wireshark? That also may provide more insight. Final comment: Did you already check out https://openquantumsafe.org/research/PQCrypto-PaqSteTam20 ?

[ibmo96]

Yea so i ran a tcpdump /wireshark on the droplet and turns out it was recieving a lot of packets from my dorms network aswell as SSH (i guess trying to see if im still logged in to the VM and the dorms network). It was on the network interface eth1, but on eth0 nothing was going on. So wondering if i could make s_time use eth0 when recieving packets from my Nginx server, giving me a less noisy network interface hmm..

In regards to the referenced paper on benchmarking TLS handshakes, i was interested in the emulated network framework veth but unsure as to how to apply that to outgoing traffic outside the given machine (non-localhost).

[baentsch]

@ibmo96 are the issues you were pondering above still open? Anything concrete we could help with? I personally wonder whether it might not be better to do "real" testing on a dedicated network (e.g., using a cloud VPN or possibly better, comprising two notebooks with a wire between them) to arrive at "reproducible" or numbers explicable by the oqs/openssl code and no other network traffic. @xvzcf might have input regarding your final question: Again, personally I have no experience with veth (let alone how to use it with "real" outgoing traffic).

[xvzcf]

In regards to the referenced paper on benchmarking TLS handshakes, i was interested in the emulated network framework veth but unsure as to how to apply that to outgoing traffic outside the given machine (non-localhost).

If you want to just apply packet loss and delay rules to outgoing traffic, this page might help.

[baentsch]

@ibmo96 May I ask how your thesis turned out? Is it online somewhere? Either way I assume we can close this issue (?)

[ibmo96]

@baentsch, hi sorry for the lack of communication, took a little post graduation break :D! It turned ok quite well, although that the critisicm was that this tool might not have been the most desired thing by industry right now, rather a tool can analyze an IT infrastructure and output what cryptographic algorithms are used and where (fancy). I can send you the thesis if you'd like.

[baentsch]

although that the critisicm was that this tool might not have been the most desired thing by industry right now, rather a tool can analyze an IT infrastructure and output what cryptographic algorithms are used and where (fancy)

Yup -- I also heard about that before. Looks like "industry" doesn't know where it put cryptography software to begin with: Buzzword "crypto agility": A nice way to say "how do we now deal with the problem that we put speed of deployment over thoroughness / understanding"? :-) If you could put a link to your thesis here, that would be great: Thanks & all the best for your next professional steps!

[ibmo96]

Link to the thesis: https://we.tl/t-kB5ehIdV9X

[dstebila]

https://we.tl/t-kB5ehIdV9X

Thanks, I've added a reference to it to https://openquantumsafe.org/research/