search

kzemek / etls

An alternative NIF-based implementation of Erlang ssl module.
Other
36 stars 2 forks source link

Load test performance might not be relevant #8

Open silviucpp opened 7 years ago

silviucpp commented 7 years ago

Hello,

Because etls doesn't support the cipher property where you can limit a client to a specific cipher the load test you did might not be accurate. You can't know if on both stacks erlang and etls was negotiated the same cipher. And some of them can be more cpu intensive and other less.

Silviu

kzemek commented 7 years ago

True, not setting the same cipher was a glaring omission by me, thanks for catching that!

I've ran benchmarks on my laptop using ECDHE-RSA-AES128-GCM-SHA256 with both ssl and etls and got ~102 MB/s and ~539 MB/s respectively. I want to stress that this was a quick benchmark while doing other things on the machine.

I will follow up with proper benchmarks on my main machine, but I'll first have to do some tweaking with the code (to allow setting ciphers) and the benchmark script. Hopefully I'll also update #4 then.

silviucpp commented 7 years ago

Hi,

As I posted on another thread where peoples start discussing about ssl vs other tls implementations:

I started working on a project to compare the erlang tls libs (https://github.com/silviucpp/tls_bench). Project is still in development so might have bugs. any feedback and help is appreciated.

Idea is that I found few problems that make me believes that etls benchmark is not realistic: kzemek/etls#8 and beside this also the send and receive buffers might not be equals. Also in elrang in case you don't set the sndbuf and recbuf the buffer doesn't take the value of max(sndbuf, recbuf) as it's recommended in documentation.

Using my benchmark with 50 concurrent connections sending 80000, messages of 30 KB each result in :

gen_tcp throughput: 3.77 GB/s (added to see the overhead of tcp) ssl throughput: 320.84 MB/s p1_tls throughput: 232.20 MB/s fast_tls throughput: 234.56 MB/s etls throughput: 491.71 MB/s

But again I'm still working to improve the benchmark and tuning and why not in the end try to compile p1_tls, fast_tls and erlang with boring ssl instead openssl.

Hope this helps you. Also I will be happy to receive feedback from your side on my benchmark solution.

Silviu

silviucpp commented 7 years ago

Oh I forgot to mention, I got almost the same results as you for ssl, but after II tuned the erlang params It jumped to almost 320 MB/s.

I suspected that something is wrong because without tuning also gen_tcp performances was very bad (almost identical with ssl) :)