Support use of faster TLS module

[heri16]

Support usage of hardware-accelerated async TLS module like Etls. Etls also uses BoringSSL, which is practically used by Google in Chrome and throughout backend servers.

Benchmarks: https://github.com/kzemek/etls

The main (and very important) benefit of using this module instead of Erlang's built-in ssl is hardware acceleration. etls module achieves an order of magnitude higher bandwidth when encoding/decoding data.

https://hex.pm/packages/etls

[benoitc]

Sounds like a good idea, there is also fast_tls from Processone.

How do you think it could be set though? A request option?

[heri16]

I think it is alright as long as it is as simple as setting the default pool. Fast_tls is great, but is it an async NIFs? Is it true that NIFs that are not async affect other processes on the BEAM scheduler?

On Monday, 11 July 2016, Benoit Chesneau notifications@github.com wrote:

Sounds like a good idea, there is also fast_tls https://github.com/processone/fast_tls from Processone.

How do you think it could be set though? A request option?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/benoitc/hackney/issues/332#issuecomment-231600083, or mute the thread https://github.com/notifications/unsubscribe/AAgK_cj-8GqgsU12L4ItZ77yPi9uHd6Kks5qUSmWgaJpZM4JIzJw .

[jlouis]

fast_tls is not a NIF but an Erlang driver. If it is written correctly, it shouldn't affect the schedulers at all.

[deadtrickster]

Hey!

Quote from BoringSSL page

Although BoringSSL is an open source project, 
it is not intended for general use, as OpenSSL is. 
We don't recommend that third parties depend upon it. 
Doing so is likely to be frustrating because there are no 
guarantees of API or ABI stability.

Moreover etls author basically just maintains his own BoringSSL copy (why not git submodule?). Not sure this is right thing to do and should be trusted.

So at least please leave standard ssl as default :-).

PS It is very interesting why such huge performance differences are observed (e.g. all computation intensive code is still from OpenSSL through crypto and crypto moves to hardware acceleration as well (see OTP-12217))

[heri16]

Standard SSL will be left as default. This is provided as a configuration for those that require a 10x increase in performance. The disclaimer on that you quoted from Google is outdated. These days the ABI is stable and many other projects use it for modern encryption. There is a good reason why Google forked OpenSSL, which is full of unreviewed code written by many unknown third parties all over the world, some sponsored by spy agencies. The code review process is tighter within Google security researchers.

On Saturday, 23 July 2016, Ilya Khaprov notifications@github.com wrote:

Hey!

Quote from BoringSSL page https://boringssl.googlesource.com/boringssl/

Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.

Moreover etls author basically just maintains his own BoringSSL copy https://github.com/kzemek/etls/tree/a5939b277e5ce705e397a4153a5f0d5248a9b277/c_src/deps/boringssl (why not git submodule?). Not sure this is right thing to do and should be trusted.

So at least please leave standard ssl as default :-).

PS It is very interesting why such huge performance differences are observed (e.g. all computation intensive code is still from OpenSSL through crypto and crypto moves to hardware acceleration as well (see OTP-12217))

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/benoitc/hackney/issues/332#issuecomment-234668056, or mute the thread https://github.com/notifications/unsubscribe-auth/AAgK_f4VLrU18AzyVIiv2YzOAal70VXuks5qYT4GgaJpZM4JIzJw .

[deadtrickster]

@benoitc May I ask you to not introduce this feature at all?

Let me explain with an example: Someone builds API wrapper library and wants more performance so he decides to use etls for example. Now each user of the library is forced to use etls as well. Moreover most of them I suspect will be simply unaware they use non standard ssl implementation which brings a lot more issues with security fixes and support. SSL stack is not something that can be 'implicit'.

Personally I'm ok with having 3 or 4 json libs in deps, but having whole different ssl stacks (maybe selected on throughput basis only by anonymous) is too much imo.

[deadtrickster]

On other hand ssl implementation for hackney could be set globally (just like fuse metric plugins). So if one wants to use custom ssl stack he explicitly sets respective hackney environment entry and all requests form all libs that use hackney will go through that ssl stack. This however will require 3d party ssl stacks to implement 'native' ssl app interfaces, including filing public_key structures.

[silviucpp]

Hello,

Starting from the fact that you should believe only your own benchmarks I started working on a project to compare the erlang tls libs (https://github.com/silviucpp/tls_bench). Project is still in development so might have bugs. any feedback and help is appreciated.

Idea is that I found few problems that make me believes that etls benchmark is not realistic: https://github.com/kzemek/etls/issues/8 and beside this also the send and receive buffers might not be equals. Also in elrang in case you don't set the sndbuf and recbuf the buffer doesn't take the value of max(sndbuf, recbuf) as it's recommended in documentation.

Using my benchmark with 50 concurrent connections sending 80000, messages of 30 KB each result in :

gen_tcp throughput: 3.77 GB/s (added to see the overhead of tcp) ssl throughput: 320.84 MB/s p1_tls throughput: 232.20 MB/s fast_tls throughput: 234.56 MB/s etls throughput: 491.71 MB/s

But again I'm still working to improve the benchmark and tuning and why not in the end try to compile p1_tls, fast_tls and erlang with boring ssl instead openssl.

Silviu