danielealbano / cachegrand

cachegrand - a modern data ingestion, processing and serving platform built for today's hardware
BSD 3-Clause "New" or "Revised" License
975 stars 34 forks source link

Add TLS support #155

Closed danielealbano closed 2 years ago

danielealbano commented 2 years ago

This PR contains a number of changes, the vast majority TLS related, although there are some minor fixes for the tests, some general code coverage improvement, some bug fixes for the worker statistics, etc.

The TLS support has been implemented via mbedTLS 2 and TLS is supported up to TLS 1.2, although TLS 1.3 should work it hasn't been fully tested. OpenSSL has been dropped from the build, although it's still needed for sentry, and some code used for the benchmarks (or to better say unused) that was using its big number implementation has been commented out because can't really be ported to openssl and needs rewriting.

The PR also contains the necessary changes to enable the kernel offload (kTLS) if the chosen cipher is supported, this considerably speed up the operations, especially if the network card supports tls offloading (e.g. mellanox connectx 6) or hardware encryption/decryption is supported.

3 new statistics have been introduced:

A new command line option has been introduced to make easy to enumerate the supported ciphers, it also indicates if kTLS is supported, e.g.

Available TLS cipher suites:

+-----------------------------------------------+-------------+-------------+------------+
| Cipher Suite                                  | Min Version | Max Version | Offloading |
+-----------------------------------------------+-------------+-------------+------------+
| TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256   |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-ECDHE-ECDSA-WITH-CHACHA20-POLY1305-SHA256 |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-DHE-RSA-WITH-CHACHA20-POLY1305-SHA256     |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384       |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384         |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-DHE-RSA-WITH-AES-256-GCM-SHA384           |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-ECDHE-ECDSA-WITH-AES-256-CCM              |     TLS 1.2 |     TLS 1.2 |            |
...
| TLS-PSK-WITH-CAMELLIA-256-CBC-SHA384          |     TLS 1.0 |     TLS 1.2 |            |
| TLS-PSK-WITH-AES-256-CCM-8                    |     TLS 1.2 |     TLS 1.2 |            |
| TLS-PSK-WITH-AES-128-GCM-SHA256               |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-PSK-WITH-AES-128-CCM                      |     TLS 1.2 |     TLS 1.2 |       kTLS |
| TLS-PSK-WITH-AES-128-CBC-SHA256               |     TLS 1.0 |     TLS 1.2 |            |
| TLS-PSK-WITH-AES-128-CBC-SHA                  |     TLS 1.0 |     TLS 1.2 |            |
| TLS-PSK-WITH-CAMELLIA-128-GCM-SHA256          |     TLS 1.2 |     TLS 1.2 |            |
| TLS-PSK-WITH-CAMELLIA-128-CBC-SHA256          |     TLS 1.0 |     TLS 1.2 |            |
| TLS-PSK-WITH-AES-128-CCM-8                    |     TLS 1.2 |     TLS 1.2 |       kTLS |
+-----------------------------------------------+-------------+-------------+------------+

Code coverage is probably not great with this PR, will improve it in separated PRs.

Closes #152

codecov[bot] commented 2 years ago

Codecov Report

Merging #155 (0d65210) into main (64f0b88) will decrease coverage by 3.07%. The diff coverage is 51.90%.

@@            Coverage Diff             @@
##             main     #155      +/-   ##
==========================================
- Coverage   81.04%   77.97%   -3.08%     
==========================================
  Files          88       92       +4     
  Lines        5514     6130     +616     
==========================================
+ Hits         4469     4780     +311     
- Misses       1045     1350     +305     
Impacted Files Coverage Δ
src/config_cyaml_schema.c 100.00% <ø> (ø)
src/log/sink/log_sink_file.c 42.85% <0.00%> (ø)
src/network/io/network_io_common.c 94.07% <ø> (-0.08%) :arrow_down:
src/program.c 26.33% <0.00%> (-2.92%) :arrow_down:
src/program_arguments.c 92.98% <0.00%> (-5.17%) :arrow_down:
src/network/channel/network_channel_tls.c 1.24% <1.24%> (ø)
src/worker/network/worker_network_iouring_op.c 69.67% <18.51%> (-9.87%) :arrow_down:
src/worker/network/worker_network_op.c 70.96% <43.85%> (-15.18%) :arrow_down:
src/network/network_tls.c 72.22% <72.22%> (ø)
src/worker/worker.c 81.17% <81.81%> (-0.19%) :arrow_down:
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 64f0b88...0d65210. Read the comment docs.

lgtm-com[bot] commented 2 years ago

This pull request introduces 2 alerts when merging 18b725c6e58636658d12831e05c23a5908e06dac into 64f0b88b98f57175b56edb00c7b1e2fb8093d866 - view on LGTM.com

new alerts:

lgtm-com[bot] commented 2 years ago

This pull request introduces 1 alert when merging 0d65210e2e256a215cf217d5364456872fdbde39 into 64f0b88b98f57175b56edb00c7b1e2fb8093d866 - view on LGTM.com

new alerts: