Supporting mTLS (mutual authentication) on the listener so that we may ask the client to prove who it is

[berrfred]

Today when using SSL on the listener we provide a server certificate so that the client can verify and trust the server. With https://github.com/grobian/carbon-c-relay/issues/442 you have further enhanced the security by offering the options to control TLS version (protomin, protomax) and also weak ciphers can be disabled (ciphers, ciphersuites).

However there is no mechanism (other than FW settings) to verify and restrict client connections so I believe it would make sense to introduce mTLS so that the listener can be configured to ask the client to provide a certificate (Certificate Request with list of supported CAs) and then the server would verify the certificate validity before allowing the connection.

[berrfred]

Could it be something like the following piece of code to integrate with a corresponding configuration parameter ?

 SSL_CTX_set_client_CA_list(ctx, SSL_load_client_CA_file(CAfile));
 SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);

[grobian]

thanks! I'll have a look

[grobian]

I think for this we need on the server side a set_verify with: SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT | SSL_VERIFY_CLIENT_ONCE

I think the client side has to use SSL_CTX_use_certificate and SSL_CTX_use_PrivateKey

[berrfred]

Yeap, I think you are right, this is the most complete and optimal combination on server side. Had not thought about the client side since I currently do not use it with SSL inside our backend but it makes sense to also manage it there.

[grobian]

I'm kind of getting delayed here, I started on this change but got lost at some point on where what code needs to go. My objective would be to allow two carbon-c-relays to have an mTLS connection

[berrfred]

Not sure where you get confused ... in my mind there are two aspects:

1) incoming ssl connections where we have to ask the client for a certificate signed by a given CA (or list of CA) and verify the existence and validity of the provided certificate ... client can either be another relay instance or any client sending metrics and this is probably the most useful situation since I may be sending across the Internet.

2) outgoing ssl connections if I want to secure also this connection towards another relay instance that would be configured to request mTLS (see point 1), then I have to provide a certificate if asked by the server.

[grobian]

I think I got it right, but I'm not entirely sure. Please reopen if anything seems off (or it's just plain disfunctional)

[berrfred]

Wanted to test the mTLS, first on the incoming connections (listen), but I am a bit confused on how to configure it.

So far I have been using "ssl pemcert" where pemcert is a file holding the private key and certificate of my carbon-c-relay server.

  type linemode transport plain
    ssl /etc/pki/tls/private/tgds_2048_cert_key.pem
      protomin tls1.2 ciphers AES256-GCM-SHA384:AES256-SHA256:AES128-GCM-SHA256
    8443 proto tcp
  ;

How should I set that I want mTLS and where should I put my Certificate Authority to validate the client certificate ? As soon as I try replacing ssl with mtls it gives me a syntax error, unexpected string, expecting 'proto'.

[berrfred]

Even more confused now that I realize there is/was a line argument -C CACertPath ... but even if set to a file containing a single CA, my python client has no problem connecting even if I use a version that does not send any client certificate.

Hereafter the read configuration as logged:

[2022-04-17 18:39:58] (MSG) starting carbon-c-relay v3.7.4 (2022-02-13), pid=25741
configuration:
    relay hostname = linux-hpg-fred
    workers = 8
    send batch size = 2500
    server queue size = 25000
    server max stalls = 4
    listen backlog = 32
    tls/ssl CA = /etc/pki/tls/clients/TI_Trust_Technologies_Private_CA_2013_2033.crt
    server connection IO timeout = 600ms
    idle connections disconnect timeout = 10m
    configuration = /etc/carbon-c-relay.conf

parsed configuration follows:
listen
    type linemode ssl /etc/pki/tls/private/tgds_2048_cert_key.pem protomin tls1.2 ciphers AES256-GCM-SHA384:AES256-SHA256:AES128-GCM-SHA256
        8443 proto tcp
    ;

[grobian]

Ok, this is the two sides I was talking about, haha. (or not ... bleh :( )

The -C argument is used for verification purposes, either to verify the server (as we'd do before mTLS), or to verify the cert from the client (mTLS case).

Whether mTLS is in effect, is determined by mtls instead of ssl. So when used on a cluster, it means that as a client, the pemcert and pemkey given via the config file are returned when the server asks for it.

For a listen directive, the mtls will ensure the server requests the client to send its certificate.

[grobian]

ok, there's certainly some bits missing from the initial commit, working on that now

[grobian]

I can't seem to get the certificates right, or the code isn't checking as expected

[grobian]

think I finally got it, missed the CA on the server to verify the client end...