search

marlam / mpop

POP3 client
https://marlam.de/mpop
GNU General Public License v3.0
13 stars 1 forks source link

feature: set domain to verify TLS manually #2

Closed ilf closed 4 years ago

ilf commented 4 years ago

In some cases, it is desirable to verify a TLS certificate against a hostname that is not the FQDN of the certificates Common or Alt Name.

This could be an IP-address (IPs of mailservers rarely change, no point to check for a new one every few minutes via an unencrypted and unsigned protocol like DNS) - or a .onion address as Tor Onion Service.

It would be awesome to allow this in mpop (and msmtp). In order to use an IP or onion as $host combined with $tls_trust_file, I would propose to add something like a $tls_verify_override_host setting which is verified against the hostname in the certificate.

An example:

account mail@example.org
   host 192.0.2.0
   tls_trust_file /etc/ca-certificates/extracted/cadir/DST_Root_CA_X3.pem
   tls_verify_override_host example.org

This isn't completely new, f.e. unbound does something like this for DNS-over-TLS:

forward-addr: 1.1.1.1#cloudflare-dns.com

Thanks, and keep up the good work!

marlam commented 4 years ago

For such corner cases you can use the existing tls_fingerprint.

ilf commented 4 years ago

Thanks. I am using tls_fingerprint extensively, but it achieves a different goal. It pins a single certificate to a single host. (From the manual: "Set the fingerprint of a single certificate to accept for TLS.")

That is great, but limited. It can be desirable to use the "standard X.509" verification - with a manual hostname.

  1. Certificate validity periods are getting ever shorter, Let's Encrypt currently has 90 days. Every time the certificate is renewed, the fingerprint changes and I have to re-set tls_fingerprint. This feature would keep the same setting even over certificate renewals - as long as hostname and $tls_trust_file stay the same.

  2. Some mail-providers have multiple servers for the same service, and sometimes each of these servers has their own certificate. But pinning only works with one cert. This feature would work with multiple certs. (This is a real-world use-case I actually currenty have.)

marlam commented 4 years ago

Yes, tls_fingerprint is limited. But I would argue that a TLS setup where the host name does not match the server certificate is simply broken (i.e. it violates the TLS specification), and therefore not worth implementing a special workaround for.

ilf commented 4 years ago

I'm not sure I understand. Using tls_fingerprint also ignores the hostname in the server certificate (from the manual: "This certificate will be trusted regardless of its contents (this overrides tls_trust_file)."

Also, the TLS specification doesn't say anything about hostnames.

I'd be interested: How would you verify the certificate when using an IP-address or an onion-address, when fingerprint pinning is not viable?

I have shown that other software uses the proposal I made.

marlam commented 4 years ago

When you connect to a TLS server, you have to check that 1) you trust the certificate and 2) you really are connected to the server you intended to connect to. 2) typically means that you check the host name against the names in the certificate (RFC 6125). If it is not in there, you cannot trust the connection. I argue that a server that requires a workaround in this regard is misconfigured.

ilf commented 4 years ago

Everything you're saying is true, and known to me.

I am not trying to use a misconfigured server. My server mail.example.org offers a valid certificate with its domain mail.example.org in Common Name (or Alt Name). This works fine:

account test@example.org
        host mail.example.org
        tls_trust_file /etc/ca-certificates/extracted/cadir/DST_Root_CA_X3.pem

But in my client (mpop), I don't want to set host mail.example.org. I would like to set host 192.0.2.0 (because I don't want to use DNS every few minutes, it never changes anyways) or host maildotexampledotorglongtorhiddenserviceaddress.onion. And the TLS certificate doesn't include the IP or onion address (yet).

So far, I can either use pinning (but that only works for one server, and needs to be renewed and re-verified every 90 days), or disable TLS verification. Both are not desirable.

My proposal would do what everyone else does: verify the hostname the client knows against the hostname in the certificate. Only that the client doesn't get that value from "host", but from different variable.

I don't see how this would violate any standard. And, other software does it, too.

ilf commented 4 years ago

How would you verify a server with an onion-address?

marlam commented 4 years ago

Regarding the DNS lookup: you can use a cache on your local system to reduce DNS lookups.

Regarding IP adresses: while these can be in certificates, this is considered a corner case not worthy of much attention (RFC 6125).

Regarding onion adresses: i don't know about those. If they can be in certificates, then the problem is solved, is it not? And if they cannot be, there should be a standardized alternative, right?

Regarding other software: you mentioned unbound, but that use case is not for onion addresses; DNS-over-TLS really is a special case.

(My rationale for arguing in this way is that I want to avoid adding a feature and associated complexity for something only useful in corner cases for 1-3 users.)

ilf commented 4 years ago

I absolutely agree that unnecessary complexity should be avoided. That's why I'm using mpop/msmtp in the first place!

But I think that this proposal is simple and small. AFAICT, the hostname verification logic in tls.c can mostly remain as-is. The local hostname value continues to default to host, but if there is a $tls_verify_override_host, take it from there. That's it. I'd try to code it myself, but my C is not sufficient.

If we're talking about "complexity for corner cases", I can see a few other candidates like source_ip ("Useful only in special cases on multi-home systems.") or ntlmdomain ("This is obsolete." :)

meskio commented 4 years ago

I have the same problem here. It's hard (and expensive) to get a valid TLS certificate for an onion address, I don't know of any email provider doing that.

Usually the TLS certificate is for the original hostname of the provider. For example using riseup.net pop server with onion address zsolxunfmbfuq7wf.onion you will get a certificate for the domain pop.riseup.net.

Will be great to be able to override the domain that will be used to validate the certificate, or at least to disable the domain validation.

dkg commented 4 years ago

.onion addresses are only one of the circumstances where "typical" DNS just doesn't work to identify the peer.

As @ilf says, another situation is where there's a desire avoid DNS lookups (that can be addressed with an entry in /etc/hosts but that only works if you have administrative access).

Additionally, there are cases where a given host has multiple IP addresses backed by different servers, but the user has a specific reason to connect to one of those addresses specifically.

In other cases, the user is on a restrictive network and needs to resort to tunneling a connection to reach a machine on the public internet (e.g. via a forwarded ssh port).

It would be great if mpop-mirror could work in all those cases.

I can see three possible ways to offer a solutionfor all the cases described above:

Any of these choices could be misused of course, but they're at least as reasonable to support as existing options that fiddle with security-related parameters (like --tls-min-dh-prime-bits or --tls-cert-check=off) for the sake of allowing connectivity.

It would be a mistake to encourage --tls-fingerprint for this use case in the modern era of short-lived certificates, as it makes a brittle mpop configuration that needs frequent adjusting; each manual adjustment is an annoyance to the user, and an opportunity for a security failure.

marlam commented 4 years ago

OK, there are several new points in the comments:

If we're talking about "complexity for corner cases", I can see a few other candidates like source_ip ("Useful only in special cases on multi-home systems.") or ntlmdomain ("This is obsolete." :)

I agree that there are some obscure options in msmtp and mpop now, but that does not mean that adding even more is ok ;-)

As @ilf says, another situation is where there's a desire avoid DNS lookups (that can be addressed with an entry in /etc/hosts but that only works if you have administrative access).

Additionally, there are cases where a given host has multiple IP addresses backed by different servers, but the user has a specific reason to connect to one of those addresses specifically.

In other cases, the user is on a restrictive network and needs to resort to tunneling a connection to reach a machine on the public internet (e.g. via a forwarded ssh port).

It would be great if mpop-mirror could work in all those cases.

We already support SOCKS proxy, and there's stunnel. Is that not enough to support all these use cases? For example, stunnel has the checkHost option which looks like it does what the suggested tls_override_verify_host option would do.

It's hard (and expensive) to get a valid TLS certificate for an onion address, I don't know of any email provider doing that.

Usually the TLS certificate is for the original hostname of the provider. For example using riseup.net pop server with onion address zsolxunfmbfuq7wf.onion you will get a certificate for the domain pop.riseup.net.

Isn't that the problem that should be solved? Convince Let's Encrypt (and/or others) to add support for onion addresses, and all software will just work. In contrast, adding workarounds to every single piece of software just because the correct solution is currently impracticable seems wrong, doesn't it?

marlam commented 4 years ago

Regarding the possibility to use basically any external program to set up the network connection, optionally including TLS, there is one thing that could help: support for UNIX domain sockets. These can be used e.g. with ssh and stunnel and offer file system level access restrictions (in contrast to SOCKS proxies or forwarded TCP ports). A request to add a socket configuration command has come up in different contexts, and I am beginning to think that it is worth adding as there is no other way to implement some special configurations. I will push an initial version soon. Maybe this helps your situations, too?

ilf commented 4 years ago

I am not sure I fully understand the proposal. Can you provide a sample config how you imagine this setup with onion and TLS verification?

There might be other potential use-cases for a socket configuration. But for this problem the solutions @dkg proposed in https://github.com/marlam/mpop-mirror/issues/2#issuecomment-606146565 sound much simpler to me. And I like the KISS principle.

But maybe I'm missing something and a proposed config can help me understand.

marlam commented 4 years ago

TLS setup and verification would be handled by stunnel, including setting a custom host name for verification. Then msmtp/mpop can either connect to a TCP port provided by stunnel (to which other users can also connect), or to a local socket provided by stunnel (which can be subject to access restrictions). So the socket option just makes this kind of thing more manageable, but not really simpler, since you still have two programs running. I understand the simplest option from your point of view is to have a tls_verify_host option. But why are onion addresses not in server certificates? I still feel this is the problem that should be addressed instead of changing all TLS client software to provide a workaround.

ilf commented 4 years ago

Let's Encrypt is working on support for onions, but it's not there yet.

But this is not just about onions. @dkg has provided three more use-cases in https://github.com/marlam/mpop-mirror/issues/2#issuecomment-606146565

marlam commented 4 years ago

I'm still not entirely convinced that any of these cases are really useful, but ok, I give up ;) I added tls_host_override and --tls-host-override to both msmtp and mpop.

ilf commented 4 years ago

Thank you very much for this! And https://github.com/marlam/mpop-mirror/commit/8b17f6b20196a281be47d8f129df7f428eb4cb2a does look rather simple. :)

I look forward to using this. But I'll wait for a release.

ilf commented 4 years ago

Works like a charm. Thanks a lot! <3