Enable TLS for Outbound

Infern1 commented 7 years ago

Expected behavior

When their are certificates (valid ones, not self signed). Outbound messages should try to use TLS

Observed behavior

If you have a maildrop forward, mail is send via qmail, so you imho qmail should deliver via TLS as well

Have a working setup now which uses qmail-tls Testing further to make sure it works (atleast gmail is saying I use TLS)

Difference when sending to gmail

Received: from vps1.hostname.nl (smtp.hostname.nl. [149.210.213.XX])
        by mx.google.com with SMTP id s17si3495657wrs.262.2017.02.22.15.11.14
        for <r.j.xxxxx@gmail.com>;
        Wed, 22 Feb 2017 15:11:15 -0800 (PST)

With SSL

Received: from vps1.hostname.nl (smtp.hostname.nl. [149.210.213.XX])
        by mx.google.com with ESMTPS id v186si6527544wme.120.2017.03.01.03.04.02
        for <r.j.xxxxx@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Wed, 01 Mar 2017 03:04:02 -0800 (PST)

TODO

[ ] Install QMAIL-TLS instead of netqmail

[ ] Make sure their is certificate, either with letsencrypt or own Did it manual now:

cat /data/dovecot/etc/ssl/private/dovecot.pem > /data/vpopmail/qmail-control/servercert.pem
cat /data/dovecot/etc/ssl/certs/dovecot.pem >> /data/vpopmail/qmail-control/servercert.pem

[ ] Create symlink between servercert.pem and clientcert.pem
[ ] Make sure make /var/qmail/bin/update_tmprsadh is executed to generate certs

[ ] Create ciphers

openssl ciphers > /var/qmail/control/tlsclientciphers
openssl ciphers > /var/qmail/control/tlsserverciphers

[x] Fix Error when forwarding with TLS Enabled in smtp_forward.ini

[ERROR] [-] [core] backend failure: 127.0.0.8:25 - EB9903AA-A234-4A0E-9C64-328B4FEDA7A4: [127.0.0.8:25] SMTP connection errored Error: 34443650048:error:140770FC:SSL routines:SSL23_GET_SERVE
R_HELLO:unknown protocol:s23_clnt.c:794:

This can be fixed by Removing fixcrio, found here: https://www.tnpi.net/wiki/OpenSSL_errors%3F

my qmail-smtpd\run

exec    /usr/local/bin/tcpserver -H -R -c10 \
        -u 89 -g 82 -x /usr/local/vpopmail/etc/tcp.smtp.cdb 0.0.0.0 25 \
        /var/qmail/bin/qmail-smtpd /usr/local/vpopmail/bin/vchkpw /usr/bin/true \
        /var/qmail/bin/splogger qmail

Unknown error

Received: from roundcube (roundcube [127.0.0.33])
    by haraka (Haraka/2.8.13) with ESMTPSA id BD43F8FE-542E-45DF-A1E6-6C87DDD80658.1
    envelope-from <rob@hostname.nl> (authenticated bits=0)
    (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256 verify=FAIL);
    Wed, 01 Mar 2017 12:27:18 +0100

msimerson commented 7 years ago

There's a longstanding problem lurking there. When you remove fixcrio, you remove the ability to receive email from some (RFC violating, true, but they still might be messages you or your clients want to get) senders. I recall this because of rejected mails early in MT6 development, which is why fixcrio made it back in there.

There may be another way to solve this. Add a smtproute to qmail and push all remote traffic back to Haraka outbound (with IP relay permission). Not only can you get TLS outbound, but you can also DKIM sign the messages as they egress.

Infern1 commented 7 years ago

Using smtproutes sounds indeed like a beter idea, will give that a try

Infern1 commented 7 years ago

Seems smptroutes doesn't do the job added to smptroutes:

:haraka

But mail seems to be looping

<r.j.host@gmail.com>:
173.194.79.26 failed after I sent the message.
Remote host said: 554-5.6.0 Message exceeded 50 hops, this may indicate a mail loop. Learn more at
554 5.6.0  https://support.google.com/mail/?p=MailLoop z17si12094826wra.327 - gsmtp

msimerson commented 7 years ago

Did you set up relay permissions for qmail, so that Haraka sees those as outbound?

Infern1 commented 7 years ago

But what I still don't get

Sending a mail to for example gmail via smpt auth, why tls isn't working. This shouldn't be anything for qmail to deliver this mail?

Infern1 commented 7 years ago

Did you set up relay permissions for qmail, so that Haraka sees those as outbound?

Yes enabled the relay plugin and relay_acl

[relay]
acl=true

configured the host in /data/config/relay_acl_allow

msimerson commented 7 years ago

Right, b/c if you hadn't, Haraka would have rejected it. Silly Matt.

So, the problem is that currently, Haraka is configured to forward everything to qmail. The main reason I set it up that way is b/c Haraka doesn't know if a message is local or not. Haraka accepts messages that are local (inbound), but also accepts messages (from clients with relaying privs) that are local or destined elsewhere (outbound).

msimerson commented 7 years ago

Qmail does know if a message is local and it won't try SMTP delivery (and thus, smtproutes) unless the message is remote.

Infern1 commented 7 years ago

Is this fixcrio: https://support.plesk.com/hc/en-us/articles/213395829 not an option, we don't do SMTP/SSL with qmail?

msimerson commented 7 years ago

to prevent the mail loop, you need to add enable_outbound=false to smtp_forward.ini. Then Haraka will use MX routing to deliver the message.

msimerson commented 7 years ago

Haraka isn't all that clever WRT to local/remote vs inbound/outbound. If relaying privs are enabled, then Haraka considers it outbound.

Infern1 commented 7 years ago

to prevent the mail loop, you need to add enable_outbound=false to smtp_forward.ini.

This works now for TLS, however DKIM now fails

Received: from haraka (smtp.bsdfreaks.nl. [149.210.213.35])
        by mx.google.com with ESMTPS id l4si8900896wre.304.2017.03.02.13.58.45
        for <r.j.host@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 02 Mar 2017 13:58:46 -0800 (PST)
Received-SPF: pass (google.com: domain of rob@host.nl designates 149.210.213.35 as permitted sender) client-ip=149.210.213.35;
Authentication-Results: mx.google.com;
       dkim=neutral (body hash did not verify) header.i=@host.nl;
       spf=pass (google.com: domain of rob@host.nl designates 149.210.213.35 as permitted sender)

msimerson commented 7 years ago

That's interesting, b/c it worked for me:

Received: from mail.theartfarm.com (mail.theartfarm.com. [66.128.51.165])
        by mx.google.com with ESMTPS id s5si3912776oie.125.2017.03.02.13.52.04
        for <matt.simerson@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 02 Mar 2017 13:52:05 -0800 (PST)
Received-SPF: pass (google.com: domain of matt@tnpi.net designates 66.128.51.165 as permitted sender) client-ip=66.128.51.165;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@tnpi.net;
       spf=pass (google.com: domain of matt@tnpi.net designates 66.128.51.165 as permitted sender) smtp.mailfrom=matt@tnpi.net;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=tnpi.net

msimerson commented 7 years ago

Does DKIM normally verify for you?

Infern1 commented 7 years ago

Does DKIM normally verify for you?

Yes, tested sending via roundcube and this is DKIM verified... now test again and works... maybe did send empty message..

msimerson commented 7 years ago

Also, I've been playing with LMTP message delivery. Instead of delivering inbound messages to qmail (qmail-smtpd -> qmail-local -> vdelivermail -> (maybe) maildrop), Haraka could deliver them directly to dovecot via LMTP.

Infern1 commented 7 years ago

Reverted the smtproute method, internal mailing seems to give problems, not found out what is causing that problem

msimerson commented 7 years ago

internal mailing seems to give problems, not found out what is causing that problem

Does MX routing work from the perspective of Haraka outbound? IE, inside the Haraka jail, if you do a MX lookup of your internal domains, do you get the destination IP of your next hop (the vpopmail jail)? If not, you need to add some entries to the unbound local.conf file. From my server:

$ grep mail.thear /data/dns/unbound.conf.local
     local-data: "8.15.16.172.in-addr.arpa PTR mail.theartfarm.com"
     local-data: "mail.theartfarm.com A 172.16.15.8"
     local-data: "mail.theartfarm.com TXT 'v=spf1 a -all"

Infern1 commented 7 years ago

Didn't do anything to the unbound conf, so most likely this is the problem. Have to stop fiddling around for the moment, uses starting to complain...

Infern1 commented 7 years ago

Do you have it running in production now? Seeing the merge of #220 I assume there is some progress.

msimerson commented 7 years ago

I started to. But then I ran into some issues that need addressing. Think: email lists, and creating Haraka aliases to replace .qmail aliases. And then I ran out of time and haven't got back to it yet.

msimerson commented 7 years ago

Current Message Handling

Haraka
- recipient validated by:
  - if a split connection (has > 1 transaction AND different forward paths), queue to outbound
    - empirical observations measure very few of these (~1-2%)
  - smtp_forward - queue via smtp_forward
  - qmail-deliverable
    - smtp_forward for destination type any (mailbox, alias, forward, list)
      - qmail
        
        qmail-remote (smtproute or MX)
        
        qmail-local
        
        .qmail -> [ vdelivermail OR maildrop & mailfilter OR ezmlm ]
- relaying
  - recipient validation not required
  - split connections are more common here (some recipients local & some off-site)
  - route through outbound
    - smtp_forward has get_mx for next-hop domains in smtp-forward.ini
      - DNS MX lookups likely point to us, so outbound needs to know about these
Edge cases
- Forwards (.qmail files) -> off-host
  - no TLS b/c qmail doesn't have it
  - possible solution: qmail-tls
  - possible solution: migrate to Haraka aliases plugin.
  - in an increasingly AUTHed SMTP world, forwards are getting increasingly complicated (DMARC & ARC).
Future Ideas
- beef up Haraka's aliases plugin. Add postmaster@$domain authentication and then permit management of aliases (flat file and/or redis).
- qmail-deliverable gives us some visibility into the ultimate destination is (alias, dir, qmail-command in dot-qmail, etc.)
- [x] for types dir (aka: mailbox), deliver directly to dovecot LMTP (removes qmail entirely)
- migrate aliases to Haraka aliases plugin, then LMTP or outbound.
- migrate forwards by Haraka, expand during rcpt_to, then outbound.

msimerson commented 6 years ago

It has taken a while but Haraka outbound has all the egregious bugs exorcised. I'm now delivering messages as described above with a twist: for messages that QMD validated, if the destination type is a mailbox, it's delivered via dovecot LMTP.

msimerson / Mail-Toaster-6