t3hmrman commented 2 years ago

Describe the bug

Maddy is reporting a MAIL FROM address as having SPF insecurely set (allowing "all"?) when it does not.

Note that other emails get through just fine (ex. GMail, ProtonMail).

Steps to reproduce

The incoming mail path looks like this:

graph LR;
  SES-->Haraka;
  Haraka-->Maddy;

I'm trying to send an email from sender.tld to recipient.tld, with sender.tld being the MAIL FROM address on SES.

The email gets all th way to maddy, but maddy fails the SPF due to "matched all" (see logs). I assume this means that the SPF policy was determined to match all, but when I check my DNS I'm 100% sure it doesn't and never has. The record is:

v=spf1 include:amazonses.com -all

I've used the google toolbox to check the record so it's not just me either. In fact I'm fairly sure this rule is inserted by AWS SES automatically when you create a MAIL FROM alias because I don't see it in my infra code (and if I try to manually add one it collides)...

(Insecure) Workaround

(Don't try this at home folks!)

So of course, to test my hypothesis I've set fail_action ignore under spf in the local routing check stanza and the email gets through.

Log files

2022-08-31T13:22:22.061Z smtp: incoming message {"msg_id":"b7f1f921","sender":"01010182f4112163-3cf399e1-2b65-476b-9647-256c9dce24d3-000000@bounce.sender.tld","src_host":"haraka-bfzsx","src_ip":"10.244.206.187:58284"}
2022-08-31T13:22:22.075Z smtp: RCPT ok  {"msg_id":"b7f1f921","rcpt":"admin@recipient.tld"}
2022-08-31T13:22:22.206Z smtp/pipeline: quarantined     {"check":"check.spf","msg_id":"b7f1f921","reason":"matched all","smtp_code":550,"smtp_enchcode":"5.7.23","smtp_msg":"SPF authentication failed"}
2022-08-31T13:22:22.243Z smtp: accepted {"msg_id":"b7f1f921"}

After doing the workaround:

2022-08-31T13:37:25.426Z smtp: incoming message {"msg_id":"df3f0b88","sender":"01010182f41ee88d-051c24d3-69a1-43f9-95f4-f9f7b994610b-000000@bounce.sender.tld","src_host":"haraka-6wxhv","src_ip":"10.244.192.190:50182"}
2022-08-31T13:37:25.448Z smtp: RCPT ok  {"msg_id":"df3f0b88","rcpt":"admin@recipient.tld"}
2022-08-31T13:37:25.517Z smtp/pipeline: no check action {"check":"check.spf","msg_id":"df3f0b88","reason":"matched all","smtp_code":550,"smtp_enchcode":"5.7.23","smtp_msg":"SPF authentication failed"}
2022-08-31T13:37:25.631Z smtp: accepted {"msg_id":"df3f0b88"}

Configuration file

    $(hostname) = mail.recipient.tld
    $(primary_domain) = recipient.tld
    $(local_domains) = $(primary_domain)

    state_dir /data
    log stderr_ts

    tls file /data/tls/tls.crt /data/tls/tls.key {
      protocols tls1.2 tls1.3
    }

    auth.pass_table local_authdb {
      table sql_table {
        driver sqlite3
        dsn credentials.db
        table_name password
      }
    }

    storage.imapsql local_mailboxes {
      driver sqlite3
      dsn imapsql.db
    }

    # SMTP
    hostname $(hostname)

    msgpipeline local_routing {
      destination postmaster $(local_domains) {
        modify {
          # Allow + aliases
          replace_rcpt regexp "(.+)\+(.+)@(.+)" "$1@$3"
          # Allow . aliases
          replace_rcpt regexp "(.+)\.(.+)@(.+)" "$1@$3"
          replace_rcpt file /etc/maddy/aliases
        }

        deliver_to &local_mailboxes
      }

      default_destination {
        reject 550 5.1.1 "No such user"
      }
    }

    smtp tcp://0.0.0.0:2525 {
      limits {
        all rate 100 1s
      }

      check {
        dnsbl
        require_mx_record
        dkim
        spf {
          softfail_action ignore
        }
      }

      source $(local_domains) {
        reject 501 5.1.8 "use submission for outgoing SMTP"
      }

      default_source {
        destination postmaster $(local_domains) {
          deliver_to &local_routing
        }

        default_destination {
          reject 550 5.1.1 "No such user"
        }
      }
    }

    submission tcp://0.0.0.0:587 tls://0.0.0.0:465 {
      limits {
        all rate 50 1s
      }

      auth &local_authdb

      source $(local_domains) {
        destination postmaster $(local_domains) {
          deliver_to &local_routing
        }

        default_destination {
          modify {
            dkim $(primary_domain) $(local_domains) mail
          }

          deliver_to &remote_queue
        }
      }

      default_source {
        reject 501 5.1.8 "Non-local sender domain"
      }
    }

    # Remote delivery
    target.remote outbound_delivery {
      limits {
        destination rate 20 1s
        destination concurrency 10
      }

      mx_auth {
        dane
        mtasts {
          cache fs
          fs_dir mtasts_cache/
        }
        local_policy {
          min_tls_level encrypted
          min_mx_level none
        }
      }
    }

    target.queue remote_queue {
      target &outbound_delivery

      autogenerated_msg_domain $(primary_domain)
      bounce {
        destination postmaster $(local_domains) {
          deliver_to &local_routing
        }
        default_destination {
          reject 550 5.0.0 "Refusing to send DSNs to non-local addreses"
        }
      }
    }

    # IMAP
    imap tcp://0.0.0.0:143 tls://0.0.0.0:993 {
      auth &local_authdb
      storage &local_mailboxes
    }

    openmetrics tcp:0.0.0.0:9749 { }

Environment information

maddy version: 0.6.2

foxcpp commented 2 years ago

Note the src_ip in maddy log being 10.244.206.187. That's probably the internal network's address of your Haraka server. maddy is trying to match that against sender.tld SPF record which fails.

In order to correctly check SPF (and also DMARC, by the way), maddy needs to know the actual IP address of the sender. maddy currently does not support any protocol extensions (XCLIENT or PROXY) that would help with this. Neither Haraka, it seems (it supports PROXY protocol server-side only, that is, accepting proxies connections only).

t3hmrman commented 2 years ago

Yep -- so this is running inside a Kubernetes cluster, and PROXY support is provided for the ingress controller.

Haraka will at least accept the PROXY protocol. For actually passing through the connection, I don't think it forwards the PROXY information....

Is there any way to use an external address?

foxcpp / maddy

SPF reported as failing due to allowing all when it is disallowed #530