search

foxcpp / maddy

✉️ Composable all-in-one mail server.
https://maddy.email
GNU General Public License v3.0
5k stars 240 forks source link

Maddy not accepting e-mail from tpexpress.co.uk due to `check.spf: result: permerror (unknown field)` #498

Closed reivilibre closed 2 years ago

reivilibre commented 2 years ago

Describe the bug

Since a few months ago, I don't receive e-mails (usually train tickets) from tpexpress.co.uk. I looked at the SPF records and couldn't immediately spot the error as being on their side These records look relevant to me:

tpexpress.co.uk:

v=spf1 a:ukbp-vdc-mail01.uk.atosorigin.com include:spf1.webtis.net a:acprelay1.uk.atosorigin.com a:tpe.delaycheck.freelygive.org.uk include:tpe-compensation.freelygive.org.uk include:eu._netblocks.mimecast.com include:amazonses.com ip4:78.154.117.130 ip4:94.236.83.186 -all

spf1.webtis.net

v=spf1 ip4:157.203.42.35 ip4:157.203.43.152 ip4:157.203.41.119 ip4:157.203.36.9 ip4:52.212.225.31 ip4:52.30.243.17 ip4:52.210.88.86 ip4:18.202.114.104 ip:3.8.227.20 ip:3.10.60.122 a:smtppost.atos.net -all

Maddy doesn't accept e-mail from here though, with an error saying check.spf: result: permerror (unknown field)

Steps to reproduce

Log files

Jun 08 06:54:17: smtp: incoming message        {"msg_id":"78676fbf","sender":"noreply@tpexpress.co.uk","src_host":"smarthost1.atos.net","src_ip":"193.56.114.176:27471"}
Jun 08 06:54:17: [debug] smtp/pipeline: initializing state for check.require_mx_record: (0xc00033c140)        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: initializing state for check.dkim: (0xc00035ab80)        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: initializing state for check.spf: (0xc0003a2180)        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: sender noreply@tpexpress.co.uk matched by default rule        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: global rcpt modifiers: ***@***.net => ***@***.net        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: per-source rcpt modifiers: ***@***.net => ***@***.net        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: recipient ***@***.net matched by domain rule 'tanukitsu.net'        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: per-rcpt modifiers: ***@***.net => ***@***.net        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] msgpipeline: sender noreply@tpexpress.co.uk matched by default rule        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] smtp/pipeline: tgt.Start(noreply@tpexpress.co.uk) ok, target = msgpipeline:local_routing        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] msgpipeline: global rcpt modifiers: ***@***.net => ***@***.net        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] msgpipeline: per-source rcpt modifiers: ***@***.net => ***@***.net        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] msgpipeline: recipient ***@***.net matched by domain rule 'tanukitsu.net'        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] msgpipeline: per-rcpt modifiers: ***@***.net => ***@***.net        {"msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] target.lmtp: connected        {"downstream_server":"","msg_id":"78676fbf"}
Jun 08 06:54:17: [debug] target.lmtp: connected        {"msg_id":"78676fbf","remote_server":""}
Jun 08 06:54:17: [debug] msgpipeline: tgt.Start(noreply@tpexpress.co.uk) ok, target = target.lmtp:local_mailboxes        {"msg_id":"78676fbf"}
Jun 08 06:54:18: smtp: RCPT ok        {"msg_id":"78676fbf","rcpt":"***@***.net"}
Jun 08 06:54:18: [debug] check.spf: result: permerror (unknown field)        {"msg_id":"78676fbf"}
Jun 08 06:54:18: [debug] autobuffer: keeping the message in RAM (read 142518 bytes, got EOF)
Jun 08 06:54:18: [debug] check.dkim: no signatures present        {"msg_id":"78676fbf"}
Jun 08 06:54:18: smtp/pipeline: no check action        {"check":"check.dkim","msg_id":"78676fbf","reason":"No DKIM signatures","smtp_code":550,"smtp_enchcode":"5.7.20","smtp_msg":"No DKIM signatures"}
Jun 08 06:54:18: smtp: DATA error        {"check":"check.spf","msg_id":"78676fbf","reason":"unknown field","smtp_code":550,"smtp_enchcode":"5.7.23","smtp_msg":"SPF authentication failed with a permanent error"}

Configuration file

## maddy 0.3 - default configuration file (2020-05-31)
# Suitable for small-scale deployments. Uses its own format for local users DB,
# should be managed via maddyctl utility.
#
# See tutorials at https://foxcpp.dev/maddy for guidance on typical
# configuration changes.
#
# See manual pages (also available at https://foxcpp.dev/maddy) for reference
# documentation.

# ----------------------------------------------------------------------------
# Base variables

debug yes

$(hostname) = ***.net
$(primary_domain) = **.net
$(local_domains) = $(primary_domain) ***.net

tls file /etc/maddy/tls/***.pem /etc/maddy/tls/***.key

# ----------------------------------------------------------------------------
# CUSTOM: OpenMetrics
# Expose Prometheus metrics over the wireguard VPN
openmetrics tcp://10.0.42.5:30734 { }

# ----------------------------------------------------------------------------
# Local storage & authentication

# pass_table provides local hashed passwords storage for authentication of
# users. It can be configured to use any "table" module, in default
# configuration a table in SQLite DB is used.
# Table can be replaced to use e.g. a file for passwords. Or pass_table module
# can be replaced altogether to use some external source of credentials (e.g.
# PAM, /etc/shadow file).
#
# If table module supports it (sql_table does) - credentials can be managed
# using 'maddyctl creds' command.

# CUSTOM
#auth.pass_table local_authdb {
#    table sql_table {
#        driver sqlite3
#        dsn credentials.db
#        table_name passwords
#    }
#}

# imapsql module stores all indexes and metadata necessary for IMAP using a
# relational database. It is used by IMAP endpoint for mailbox access and
# also by SMTP & Submission endpoints for delivery of local messages.
#
# IMAP accounts, mailboxes and all message metadata can be inspected using
# imap-* subcommands of maddyctl utility.

# CUSTOM
#storage.imapsql local_mailboxes {
#    driver sqlite3
#    dsn imapsql.db
#}

# ----------------------------------------------------------------------------
# SMTP endpoints + message routing

hostname $(hostname)

msgpipeline local_routing {
    # Insert handling for special-purpose local domains here.
    # e.g.
    # destination lists.example.org {
    #     deliver_to lmtp tcp://127.0.0.1:8024
    # }

    destination postmaster $(local_domains) {
        modify {
            replace_rcpt regexp "(.+)\+(.+)@(.+)" "$1@$3"

            # CUSTOM: dot aliases
            replace_rcpt regexp "(.+)\.(.+)@(.+)" "$2@$3"

            # CUSTOM: lookup aliases in the database
            replace_rcpt sql_query {
                driver postgres
                dsn "host=127.0.0.1 dbname=courriel user=courriel password=***"
                lookup "SELECT destination FROM aliases WHERE source = $1"
            }

            # CUSTOM: we'll use SQL for aliases instead
            #replace_rcpt file /etc/maddy/aliases
        }

        deliver_to &local_mailboxes
    }

    default_destination {
        reject 550 5.1.1 "User doesn't exist"
    }
}

smtp tcp://0.0.0.0:25 {
    limits {
        # Up to 20 msgs/sec across max. 10 SMTP connections.
        all rate 20 1s
        all concurrency 10
    }

    dmarc yes
    check {
        require_mx_record
        dkim
        spf
    }

    source $(local_domains) {
        reject 501 5.1.8 "Use Submission for outgoing SMTP"
    }
    default_source {
        destination postmaster $(local_domains) {
            deliver_to &local_routing
        }
        default_destination {
            reject 550 5.1.1 "User doesn't exist"
        }
    }
}

# CUSTOM submission tls://0.0.0.0:465 tcp://0.0.0.0:587 {
submission tls://0.0.0.0:465 {
    limits {
        # Up to 50 msgs/sec across any amount of SMTP connections.
        all rate 50 1s
    }

    # CUSTOM use Dovecot for auth
    #auth &local_authdb
    auth dovecot_sasl unix:///var/run/dovecot/auth-maddy-client

    source $(local_domains) {
        destination postmaster $(local_domains) {
            deliver_to &local_routing
        }
        default_destination {
            modify {
                dkim $(primary_domain) $(local_domains) default
            }
            deliver_to &remote_queue
        }
    }
    default_source {
        reject 501 5.1.8 "Non-local sender domain"
    }
}

# CUSTOM: use Dovecot for delivery to mailboxes
target.lmtp local_mailboxes {
    targets unix:///var/run/dovecot/lmtp-maddy
    attempt_starttls no
}

target.remote outbound_delivery {
    limits {
        # Up to 20 msgs/sec across max. 10 SMTP connections
        # for each recipient domain.
        destination rate 20 1s
        destination concurrency 10
    }
    mx_auth {
        dane
        mtasts {
            cache fs
            fs_dir mtasts_cache/
        }
        local_policy {
            # conceals hotmail blacklisting (hotmail won't offer starttls if you're BLed) min_tls_level encrypted
            min_tls_level none
            min_mx_level none
        }
    }
}

target.queue remote_queue {
    target &outbound_delivery

    autogenerated_msg_domain $(primary_domain)
    bounce {
        destination postmaster $(local_domains) {
            deliver_to &local_routing
        }
        default_destination {
            reject 550 5.0.0 "Refusing to send DSNs to non-local addresses"
        }
    }
}

Environment information

foxcpp commented 2 years ago

Duplicate of #485.