search

ietf-tools / postconfirm

BSD 3-Clause "New" or "Revised" License
3 stars 9 forks source link

Duplicate delivery of mailman emails causes SPF rejection on one copy #35

Open andrewgdotcom opened 1 year ago

andrewgdotcom commented 1 year ago

I'm a member of openpgp@ietf.org, and some list emails have been failing SPF validation unexpectedly. I see the same emails being delivered to my server twice, once with an unchanged sender, which fails SPF, and a second time with the sender correctly rewritten to the list bounces address, which succeeds.

AFAICT, this happens when a reply-all email is addressed to both me directly and the mailman list. It causes no problems at my end, since at least one copy always gets delivered, but the sender gets a concerning bounce message saying that the mail was not delivered (which is technically true as one of the copies did fail, but misleading).

In an email conversation with the IETF operations team this appears to be caused by a DMARC vs SPF policy-handling mismatch in postconfirm:

That reverse rewriter is preserving the [sender] address, but sending it "through" IETFA, alias-style, to [the recipient], who is then rejecting it based on SPF violations. Which means, if I am interpreting everything correctly, that because Postconfirm does not see a p=reject DMARC policy on [the sender's domain], it is not rewriting the email; however, because [the sender] DOES have a "-all" SPF policy, [the recipient] is, more or less, within their rights to hard-reject the email being resent back through Postconfirm as an SPF violation.

A fuller copy of this conversation (with logs) has been sent to tools-help@ietf.org

A

jrlevine commented 1 year ago

Two things: the mail is coming from the IETF so it's forwarded. We need some message headers to tell what the forwarding address is.

Also, if your mail system rejects on SPF -all, you've set it to refuse all forwarded mail. You can certainly set up your mail however you want, but don't be surprised when it does what you tell it to. I know a lot of large mail systems and exactly none reject on SPF failure because (as you just saw) it loses large amounts of mail that people want.

andrewgdotcom commented 1 year ago

Apologies for the slow response, I was on leave for two weeks.

I think you misunderstand my problem. My mail system is doing exactly what I want it to do. Nobody (including the IETF) should be forwarding emails to my address without invoking SRS - any email so forwarded without SRS is spam, and I am deliberately rejecting it. My problem is that the IETF mail system is incorrectly sending such badly-forwarded emails as duplicates of (correctly-processed) mailing list messages, and my correspondents are receiving technically-correct but highly misleading error messages as a result.

Here are the headers of the correctly-delivered (list) copy of the last offending email.

Return-Path: <SRS0=+pb3=D3=ietf.org=openpgp-bounces@andrewg.com>
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fum.andrewg.com
X-Spam-Level: 
X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
    DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,
    MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H4,
    RCVD_IN_MSPIKE_WL,SPF_PASS autolearn=ham autolearn_force=no
    version=3.4.6
X-Original-To: andrewg@andrewg.com
Delivered-To: andrewg@andrewg.com
Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=50.223.129.194; helo=mail.ietf.org; envelope-from=openpgp-bounces@ietf.org; receiver=<UNKNOWN> 
Authentication-Results: fum.andrewg.com;
    dkim=pass (1024-bit key; unprotected) header.d=ietf.org header.i=@ietf.org header.a=rsa-sha256 header.s=ietf1 header.b=OAgrWKRP;
    dkim=fail reason="signature verification failed" (1024-bit key) header.d=ietf.org header.i=@ietf.org header.a=rsa-sha256 header.s=ietf1 header.b=Ybp4MzkA;
    dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=andrewg.com header.i=@andrewg.com header.a=rsa-sha256 header.s=andrewg-com header.b=DmDV0NVJ;
    dkim-atps=neutral
Received: from mail.ietf.org (mail.ietf.org [50.223.129.194])
    (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
     key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256)
    (No client certificate requested)
    by fum.andrewg.com (Postfix) with ESMTPS id A22DD5F7D9
    for <andrewg@andrewg.com>; Thu, 10 Aug 2023 08:44:16 +0000 (UTC)
Received: from ietfa.amsl.com (localhost [IPv6:::1])
    by ietfa.amsl.com (Postfix) with ESMTP id 6A639C16B5C0
    for <andrewg@andrewg.com>; Thu, 10 Aug 2023 01:44:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ietf.org; s=ietf1;
    t=1691657050; bh=Yg+vtbomJ13Tb5NuZWmCAvV43icussguJHpqlX0tRoU=;
    h=In-Reply-To:Date:Cc:References:To:Subject:List-Id:
     List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
     From;
    b=OAgrWKRPKIGblp3GzsGAM4+g3yYZ6duWfqVW4ISPpU99h5OX7RLdrRrhnHtOAGCfr
     KnFx7RVcCzqfJsrAbpB5KSVQfezdBRpHF1qDyetDBcheNBVw/MmmiiRzyQ5CUKhojp
     vMEir4lDNq8sq+dlQFC8SwlGM85bR57OeZOTDy/0=
Received: from ietfa.amsl.com (localhost [IPv6:::1])
 by ietfa.amsl.com (Postfix) with ESMTP id D5D9FC13AE39;
 Thu, 10 Aug 2023 01:44:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ietf.org; s=ietf1;
 t=1691657050; bh=Yg+vtbomJ13Tb5NuZWmCAvV43icussguJHpqlX0tRoU=;
 h=From:In-Reply-To:Date:Cc:References:To:Subject:List-Id:
 List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe;
 b=Ybp4MzkAIqd3l2m6sbXHstqubsnJwkr2edqQt2MaKuFA7PKW6YZTyfOsviXxcj3RC
 pNemIArlvyFfiEvgUfILc2+vrjeoihw+N1gt7KiO/TOlWv/d8h/QAS+11TjDBJh5Mv
 6Mv1UR0i2o4ofCYC2QneTYkAZbYuuXCxfNsDV1HM=
X-Original-To: openpgp@ietfa.amsl.com
Delivered-To: openpgp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1])
 by ietfa.amsl.com (Postfix) with ESMTP id 67FFCC13AE38
 for <openpgp@ietfa.amsl.com>; Thu, 10 Aug 2023 01:44:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key)
 header.d=andrewg.com
Received: from mail.ietf.org ([50.223.129.194])
 by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id iD_lT-gxQCSz for <openpgp@ietfa.amsl.com>;
 Thu, 10 Aug 2023 01:44:04 -0700 (PDT)
Received: from fum.andrewg.com (fum.andrewg.com [IPv6:2a01:4f9:c011:23ad::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by ietfa.amsl.com (Postfix) with ESMTPS id 8DD28C1522A0
 for <openpgp@ietf.org>; Thu, 10 Aug 2023 01:44:03 -0700 (PDT)
Received: from smtpclient.apple (unknown [IPv6:fc93:5820:7349:eda2:99a7::1])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by fum.andrewg.com (Postfix) with ESMTPSA id ED1BE5F4AD;
 Thu, 10 Aug 2023 08:44:00 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=andrewg.com;
 s=andrewg-com; t=1691657041;
 bh=jLXHvhC+oLgL+JguNzvXxzkS2BLlH9jSUl5qPtkhZA8=;
 h=Subject:From:In-Reply-To:Date:Cc:References:To:From;
 b=DmDV0NVJIuUZmtdbZ6xd290vf1DJ/QcedEsKyVuq6C67CVyz1FywTLgh5O2k1rRz0
 RZgUVK9nalkxnUC8jzeaczbwhYg6uWSDLh6sxCwhnOSM5d3uPLTHwQmkzLtXJ742wf
 ZEZbebuRHkn2zO3lV/dxqYdtuPExraDHU9AYgj3cgYNNoUunX9IrUKTyguz/dytEPT
 RxpF3c61QHimUDSxreDJRlX8UPL870gFdrybzi5C9L2bn0dGjYZNLfVfFcVa2WRH0l
 5vOe7SbGy1ZwrYu34G+M3pQS912zqxS3obkbErtxArGhhklCEit4o2zFUpiJXSCpnY
 4gz3ytfFEHVzQ==
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\))
In-Reply-To: <17566aae-8b29-4223-b641-26846f3d64f7@kuix.de>
Date: Thu, 10 Aug 2023 09:43:41 +0100
Cc: Andrew Gallagher <andrewg=40andrewg.com@dmarc.ietf.org>,
 Wiktor Kwapisiewicz <wiktor=40metacode.biz@dmarc.ietf.org>,
 IETF OpenPGP WG <openpgp@ietf.org>
Message-Id: <A6E69047-F50E-4122-954E-993253193577@andrewg.com>
References: <48be3fcf-cdce-9ef4-655b-63b6dddf9310@kuix.de>
 <20201211095836.5218a72e@computer>
 <cd02d2db-0671-dfc0-dab3-dc793a2c1605@metacode.biz>
 <878sa4y7hy.wl-neal@walfield.org>
 <4dbaf770-2b2e-47cc-afb5-3ba07706aafd@kuix.de>
 <87a5v1j4xo.fsf@wheatstone.g10code.de>
 <db447915-fc25-4759-879e-b64020c0ec0e@kuix.de>
 <87zg31hoee.fsf@wheatstone.g10code.de>
 <ba560bb0-0fa5-40a2-b70d-83f36859e17e@metacode.biz>
 <87v8dphmec.fsf@wheatstone.g10code.de>
 <17a06888-8516-457f-8ef3-85b7c77ce2f6@kuix.de>
 <51A7082C-D4E0-4577-B3E5-0688664FDD0F@andrewg.com>
 <17566aae-8b29-4223-b641-26846f3d64f7@kuix.de>
To: Kai Engert <kaie@kuix.de>
X-Mailer: Apple Mail (2.3731.600.7)
Archived-At: <https://mailarchive.ietf.org/arch/msg/openpgp/VLrBWpi4n7SjFEHKE7chzbbl79Q>
Subject: Re: [openpgp] Put Signature in an Email's Header
X-BeenThere: openpgp@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Ongoing discussion of OpenPGP issues." <openpgp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/openpgp>,
 <mailto:openpgp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/openpgp/>
List-Post: <mailto:openpgp@ietf.org>
List-Help: <mailto:openpgp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/openpgp>,
 <mailto:openpgp-request@ietf.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============5428654577494993766=="
Errors-To: openpgp-bounces@ietf.org
Sender: "openpgp" <openpgp-bounces@ietf.org>
X-Original-From: Andrew Gallagher <andrewg@andrewg.com>
From: Andrew Gallagher <andrewg=40andrewg.com@dmarc.ietf.org>

And the corresponding mail logs (note one outgoing, one successful incoming and one failed incoming delivery):

Aug 10 08:44:01 fum postfix/qmgr[1728403]: ED1BE5F4AD: from=<andrewg@andrewg.com>, size=3803, nrcpt=4 (queue active)
Aug 10 08:44:02 fum postfix/smtp[1438733]: Trusted TLS connection established to mail.ietf.org[50.223.129.194]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
Aug 10 08:44:03 fum postfix/smtp[1438734]: Trusted TLS connection established to mail.ietf.org[2001:559:c4c7::100]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
Aug 10 08:44:04 fum postfix/smtp[1438734]: ED1BE5F4AD: to=<openpgp@ietf.org>, relay=mail.ietf.org[2001:559:c4c7::100]:25, delay=3.9, delays=0.42/0.02/2.1/1.4, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 8DD28C1522A0)
Aug 10 08:44:05 fum postfix/smtp[1438733]: ED1BE5F4AD: to=<andrewg=40andrewg.com@dmarc.ietf.org>, relay=mail.ietf.org[50.223.129.194]:25, delay=4.7, delays=0.42/0.01/1.8/2.4, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 70D1FC15108A)
Aug 10 08:44:05 fum postfix/smtp[1438733]: ED1BE5F4AD: to=<wiktor=40metacode.biz@dmarc.ietf.org>, relay=mail.ietf.org[50.223.129.194]:25, delay=4.7, delays=0.42/0.01/1.8/2.4, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 70D1FC15108A)
Aug 10 08:44:15 fum postfix/smtpd[1438738]: connect from mail.ietf.org[50.223.129.194]
Aug 10 08:44:16 fum postfix/smtpd[1438738]: Anonymous TLS connection established from mail.ietf.org[50.223.129.194]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256
Aug 10 08:44:16 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:16 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:16 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:16 fum policyd-spf[1438741]: prepend Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=50.223.129.194; helo=mail.ietf.org; envelope-from=openpgp-bounces@ietf.org; receiver=<UNKNOWN>
Aug 10 08:44:16 fum postfix/smtpd[1438738]: A22DD5F7D9: client=mail.ietf.org[50.223.129.194]
Aug 10 08:44:16 fum postsrsd[1438708]: srs_forward: <openpgp-bounces@ietf.org> rewritten as <SRS0=+pb3=D3=ietf.org=openpgp-bounces@andrewg.com>
Aug 10 08:44:16 fum postsrsd[1438708]: srs_forward: <SRS0=+pb3=D3=ietf.org=openpgp-bounces@andrewg.com> not rewritten: Valid SRS address for <openpgp-bounces@ietf.org>
Aug 10 08:44:16 fum postfix/cleanup[1438707]: A22DD5F7D9: message-id=<A6E69047-F50E-4122-954E-993253193577@andrewg.com>
Aug 10 08:44:16 fum opendkim[1022]: A22DD5F7D9: message has signatures from ietf.org, ietf.org, andrewg.com
Aug 10 08:44:16 fum opendkim[1022]: A22DD5F7D9: s=ietf1 d=ietf.org a=rsa-sha256 SSL
Aug 10 08:44:17 fum postfix/qmgr[1728403]: A22DD5F7D9: from=<SRS0=+pb3=D3=ietf.org=openpgp-bounces@andrewg.com>, size=8832, nrcpt=1 (queue active)
Aug 10 08:44:17 fum postfix/smtpd[1438738]: disconnect from mail.ietf.org[50.223.129.194] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Aug 10 08:44:18 fum postfix/local[1438743]: A22DD5F7D9: to=<andrewg@andrewg.com>, relay=local, delay=1.9, delays=0.52/0.01/0/1.4, dsn=2.0.0, status=sent (delivered to command: procmail -a "$EXTENSION")
Aug 10 08:44:18 fum postfix/qmgr[1728403]: A22DD5F7D9: removed
Aug 10 08:44:25 fum postfix/smtpd[1438738]: connect from mail.ietf.org[50.223.129.194]
Aug 10 08:44:27 fum postfix/smtpd[1438738]: Anonymous TLS connection established from mail.ietf.org[50.223.129.194]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256
Aug 10 08:44:27 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:27 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:27 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:27 fum policyd-spf[1438741]: 550 5.7.23 Message rejected due to: SPF fail - not authorized. Please see http://www.openspf.net/Why?s=mfrom;id=andrewg@andrewg.com;ip=50.223.129.194;r=<UNKNOWN>
Aug 10 08:44:27 fum postfix/smtpd[1438738]: NOQUEUE: reject: RCPT from mail.ietf.org[50.223.129.194]: 550 5.7.23 <andrewg@andrewg.com>: Recipient address rejected: Message rejected due to: SPF fail - not authorized. Please see http://www.openspf.net/Why?s=mfrom;id=andrewg@andrewg.com;ip=50.223.129.194;r=<UNKNOWN>; from=<andrewg@andrewg.com> to=<andrewg@andrewg.com> proto=ESMTP helo=<mail.ietf.org>
Aug 10 08:44:27 fum postfix/smtpd[1438738]: disconnect from mail.ietf.org[50.223.129.194] ehlo=2 starttls=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=6/8
Aug 10 08:44:28 fum postfix/smtpd[1438738]: connect from mail.ietf.org[50.223.129.194]
Aug 10 08:44:29 fum postfix/smtpd[1438738]: Anonymous TLS connection established from mail.ietf.org[50.223.129.194]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256
Aug 10 08:44:29 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:29 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:29 fum postfix/smtpd[1438738]: warning: permit_tls_clientcerts is requested, but "smtpd_tls_ask_ccert = no"
Aug 10 08:44:29 fum policyd-spf[1438741]: prepend Received-SPF: None (no SPF record) identity=helo; client-ip=50.223.129.194; helo=mail.ietf.org; envelope-from=<>; receiver=<UNKNOWN>
Aug 10 08:44:29 fum postfix/smtpd[1438738]: D20055F7D9: client=mail.ietf.org[50.223.129.194]
Aug 10 08:44:29 fum postsrsd[1438708]: srs_forward: <""> not rewritten: No at sign in sender address
Aug 10 08:44:30 fum postfix/cleanup[1438707]: D20055F7D9: message-id=<20230810084427.AC694C15108A@ietfa.amsl.com>
Aug 10 08:44:30 fum postfix/qmgr[1728403]: D20055F7D9: from=<>, size=9651, nrcpt=1 (queue active)
Aug 10 08:44:30 fum postfix/smtpd[1438738]: disconnect from mail.ietf.org[50.223.129.194] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Aug 10 08:44:30 fum postfix/local[1438743]: D20055F7D9: to=<andrewg@andrewg.com>, relay=local, delay=0.99, delays=0.18/0/0/0.8, dsn=2.0.0, status=sent (delivered to command: procmail -a "$EXTENSION")
Aug 10 08:44:30 fum postfix/qmgr[1728403]: D20055F7D9: removed

It seems to me that what is happening is as follows:

There appear therefore to be two errors:

jrlevine commented 1 year ago

tl:dr: No. As I hope everyone knows, SPF has a fundamental design flaw that breaks forwarded mail. SES is one of a variety of hacks intended to push the cost of that flaw onto other people. RFC 7208, which defines SPF, does not mention SES, and not because we didn't know about it. Surely you know how to adjust your mail system to accept the mail from the IETF you say your users want.

By the way the stuff you're seeing with =40 is a workaround for a DMARC bug and has nothing to do with SES.

andrewgdotcom commented 1 year ago

Again, you misunderstand my problem.

Cc: Andrew Gallagher <andrewg=40andrewg.com@dmarc.ietf.org>,
 Wiktor Kwapisiewicz <wiktor=40metacode.biz@dmarc.ietf.org>,
 IETF OpenPGP WG <openpgp@ietf.org>

Your system is incorrectly mangling CC addresses, which causes reply-all messages to be mis-routed via dmarc.ietf.org when they should be routed either directly (for me and Wiktor) or via mailman (for openpgp). The SPF failures are merely a symptom of this problem.

Surely you know how to adjust your mail system to accept the mail from the IETF you say your users want.

I would appreciate greatly if we could keep this discussion civil.

jrlevine commented 1 year ago

Once again, the =40 stuff has nothing to do with SES. It's a DMARC workaround.

andrewgdotcom commented 1 year ago

Forget I mentioned SRS. Regardless of what address-rewriting system you are using or why, it should not be applied to CC recipients that are outside your routing domain.

jrlevine commented 1 year ago

I happen to be working on the daemon this week, will take a look. The code was written a long time ago.

andrewgdotcom commented 1 year ago

Thanks!