Mailer drops some recipients (CCs)

[kwatsen]

First, IDK if this ticket belongs here or if there's a better/different way to report such an issue.

Second, IDK if this issue is with any of IETF's tooling, or some intermediate SMTP-relay. I'm hoping that someone can take a peak at the IETF mailer's logs to help determine that...

What is the issue? The issue has occurred twice now (July 26 and again on July 30). In both cases, I sent an email CC-ing some recipients along with an IETF-managed list, but somehow the email sent out to all list recipients, and recorded in the mail archive, drops some of the CC's.

For instance, yesterday I sent this message to an individual (Juergen), CC-ing the list (netconf@ietf.org) and three other individuals (Mahesh, Joseph and Rob). But Mahesh and Rob were dropped from the CC.

Details:

1. my local SMTP-client shows Mahesh/Rob in the CC (see [1] below)
2. my SMTP-server shows that it relayed the message to my SMTP-relay (Amazon AWS SES)
3. all individuals CC'ed report receiving the message (directly from me)
4. but the headers shown in the mail archive don't list Rob (see [2] below)

[1] This line from my SMTP-client's "Sent folder": Cc: Mahesh Jethanandani <mjethanandani@gmail.com>, Robert Wilton <rwilton@cisco.com>, Joseph Nicholson <joseph@ntt.net>, "netmod@ietf.org" <netmod@ietf.org>

[2] This line from the mail archive's header: CC: Joseph Nicholson <joseph@ntt.net>, "netmod@ietf.org" <netmod@ietf.org>

In case it helps, following is the complete header from my SMTP-client's "Sent" folder:

Content-Type: text/plain;
    charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\))
Subject: Re: [netmod] AD review of draft-ietf-netmod-rfc6991-bis-15
X-Universally-Unique-Identifier: EBC66DA2-0FEF-4BC3-8B91-55F6E57A6493
From: Kent Watsen <kent+ietf@watsen.net>
In-Reply-To: <bffad8a0-7321-463a-aca3-278527e570e6@constructor.university>
Date: Tue, 30 Jul 2024 13:02:17 -0400
Cc: Mahesh Jethanandani <mjethanandani@gmail.com>,
 Robert Wilton <rwilton@cisco.com>,
 Joseph Nicholson <joseph@ntt.net>,
 "netmod@ietf.org" <netmod@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <ABE1ACE1-BA13-4DD8-877D-6B82E7191A06@watsen.net>
References: <BY5PR11MB4196AF276BC24AB7BEC310A6B5869@BY5PR11MB4196.namprd11.prod.outlook.com>
 <20230323111314.gd5xis346eyylygt@anna>
 <AM7PR07MB624840744B6F925E96C6F526A0879@AM7PR07MB6248.eurprd07.prod.outlook.com>
 <CABCOCHRBPrHk8Wk0xJtYzhB8aTiS+bwUPxi70LtAPEL5Qy87kw@mail.gmail.com>
 <7CC7D3A9-B5DD-4857-B50B-82B7903B32C6@gmail.com>
 <bffad8a0-7321-463a-aca3-278527e570e6@constructor.university>
To: =?utf-8?B?SsO8cmdlbiBTY2jDtm53w6RsZGVy?= <jschoenwaelder@constructor.university>

[kesara]

@kwatsen With mailman3 there's a personal option to opt-out of receiving duplicates. Mailman will drop any CCs for those personal email addresses if they are already subscribed to the mailing list. I think this is what's happening here.

BTW I'll move this issue to the correct repository.

[kwatsen]

Thanks @kesara for your quick response.

I looked into mailman preferences, but don't see the option you mention.

To clarify, my issue doesn't regard what I receive, so much as what others receive.

Are you pointing to the Receive list copies (possible duplicates) setting?

[yaseppochi]

Yes, that's the option in question. Postfix handles the other addressees independently of the list. They do get the message.

[kwatsen]

I flipped all my Receive list copies (possible duplicates) settings to Yes. This both in the Global Mailman preferences tab (which was already Yes) and also in the List based preferences (which had the lists in question set to No). Should the setting be No?

I then sent a test-message, but again Rob was removed from the CC: https://mailarchive.ietf.org/arch/msg/netmod/tr4R0QUyLET2tSDhPYXtxfSnyaE/

The setting's description says this:

When you are listed explicitly in the To: or Cc: headers of a list message, you can opt to not receive another copy from the mailing list. Select Yes to receive copies. Select No to avoid receiving copies from the mailing list

Honestly, this doesn't sound like my issue...

Please advise.

[rjsparks]

Hi Kent -

What you're seeing is (as I understand it) the way mailman behaves (mailman2 behaved similarly). If someone is subscribed to a list but also appears on the CC - they are removed from the CC before the message is sent on to the list recipients (and to any archiver). The origin of the behavior was to control CC growth during reply-alls, and there's been back-and-forth over time in the mailman community about whether the current behavior is "right" or not, but it's the compromise that currently stands.

I'm looking for specific documentation to point to, and history of the back-and-forth, and will send that when I find it (hopefully someone else beats me to it), but you won't be able to find a setting to make that stop - changing it would involve forking from mailman and making a different choice than the one that's currently in the code, and i'm told that will likely disrupt other users.

[rjsparks]

If I read correctly, this is the relevant bit of code:

git clone https://gitlab.com/mailman/mailman.git
cd mailman
cat ./src/mailman/handlers/avoid_duplicates.py

[kwatsen]

Hi @rjsparks, that does look like the code.

Okay, are you suggesting that the other recipient (Rob), the CC that was dropped, must have their Receive list copies (possible duplicates) setting set to No? That is, it's not my setting so much as their setting, and maybe they recently changed it?

FWIW, I have many times over the years CC-ed folks that I know are on the list, this is name-references are clear. For instance, as in this specific case, I'm sending email to X, but wanting X to know the Y and Z are involved. I do this instead of writing out full names. IDK, maybe it never worked and I'm just now noticing.

[kwatsen]

Regarding the philosophy behind this logic, I'm sure I'm not going to change any minds, but maybe there is a middle-path solution, which is, leave the headers alone, but don't actually send the duplicate message.

I suggest this because it seems that mailman shouldn't alter the CC's seen by others, as doing so can change the semantic (legal?) meaning of the message (such as in my case). And besides, why should software nanny users? Everyone is free to reduce the CC's on their own, right? [I realized that many don't, but that's a different problem]

[kwatsen]

Update and Proposal

I reached out to Rob and, sure enough, he had some "No" settings. He switched them to "Yes" and now my CC's to him are NOT dropped - yes!

I also reached out to Reshad, who I had a similar issue with. In all cases (myself included), none of us recall ever configuring the List based preferences. In fact, looking at my own List based preferences, I don't know why any of them are there, overriding my global defaults.

My speculation is that the Mailman3 upgrade added a bunch of List based preferences and (incorrectly) set some Receive list copies (possible duplicates) settings to No. IDK if true but, if so, would be a problematic/unannounced change in behavior. It would be great if this theory could be tested.

In any case, it seems that an email to all list-users may be needed.