Message sent to list does not arrive at sender when using Gmail

cirnod commented 2 years ago

Version

Sympa 6.2.68 (https://sympa-community.github.io/)

Installation method

unknown (I'm not managing the installation)

Expected behavior

Messages sent to a mailing list should arrive at sender of the message if the sender is a subscriber of the mailing list.

Actual behavior

When the sender uses a Gmail account for sending a message to a list where he is a subscriber he does not get a new message in the mailbox.

Steps to reproduce

Create mailing list with Gmail address as subscriber.
Use Gmail account to send email to the mentioned list.
Inspect Gmail mailbox and verify that no new message arrived.

Additional information

We think the issue is related to the Message-ID of the header of the email.
There are other reports of similar issues related to the use of sympa together with a Gmail account as a sender.
It is unclear whether the bug is caused by a non-conform implementation on the side of sympa or Gmail.
It seems that there is a conflicting interpretation of how the Message-ID in the header is used. Apparently, Gmail relies on the Message-ID to deduplicate messages. According to the statement on Wikipedia related to the Message-ID "No two different messages must ever have the same Message-ID. If two messages have the same Message-ID, they are assumed to be the same and one version is discarded." this seems to be fine. However, as far as we have seen, sympa is not altering the Message-ID when receiving a message from a sender and distributing it to all subscribers of the list. It is unclear whether this behavior is intentional. According to a closed issue, sympa is expected to alter the Message-ID in certain cases.
In case sympa actually should always alter the Message-ID, fixing this issue potentially has effects on loop detection.

Excerpt of RFC 5322 that seems relevant to this issue:

   The "Message-ID:" field provides a unique message identifier that
   refers to a particular version of a particular message.  The
   uniqueness of the message identifier is guaranteed by the host that
   generates it (see below).  This message identifier is intended to be
   machine readable and not necessarily meaningful to humans.  A message
   identifier pertains to exactly one version of a particular message;
   subsequent revisions to the message each receive new message
   identifiers.
      Note: There are many instances when messages are "changed", but
      those changes do not constitute a new instantiation of that
      message, and therefore the message would not get a new message
      identifier.  For example, when messages are introduced into the
      transport system, they are often prepended with additional header
      fields such as trace fields (described in section 3.6.7) and
      resent fields (described in section 3.6.6).  The addition of such
      header fields does not change the identity of the message and
      therefore the original "Message-ID:" field is retained.  In all
      cases, it is the meaning that the sender of the message wishes to
      convey (i.e., whether this is the same message or a different
      message) that determines whether or not the "Message-ID:" field
      changes, not any particular syntactic difference that appears (or
      does not appear) in the message.

ikedas commented 2 years ago

Hi @cirnod , Are you a listmaster of the mailing list server where you have problem? Or, are you an owner of the maling list where you have problem? If you are, please ask the listmaster for investigating the problem. Neither, at first please ask the owner of that mailing list for investigating the problem.

cirnod commented 2 years ago

I'm neither a listmaster nor a the owner of the mailing list where I had problems. However, I tried to investigate the problems together with the owner of the mailing list. In this issue, I tried to describe the findings from the investigations.

ikedas commented 2 years ago

Please contact the listmaster at first and ask them for investigating the problem and (if necessary) submitting report/question on this GitHub site. The information you or list owner can gather are not enough to ask for help on this site and to solve the problem.

ikedas commented 2 years ago

Hi @cirnod ,

Are you talking about this?

Messages sent to email alias or Group aren't in my inbox

To prevent clutter, Gmail doesn't deliver messages that you send to your own alias (or to a Group you belong to) to your inbox. You can find the message in Sent Mail or All Mail. ...

This is the feature of gmail and not the problem of Sympa.

cirnod commented 2 years ago

@ikedas: yes, I think the linked content seems to be related to this issue. However, as I tried to explain in the original message, I'm not (yet) convinced that this is an issue solely to be blamed on Gmail.

Could you please confirm that the behavior of Sympa to not modify the Message-ID is intentional?

ikedas commented 2 years ago

Could you please confirm that the behavior of Sympa to not modify the Message-ID is intentional?

The term "Message ID" explains by its own: Since it is unique ID of message, it should basically not be changed. Sympa is merely complying with that requirement.

The only exception in Sympa is the feature of message anonymization mode. The Message-ID is intentionally changed to meet that feature: To hide the sender’s information.

cirnod commented 2 years ago

As far as I understand, there are clearly cases where the Message-ID should be changed (see excerpt of RFC 5322 above). Mailing lists typically change properties of a message (e.g. subject, reply-to address, etc.). Whether the resulting message should be interpreted as a revision of the original message (which would require a new Message-ID) or not is in my opinion worth a discussion (especially in light of the fact that there exist email service providers that interpret rules regarding Message-ID very narrowly and therefore assume that messages are 100% identical if they have the same Message-ID).

ikedas commented 2 years ago

You want to solve the problem you describe in the title of this Issue, don't you?

To do so, it is impossible to have all the mailing list services and mail aliases in the world rewrite the message IDs. Because they are not doing so.

On the other hand, after successful delivery between MTAs, it is up to each site to decide how to deliver (or not to deliver) messages to the mailbox. As the quote above indicates, Gmail "doesn't deliver messages that you send to your own alias (or to a Group you belong to) to your inbox" because they consider them "clutter" (however they not make the mail unreadable).

This is the feature of Gmail and not the bug in mailing list services and maili aliases in the world.

To solve the problem you describe in the title of this Issue, you'd be better to do either:

Tell Gmail to change their feature.
Stop using Gmail or tell the users to stop.
Explain the feature of Gmail quoted above to the users so that they can find the desired email in their mailboxes.

bboyle262 commented 2 years ago

For what it's worth I help manage a sympa list service with quite a few lists ; we had a report of this occurring but I was unable to replicate it on our service with my own testing with a test sympa list and Gmail.

I would check that your reception mode to the list in Sympa is NOT selected as "not_me", since that is designed not to deliver a copy of your own message to you, the sender.
Some people are apparently in the habit of CCing or BCCing themselves on every message they send out, and if you do this to a sympa list most MTAs will discard the duplicate which would be the version arriving again from Sympa.

ikedas commented 1 year ago

This is not a bug of Sympa. Close.

sympa-community / sympa