Gmail rejects Forwarded Emails

mindrunner commented 4 years ago

Hi, you can see my setup working here: https://github.com/trusteddomainproject/OpenARC/issues/124

However, I am getting those messages in my Log whe forwarding mails to Gmail.

Jan 17 14:14:39 mx0 postfix/smtp[319182]: 3C3A2614B2: to=<me@gmail.com>, relay=gmail-smtp-in.l.google.com[74.125.140.26]:25, delay=0.42, delays=0.03/0/0.16/0.23, dsn=5.7.26, status=bounced (host gmail-smtp-in.l.google.com[74.125.140.26] said: 550-5.7.26 Unauthenticated email from genesis-mining.com is not accepted due to 550-5.7.26 domain's DMARC policy. Please contact the administrator of 550-5.7.26 genesis-mining.com domain if this was a legitimate mail. Please 550-5.7.26 visit 550-5.7.26  https://support.google.com/mail/answer/2451690 to learn about the 550 5.7.26 DMARC initiative. y80si2990899wmc.117 - gsmtp (in reply to end of DATA command))

Isn't OpenARC supposed to exactly fix this problem?

mindrunner commented 4 years ago

This is the headers if I catch this email on my server befor forwarding it to google.

Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=198.37.146.36; helo=o24.email.freshdesk.com; envelope-from=bounces+4491982-1251-genesismining=domain.com@fddkim.genesis-mining.com; receiver=<UNKNOWN> 
MIME-Version: 1.0
Date: Fri, 17 Jan 2020 14:42:07 +0000 (UTC)

mindrunner commented 4 years ago

Is that a configuration issue on their side? Is there any way for me to forward such faulty emails to Gmail? Or at least put them into a separate inbox somewhere?

At the moment, I am losing those mails without any notification. And the odd thing is, that this seem to never happen with EMails I do not want to receive, but only with Emails I DO want to receive (e.g. dhl.com, deutsche-post.de, genesis-mining.com, etc)

jcapo commented 4 years ago

Deliver to a mailbox on your server. Poll the mailbox on your server with the Google Mail Fetcher.

ARC requires trust between the ARC signer and the recipient server before its useful. All an ARC seal says is that your domain created the ARC seal.

John Capo Tuffmail.com

On Fri, January 17, 2020 09:53, lukas wrote:

Is that a configuration issue on their side? Is there any way for me to forward such faulty emails to Gmail? Or at least put them into a separate inbox somewhere?

At the moment, I am losing those mails without any notification. And the odd thing is, that this seem to never happen with EMails I do not want to receive, but only with Emails I DO want to receive (e.g. dhl.com, deutsche-post.de, genesis-mining.com, etc)

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/trusteddomainproject/OpenARC/issues/129#issuecomment-575657662

mindrunner commented 4 years ago

Poll the mailbox on your server with the Google Mail Fetcher.

Not an option. Have had this setup before and it sucks. Google has no option to configure the fetchmail frequency. Thus, sometimes a mail is delayed by 10 minutes or more. This is why I went for the forwarding setup.

ARC requires trust between the ARC signer and the recipient server before its useful.

So how do I create trust? Which problem is ARC actually solving then if not the forwarding-one?

jcapo commented 4 years ago

On Fri, January 17, 2020 11:32, lukas wrote:

ARC requires trust between the ARC signer and the recipient server before its useful.

So how do I create trust. Which problem is ARC actually solving then if not the forwarding-one?

IMO, none.

John Capo Tuffmail.com

mindrunner commented 4 years ago

Ok, I am asking a slightly different question then:

Which problem is ARC actually trying to solve?

I think I have misunderstood something about ARC then...

jcapo commented 4 years ago

On 2020-01-17 11:37, lukas wrote:

Ok, I am asking a slightly different question then:

Which problem is ARC actually trying to solve?

As I understand it, the intention is to solve the SPF forwarding problem and maybe busted DKIM signatures. But, unless Google trusts your ARC signature the message will not be accepted.

We forward boatloads of mail with valid DKIM signatures from domains that publish reject DMARC records to Google and it is accepted. Mail forwarded from domains publishing only SPF records and publiching reject DMARC records is rejected as it probably should be.

Maybe domains publishing a reject DMARC record without DKIM don't want their mail forwarded.

John Capo Tuffmail.com

mindrunner commented 4 years ago

My observation is still, that google does not reject all mails from a sender who has a reject-policy. Sometimes they are coming through. Most of the time marked as spam/scam, though.

Are you working for tuffmail? May I contact you in a private chat/email for some questions which are off-topic here?

jcapo commented 4 years ago

On Fri, January 17, 2020 12:04, lukas wrote:

My observation is still, that google does not reject all mails from a sender who has a reject-policy. Sometimes they are coming through. Most of the time marked as spam/scam, though.

Are you working for tuffmail? May I contact you in a private chat/email for some questions which are off-topic here?

Yes, any time.

John

flowerysong commented 4 years ago

ARC provides a trustable means to assert a message's authentication status as seen by each handler in the chain; this does not automatically create a trust relationship between all parties who implement ARC in their message handling. Individual mail recipients decide whose assertions they trust, and that local policy and its effects on message handling have nothing to do with OpenARC.

mindrunner commented 4 years ago

Does that mean I have to make google trusting my mailserver to accept those emails?

slashrolf commented 4 years ago

Hi, Lukas,

sorry for the top-posting. I forwarded your question to Brandon Long from Google. He answered:

<Brandon>
Feel free to forward this with my email address.

The example doesn't have an authres header for the spf pass, so it
wasn't included in the aar header, so there's no way for us to
validate dmarc with our system.  We could trust the dmarc=pass
itself, but don't currently.  I assume this is some misconfig on
their part.

That said, we currently only trust a limited number of arc signers,
which doesn't include this one, so we wouldn't accept the arc
anyways.  Figuring out how to know whether to trust any given arc
signer is TBD, that's why arc is experimental.  Figuring out why
you're breaking the dkim signature would probably be better for now,
or using pop fetching for this case.

Also, the message shouldn't be lost, it should be rejected at smtp
time, and presumably your server should generate a bounce in that
case, not sure of to that srs address or the original sender.

Brandon
</Brandon>

Brandons mail address is in the cc of this message.

Regards, /rolf

On 17-01-2020 19:36, lukas wrote:

Does that mean I have to make google trusting my mailserver to accept those emails?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/trusteddomainproject/OpenARC/issues/129?email_source=notifications&email_token=AKB3YS7JYGI27Z35MND7TRTQ6H3C3A5CNFSM4KIJJJX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJISSOI#issuecomment-575744313, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKB3YS63JV24QMDIKSE3X73Q6H3C3ANCNFSM4KIJJJXQ.

mindrunner commented 4 years ago

Hi Rolf! Wow, that is insightful

sorry for the top-posting. I forwarded your question to Brandon Long from Google

No worries! That's really appreciated. :)

The example doesn't have an authres header for the spf pass, so it wasn't included in the aar header, so there's no way for us to validate dmarc with our system. We could trust the dmarc=pass itself, but don't currently. I assume this is some misconfig on their part.

I do not understand this part. Is that a misconfiguration on my side or on the original sender's side? What is aar? What is authres?

That said, we currently only trust a limited number of arc signers, which doesn't include this one, so we wouldn't accept the arc anyways.

That said, OpenArc is useless on my server, because noone trusts me anyways?

Figuring out how to know whether to trust any given arc signer is TBD, that's why arc is experimental.

It's been 3 years now.. Or 4? So what work is being done on this? Is this eventually gonna happen or are we waiting for a dead man?

Figuring out why you're breaking the dkim signature would probably be better for now

So am I breaking this? Most of the mails come through without a broken DKIM signature. Is that maybe some sort of zendesk-issue again? (https://github.com/trusteddomainproject/OpenARC/issues/126) Then I would say, it's a configuration issue on the sender's side, right?

or using pop fetching for this case.

It's not a secret that Googles pop fetching sucks for every-day use. It is good if you want to integrate a legacy account, but for mails which have to arrive instanty, this is useless.

Also, the message shouldn't be lost, it should be rejected at smtp time, and presumably your server should generate a bounce in that case, not sure of to that srs address or the original sender.

Yes, Gmail rejects it and bounces it back. Probably the original sender gets a bounce notification. But they do not care and have no clue anyways. It's me who wants to get the notification. How can I tell postfix to hand a copy of the bounced message to me?

adam-iris commented 4 years ago

I've been struggling with a similar situation for several days now. (It would really be super convenient if Google gave any sort of diagnostic info here, or indeed if I could find any tool that would clearly analyze ARC headers.)

I noticed, though, that Google appears to be doing ARC validation on emails even when the DMARC record doesn't indicate p=reject.

So my question is, if Google's aar header says arc=pass; dmarc=fail on this sort of message, does this mean that the ARC validation succeeded, and that it's bouncing other messages not due to any problem with our ARC headers, but just because we aren't a trusted intermediary?

The full ARC headers look like this:

Arc-Seal: i=2; a=rsa-sha256; t=1581538352; cv=pass; d=google.com; s=arc-20160816; b=EgZgksOvJoSFt/kHCR84bEfI59RmGg87Cip6YVC9ZXHEDFovDdyBoHqyALRvDfNo0/ pTYYe70DvEW2+vPO7RmSjMYciqdUxF42zlc4hsCqaQprl7mlWSLHzU5x/tq23KcVMF7x T1gojdS7/5ibXaBj2i7M+54x5WX9bvC3Qnk1Z6ciTcX0fMaqowF64YNsaWUvpGyTNYWD VFnxJMk2eXSr7yVWht0a1DijRjsDrdEtnd4ArLdFzgEy6tMcJXUGvW8SOTO+oBXpZapU 06KSOCbTTYtDXRVhdEPzyAO/Rf/kCyqiYoM7ljjG6ybaYlL2bhTYGPotEOSqnGsMRDSm yk/w==
Arc-Seal: i=1; cv=none; a=rsa-sha256; d=iris.washington.edu; s=default; t=1581538351; b=WifQ/vKCB7AOQNCzHr5DL/p2YovUX7pQDDB+oChr7R1lmNNuTWvxenqqG7udVX06JqrK8 R9UyFRB4DNmj+jbBWHT4FTx6AZfK0WsTrq2g7Va8XRcKQky9AmUUnXUMLSNf+0vDk4Azi/9 hQyCd+/EkFdgqL9v8O5DCSak8As2Zd4=
Arc-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@iris.washington.edu header.s=default header.b=NuUFqrKJ; dkim=fail header.i=@gmail.com header.s=20161025 header.b=erRanm81; arc=pass (i=1 dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of adam@iris.washington.edu designates 128.95.166.28 as permitted sender) smtp.mailfrom=adam@iris.washington.edu; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com
Arc-Authentication-Results: i=1; iris.washington.edu; dkim=pass header.d=gmail.com; arc=none; dmarc=none
Arc-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=to:date:message-id:subject:mime-version:content-transfer-encoding :from:dkim-signature:dkim-filter:dkim-signature:dkim-filter; bh=eC20HmzbMPm50ripBzOh7T5g0amfPaODvY3WjxrTPcc=; b=RJjnJBnuPKM3PbdArXBvS2YW/ioHBFLXkjOnxvJ/xexjTinAE1pP0EXULrE77T+ZyT IvgPWMV/P12ZOb//asZQN8cEYSqzZ/gawsDRwGfMu/7w7yyuz6ob5wIJfQWRxp2GUPEJ XDE9Ozb+mPF9aK1m7UgcxoKb2RBs7WLBFCcb5TXJcuWgFiK0QxwfROoWtfLj2nZXDIoZ u6SqBx77Zq/123P9dGUA9R6ifMReHAEPolfe5vf1uVU0n8xbKdTbYu3u4bB4An4jMv8c J8c8wKfSZKats4/z9ahw4XO5YiePNFp3lOqug0ElJJVFSapbD9T9oiPPcIs0fsQ5b3O1 ZlKA==
Arc-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=iris.washington.edu; s=default; t=1581538351; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=eC20HmzbMPm50ripBzOh7T5g0amfPaODvY3WjxrTPcc=; b=blQOqnI7IoLG3p6v/pWJo3oEezKOtIIo0gqw1kf3axAZEWUAorpxDcyAUfJjXwFC9Hhk5 rdszvTNWE0eC+idtG8MQZDj9oPP0Wu+YuaTl0hMTMJrt2G7haTSMgqh7W63G6PhF4ps6GtW DWTfS6/xIc1920mcm0ZYswyzgYwH8h8=

kurta commented 4 years ago

On Wed, Feb 12, 2020 at 12:55 PM Adam Clark notifications@github.com wrote:

So my question is, if Google's aar header says arc=pass; dmarc=fail on this sort of message, does this mean that the ARC validation succeeded, and that it's bouncing other messages not due to any problem with our ARC headers, but just because we aren't a trusted intermediary?

If they are bouncing "other messages" then presumably there is something about those messages they don't like. You are properly understanding header combo you cited. Note that ARC results are only part (if any) of the mix in determining the disposition of any message at any receiver.

--Kurt

kurta commented 4 years ago

On Sat, Jan 18, 2020 at 5:43 AM lukas notifications@github.com wrote:

It's been 3 years now.. Or 4? So what work is being done on this? Is this eventually gonna happen or are we waiting for a dead man?

Technically the ARC RFC was only published about 6 months ago. Based on that there is implementation work in the pipeline at a variety of MTA vendors and service providers (who have to remain unnamed for now).

--Kurt

adam-iris commented 4 years ago

If they are bouncing "other messages" then presumably there is something about those messages they don't like.

Presumably what they don't like is that the DMARC policy for the originating domain says to reject these messages. My understanding of ARC is that there are 3 possible reasons for this:

The receiver doesn't implement ARC
The ARC headers are invalid
The receiver ignores the ARC headers because the intermediary isn't trusted

Since option 2 is the only one I have control over, I'm trying to figure out how to distinguish option 2 from option 3.

martinbogo commented 4 years ago

Since none of this resolves into a bug that needs to be fixed in OpenARC, I am closing this ticket. It's up to Google if they trust the ARC signer and that is outside the scope of any code changes to OpenARC as it's a policy issue.

trusteddomainproject / OpenARC

Gmail rejects Forwarded Emails #129