search

hardware / mailserver

:warning: UNMAINTAINED - Simple and full-featured mail server using Docker
https://store.docker.com/community/images/hardware/mailserver
MIT License
1.29k stars 324 forks source link

Outgoing email ended up in SPAM in Gmail #210

Closed sknight80 closed 6 years ago

sknight80 commented 6 years ago

Hi Team,

I noticed that when I set up a brand new VPS and check-out the latest version of the mail server (1.1 branch) the first real outgoing email to my Gmail account end up in spam. However before I send it to the Gmail mailbox, I check the setup on mail-tester.com. Are there some extra steps that I missed? Or it just a simple "fresh" IP address, be careful with it thing?

NOTE: The test email sent from a Wordpress website via this mail server after I authenticated with my account.

hardware commented 6 years ago

Look at the message headers with Gmail web ui.

jaguar1975cn commented 6 years ago

I have the same issue, messages sent to gmail was marked as spam. In the message source,

I can see:

arc=fail (signature failed);

Delivered-To: z****@gmail.com
Received: by 10.107.12.89 with SMTP id w86csp6409679ioi;
        Wed, 31 Jan 2018 02:18:13 -0800 (PST)
X-Google-Smtp-Source: AH8x224vH3QbsAQpYnejgrlqAttAAKxOJy5nDvVnO10dCHLMH3mVNLKrXQBsIA3/SQNAwhY4NnmO
X-Received: by 10.28.32.15 with SMTP id g15mr21123384wmg.22.1517393893354;
        Wed, 31 Jan 2018 02:18:13 -0800 (PST)
Return-Path: <j****@**acme.com>
Received: from mail.**acme.com (mail.**acme.com. [x.x.x.x])
        by mx.google.com with ESMTPS id 88si3376837wrg.254.2018.01.31.02.18.12
        for <z****@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
        Wed, 31 Jan 2018 02:18:12 -0800 (PST)
Received-SPF: pass (google.com: domain of j****@**acme.com designates x.x.x.x as permitted sender) client-ip=x.x.x.x;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@**acme.com header.s=mail header.b=nUEPf7+M;
       arc=fail (signature failed);
       spf=pass (google.com: domain of j****@**acme.com designates x.x.x.x as permitted sender) smtp.mailfrom=j****@**acme.com;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=**acme.com
Received: from authenticated-user (mail.**acme.com [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: j****@**acme.com) by mail.**acme.com (Postfix) with ESMTPSA id 9D8478038783 for <z****@gmail.com>; Wed, 31 Jan 2018 10:18:11 +0000 (UTC)
To: z****@gmail.com
From: John Smith <j****@**acme.com>
Subject: Email server test
Message-ID: <c7fbd59f-2433-4f3e-4737-f43396c5e40e@**acme.com>
Date: Wed, 31 Jan 2018 10:18:11 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=**acme.com; s=mail; t=1517393891; h=from:subject:date:message-id:to:mime-version:content-type:content-transfer-encoding; bh=2TW1tt/r/qTUVXw2uFLc5pQEFdee6J5pKHAn2eaHaNk=; b=C7xNFTBXB5iEod0F4+PoG7ZFEDrNkxnnDqOHtgVkGK8wf4evRaLLd7pbFTZDnmVD4j+DJ0 SZPwru4L9pkXLX3VFqKVCfw+FvxRbszoauxSlmbccs77QVj9AJq/M2Lwu2ILL1Mdj7rgj1 VqwA1ELQKAAPxmA+ApmvLS0oX2zeKZc=
ARC-Authentication-Results: i=1; auth=pass smtp.auth=j****@**acme.com smtp.mailfrom=j****@**acme.com
ARC-Seal: i=1; s=mail; d=**acme.com; t=1517393891; a=rsa-sha256; cv=none; b=O5SqVfkyHisV847x6ss9xdRYEzefOJdjmDRkbUJujICwBRKkqu25djuOAC+UFdk5EDZ0RfEj8+DwEZrI/xTV/rrP292d+5Z/juqr7oJZ52hl5CYAK/MlTdVDr5Vn9uAXei7IKnMfle+lN3vEdTy/qAyBIoA5lbjqZIclz0ZMA1Y=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=**acme.com; s=mail; t=1517393891; h=from:subject:date:message-id:to:mime-version:content-type:content-transfer-encoding; bh=2TW1tt/r/qTUVXw2uFLc5pQEFdee6J5pKHAn2eaHaNk=; b=nUEPf7+MMSkDKrl7nrzGNb16zb6qhOEXNrbybN17RM6Q8zmZMse0/dEJSu4PiUnC64kDPK 9F1S8pO7CiBqp9kjwz05aJipxyoKlFVrMxRLBDSOwWeF3JQYf96PTG892Xl6SOh8+Q5Rm4 bgV9GgMXO7XlxIS5cmmcYYeyBq7hxNQ=

Hi, Test,

This is a test from **acme.com.

Thanks,

John

And also, I got 10/10 in mail-test:

SpamAssassin likes you
-0.1 | DKIM_SIGNED | Message has a DKIM or DK signature, not necessarily valid
    This negative score will become positive if the signature is validated. See immediately below.
0.1 | DKIM_VALID | Message has at least one valid DKIM or DK signature
    Great! Your signature is valid
0.1 | DKIM_VALID_AU | Message has a valid DKIM or DK signature from author's domain
   Great! Your signature is valid and it's coming from your domain name
0.001 | SPF_PASS | SPF: sender matches SPF record
    Great! Your SPF is valid
0.01 | T_RP_MATCHES_RCVD | Envelope sender domain matches handover relay domain

Can you look into it? Thanks!

sknight80 commented 6 years ago

I have similar result from mail-tester.com as @jaguar1975cn . Here is my gmail log (after I added to the whitelist)

Delivered-To: *****+example@gmail.com
Received: by 10.2.182.1 with SMTP id h1csp2475212jam;
        Wed, 31 Jan 2018 00:09:03 -0800 (PST)
X-Google-Smtp-Source: AH8x227rMPT/Z233SW2RtCEpV+k+csmZnztcpyHg0xhYA/c7ofo+4tTiA49Wg7FeY8X3Jg0MEXHy
X-Received: by 10.28.100.213 with SMTP id y204mr23694969wmb.24.1517386143369;
        Wed, 31 Jan 2018 00:09:03 -0800 (PST)
ARC-Seal: i=2; a=rsa-sha256; t=1517386143; cv=pass;
        d=google.com; s=arc-20160816;
        b=AediJZPx8IznNRofv7N30V5MT6hQ1dKnsnbTiAZiJnZjgzZ6uhSVosRgeiN+F7V2gD
         2Tzi/4qAYqWL3JCh9rKNV4JbOM6lANEUpy4kPKl848KU3f4vSr+Bv+3nmvmP89cMG7zw
         A1XilLuIHlYzAo5jT/KBj7gGwACDEcq8agbMBjEPYNSAyXWTv9U/2phQsQT/DIMU+WPA
         ZO/uACsdtsW8VRkbdb1tG9TYfh7jTU6pbVobhjRV6LYRa1vWuYRcxH03TUYNxTpiCIgh
         8OUpQL57Eoy3nNfS4YVhTwKMfVdQvVlenHurADqbrshljqYHM+vDWrcU1jKJJ5Ykj1NM
         Qc5A==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=dkim-signature:arc-authentication-results:arc-message-signature
         :content-transfer-encoding:mime-version:message-id:subject:from:to
         :date:arc-authentication-results;
        bh=hhOey6ul9wcOCWQgTO6SGpLYGfTL7U+ctjiVIAD+Woo=;
        b=Up8Lwpja5vQZP/QbWGXmpWw2pNABzeiDTKVyrdvpbkhuTGGJ94nS2pR5JoCW+xoE6J
         j8AQyToqMbcli9hsXJjyUYUXSdvuHg+74u921glcGv96O8igCO/fEDae8OVZo5NvBQ2T
         wBON55pVPIS30mxvmhEgoceAJcbEqFh2gUBtfgv6EGgb0oRA5vqUpLScQivYWzNaz7BN
         OyHAceBDs6bo1uqm8QEO6+CUJqK7wLn5hUSG6IOexwRtpZf2Fsz6xOieTTIfC7oT2e7M
         jigy9pghmMfnhvHH/lHzfk+McBGFa/N8oHNh4mCgYj9CAZ3UMyhWeixJqEyF2d9jKlwI
         l95A==
ARC-Authentication-Results: i=2; mx.google.com;
       dkim=pass header.i=@<MYFQDN>.com header.s=mail header.b=PpnUMhgk;
       arc=pass (i=1);
       spf=pass (google.com: domain of shop@<MYFQDN>.com designates x.x.x.x as permitted sender) smtp.mailfrom=shop@<MYFQDN>.com
Return-Path: <shop@<MYFQDN>.com>
Received: from mail.<MYAWESOMEMAILSERVER> (mail.******. [x.x.x.x])
        by mx.google.com with ESMTPS id 134si10791969wmt.166.2018.01.31.00.09.03
        for <*****+example@gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
        Wed, 31 Jan 2018 00:09:03 -0800 (PST)
Received-SPF: pass (google.com: domain of shop@<MYFQDN>.com designates x.x.x.x as permitted sender) client-ip=x.x.x.x;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@<MYFQDN>.com header.s=mail header.b=PpnUMhgk;
       arc=pass (i=1);
       spf=pass (google.com: domain of shop@<MYFQDN>.com designates x.x.x.x as permitted sender) smtp.mailfrom=shop@<MYFQDN>.com
Received: from authenticated-user (mail.<MYAWESOMEMAILSERVER> [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: shop@<MYFQDN>.com) by mail.<MYAWESOMEMAILSERVER> (Postfix) with ESMTPSA id 5F21D1E753; Wed, 31 Jan 2018 08:09:02 +0000 (UTC)
Date: Wed, 31 Jan 2018 08:09:02 +0000
To: ************, *******+example@gmail.com
From: "Example Wordpress" <shop@<MYFQDN>.com>
Subject: Sucuri Alert, <MYFQDN>.com, Post Update, y.y.y.y
Message-ID: <dec20761f803f1ad5ada55dc3e8ced26@<MYFQDN>.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=<MYFQDN>.com; s=mail; t=1517386142; h=from:subject:date:message-id:to:mime-version:content-type:content-transfer-encoding; bh=hhOey6ul9wcOCWQgTO6SGpLYGfTL7U+ctjiVIAD+Woo=; b=rOMBrzCpMUCrgz7OmWtPMaEY6hnUVmzJ6c5rmkxUc/pWbwBQtDNeMWkk0tbH+6gZCBpfHQ xo1cuSlB6v1R7zXgmJwUiho1pv1bpMz6bVXV1J/px3i37RtjSrhOZahbG4mIL5FNXAsBzb z4LiunoyaOizGgOXrRNx5hg8sPpoXKs=
ARC-Authentication-Results: i=1; auth=pass smtp.auth=shop@<MYFQDN>.com smtp.mailfrom=shop@<MYFQDN>.com
ARC-Seal: i=1; s=mail; d=<MYFQDN>.com; t=1517386142; a=rsa-sha256; cv=none; b=Zxw/vo/BaQiraPpVXGMpGnojY9RVR6hyDS4MZJQQxERkrfzhyTBqOxR+n00Av7HtIOhtBHyqyx7IWEKhmg9nR75z9XBsc2QdkvFGEEurzWN1IzokhH6ptAbVW0eAtyJoheCACZ1TfOlqM8mAvB35z0IbpKlmpiUsnMOzfSLivx4=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=<MYFQDN>.com; s=mail; t=1517386142; h=from:subject:date:message-id:to:mime-version:content-type:content-transfer-encoding; bh=hhOey6ul9wcOCWQgTO6SGpLYGfTL7U+ctjiVIAD+Woo=; b=PpnUMhgkDO2EjBG4mRocYMhK8TtJQrGcYhLvlanfpPDU476J1e81rLz9ngDaPGfIS14WpO I/w3+YL1lLB1+s0CvNmVPupE3sdQrO1VI7BhPXHH7C5Yk7B7y42YO6l0ink8D52cbn9Lw2 nfIm30FkAz1WeC2EE3sqs7qcCdC5fn8=
hardware commented 6 years ago

I tested with 2 domains without any problem.

@sknight80 Nothing wrong in your mail header. Contact the gmail support and add your domain on Postmaster Tools to have a feedback loop and spam reports.

@jaguar1975cn can you send me a mail on my email address and check-auth@verifier.port25.com ?

Important steps :

jaguar1975cn commented 6 years ago

I mapped these aliases to an existing mailbox,

abuse@domain.tld    admin@domain.tld
hostmaster@domain.tld   admin@domain.tld
postmaster@domain.tld   admin@domain.tld
webmaster@domain.tld    admin@domain.tld

Now I can see the authentication header shows "pass" now:

ARC-Authentication-Results: i=2; mx.google.com;
       dkim=pass header.i=@domain.tld header.s=mail header.b=G9svg8cV;
       arc=pass (i=1);
       spf=pass (google.com: domain of user@domain.tld designates x.x.x.x as permitted sender) smtp.mailfrom=user@domain.tld;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=domain.tld

I don't know if adding aliases related to the result.

But my emails are still going to the spam folder in Gmail.

I will send an email to your mailbox, please have a look.

jaguar1975cn commented 6 years ago

Got the result from check-auth@verifier.port25.com:

==========================================================
Summary of Results
==========================================================
SPF check:          pass
"iprev" check:      pass
DKIM check:         pass
SpamAssassin check: ham

----------------------------------------------------------
SpamAssassin check details:
----------------------------------------------------------
SpamAssassin v3.4.1 (2015-04-28)

Result:         ham (-0.1 points, 5.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 0.0 NO_DNS_FOR_FROM        DNS: Envelope sender has no MX or A DNS records
-0.0 T_RP_MATCHES_RCVD      Envelope sender domain matches handover relay
                            domain
 0.0 T_SPF_TEMPERROR        SPF: test of record failed (temperror)
 0.0 T_SPF_HELO_TEMPERROR   SPF: test of HELO record failed (temperror)
-0.0 BAYES_20               BODY: Bayes spam probability is 5 to 20%
                            [score: 0.0790]
-0.1 DKIM_VALID_AU          Message has a valid DKIM or DK signature from author's
                            domain
 0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily valid
-0.1 DKIM_VALID             Message has at least one valid DKIM or DK signature

Did I miss something here? I configured DNS according to your instruction in the readme. And the mail-test gave me 10/10 score. Why the spam check failed?

hardware commented 6 years ago

Did I miss something here ?

I don't think so.

Why the spam check failed ?

The test didn't failed, all checks pass, you can ignore the SPF temperror.

The same thing I said to @sknight80, contact the gmail support and add your domain on Postmaster Tools to have a feedback loop and spam reports. Build up a good sender reputation can take time.

I don't know if adding aliases related to the result.

No, it's not related to your previous result.

sknight80 commented 6 years ago

Thank you @hardware for the quick response. I signed up for Google Postmaster tool. I hope I will be able to see any extra information from them. I will check out the remaining URL that you mentioned.

jaguar1975cn commented 6 years ago

@hardware Thank you very much, it seems we have a long list to go through. :cry:

jaguar1975cn commented 6 years ago

It seems the dkim still not quite right, I got simular report from google and yahoo etc:

I can see lots of failures:

<dkim>fail</dkim>

But it is weird, not all of them failed, got some "pass" as well.

Google:

<?xml version="1.0" encoding="UTF-8"?>
<feedback>
  <report_metadata>
    <org_name>google.com</org_name>
    <email>noreply-dmarc-support@google.com</email>
    <extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info>
    <report_id>9573019302659315740</report_id>
    <date_range>
      <begin>1517356800</begin>
      <end>1517443199</end>
    </date_range>
  </report_metadata>
  <policy_published>
    <domain>domain.tld</domain>
    <adkim>s</adkim>
    <aspf>s</aspf>
    <p>reject</p>
    <sp>reject</sp>
    <pct>100</pct>
  </policy_published>
  <record>
    <row>
      <source_ip>x.x.x.x</source_ip>
      <count>40</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>pass</dkim>
        <spf>pass</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>domain.tld</header_from>
    </identifiers>
    <auth_results>
      <dkim>
        <domain>domain.tld</domain>
        <result>pass</result>
        <selector>mail</selector>
      </dkim>
      <spf>
        <domain>domain.tld</domain>
        <result>pass</result>
      </spf>
    </auth_results>
  </record>
  <record>
    <row>
      <source_ip>x.x.x.x</source_ip>
      <count>24</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>fail</dkim>
        <spf>pass</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>domain.tld</header_from>
    </identifiers>
    <auth_results>
      <dkim>
        <domain>domain.tld</domain>
        <result>fail</result>
        <selector>mail</selector>
      </dkim>
      <spf>
        <domain>domain.tld</domain>
        <result>pass</result>
      </spf>
    </auth_results>
  </record>
  <record>
    <row>
      <source_ip>x.x.x.x</source_ip>
      <count>15</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>fail</dkim>
        <spf>pass</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>domain.tld</header_from>
    </identifiers>
    <auth_results>
      <spf>
        <domain>domain.tld</domain>
        <result>pass</result>
      </spf>
    </auth_results>
  </record>
</feedback>

Yahoo:

<?xml version="1.0" encoding="UTF-8"?>
<feedback>
  <report_metadata>
    <org_name>Yahoo! Inc.</org_name>
    <email>postmaster@dmarc.yahoo.com</email>
    <report_id>1517449201.964104</report_id>
    <date_range>
      <begin>1517356800</begin>
      <end>1517443199</end>
    </date_range>
  </report_metadata>
  <policy_published>
    <domain>domain.tld</domain>
    <adkim>s</adkim>
    <aspf>s</aspf>
    <p>reject</p>
    <pct>100</pct>
  </policy_published>
  <record>
    <row>
      <source_ip>x.x.x.x</source_ip>
      <count>4</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>fail</dkim>
        <spf>pass</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>domain.tld</header_from>
    </identifiers>
    <auth_results>
      <dkim>
        <domain>domain.tld</domain>
        <result>neutral</result>
      </dkim>
      <spf>
        <domain>domain.tld</domain>
        <result>pass</result>
      </spf>
    </auth_results>
  </record>
  <record>
    <row>
      <source_ip>x.x.x.x</source_ip>
      <count>8</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>pass</dkim>
        <spf>pass</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>domain.tld</header_from>
    </identifiers>
    <auth_results>
      <dkim>
        <domain>domain.tld</domain>
        <result>pass</result>
      </dkim>
      <spf>
        <domain>domain.tld</domain>
        <result>pass</result>
      </spf>
    </auth_results>
  </record>
</feedback>

It seems the dkim signature is not always correct.

hardware commented 6 years ago

Aggregate reports allows to see legitimate and/or fraudulent email and abuses on your domain by source ip. So you can have multiple record nodes with different results (failed or pass).

More info : https://blog.returnpath.com/how-to-read-your-first-dmarc-reports-part-1/

jaguar1975cn commented 6 years ago

Thank you @hardware. But the question is: the IP address is exactly my server's IP address, why it still has <dkim>fail</dkim> This is just a test server, a few of my co-workers are testing it. I don't believe they will use fake "from" address in their test. By default, this docker will disallow unauthenticated relaying, isn't it? It means nobody can connect to the SMTP server without login then send email to Gmail.

Sorry to be painful, appreciate all the help you have provided.

hardware commented 6 years ago

By default, this docker will disallow unauthenticated relaying, isn't it ?

Absolutely.

the IP address is exactly my server's IP address

When did you install your mail server ? Maybe your DNS record wasn't properly configured or spread around the world during the first mailings. Wait some other reports from Google and Yahoo. Do you have any test app that sends e-mails with your mailserver ? Like a web app (PHP, node.js...etc), CMS, forum, e-commerce platform...etc ? Maybe you send unsigned email with this IP with something other than my docker image.

Anyway, my mailserver validated your mail setup and DNS records 2 days ago with a very good score :)

navossoc commented 6 years ago

@jaguar1975cn I'm using my DMARC record with adkim=r and aspf=r (both relaxed). I had a lot of issues of emails being rewritten that were invalidating the DKIM signature.

If you want to read your DMARC reports in a more human readable way, try this: https://dmarcian.com/dmarc-inspector/

They offer a free trial, it should be time enough to you set up your mail server properly.

jaguar1975cn commented 6 years ago

@hardware I installed the mail server at 30th of January, perhaps you are right, I didn't fully configure the DNS properly when I did the first a few test. But only after a few moment (less than one hour), I corrected them all. Perhaps that's the reason why Google had bad record for it?

We do have a python program, which is using the SMTP server in the docker, via port 587. I think that should be considered as sending email with the docker image. There is no other program sends email via this IP.

jaguar1975cn commented 6 years ago

@navossoc It's a good idea to try adkim=r and aspf=r. I will test that, thanks!

jaguar1975cn commented 6 years ago

Although @hardware has closed this issue, I think I need to report back. Changed to "adkim=r and aspf=r" has no difference, Google still reports "fail" everyday. Will try to contact Google to solve it.

Thanks!

navossoc commented 6 years ago

@jaguar1975cn Just to be sure... Are you using the right private/public key set?

Did you check your TXT record if there are no spaces or quotes in the wrong place?

[]'s

1n5aN1aC commented 6 years ago

Just to add- try querying your dkim directly from DNS, and verifying its right.

Some hosts, (mine) limit the number of characters in the record. I had to drop to a lower number of bits to have mine not be truncated, and actually work.

On Feb 8, 2018, at 6:03 AM, Rafael Cossovan notifications@github.com wrote:

@jaguar1975cn Just to be sure... Are you using the right private/public key set?

Did you check your TXT record if there are no spaces or quotes in the wrong place?

[]'s

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

sknight80 commented 6 years ago

I know, this question would a noob question, but how could you track your email server spam rate and other things via different services that @jaguar1975cn mentioned? I am not able to setup google postmaster to track my outgoing email rate. I registered my domain there (status is verified), but the report is totally empty. However, I am sending out a couple of emails under my domain. Could some help me either in private or here? (email: knight.secret@gmail.com)

jaguar1975cn commented 6 years ago

Here is my result from dig:

299 IN  TXT "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDSHQ2qarMlPVgUQV+U/iXW5mRyo3rrFB3g7wboVvZmkydNJVTzYhy2pflvMEQtJtePP0XOk7jEQ3nS1yl6eKIHIZH36pJe9iyk+ke9pqdD6t7G5bLiwSxVspY+pAGG5URcgtRtSbXJxtF4Uzs6jf4Mw6qkKHg8qwG1ySKbkP3/WQIDAQAB"

It looks fine to me. It matches the value in public.key.

The problem is I tested the DKIM everywhere, they all reported "DKIM check: pass", and even in the gmail's show "Show original", I can see 

ARC-Authentication-Results: i=2; mx.google.com;
       dkim=pass header.i=@domain.tld header.s=mail header.b=P1cMT6XL;
       arc=pass (i=1);
       spf=pass (google.com: domain of user@domain.tld designates x.x.x.x as permitted sender) smtp.mailfrom=user@domain.tld;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=domain.tld

But the email still went into the spam folder. What I am thinking now is maybe this domain is in some kind of blacklist in Google. I will contact Google ASAP.

jaguar1975cn commented 6 years ago

@sknight80 I don't know much about it either. Last week I tried https://app.glockapps.com, it seems you can only use it for three times. After that, you need to pay for it. But it does give some useful statistics about mainstream email providers.

sknight80 commented 6 years ago

Thank you, @jaguar1975cn . I will check this link. I need to set up a new mail server for one of my projects and I feel that this is a good time to nail everything down around email sending.

hardware commented 6 years ago

What I am thinking now is maybe this domain is in some kind of blacklist in Google.

Not your domain, probably your server IP or some IP range of your hosting provider.

denji commented 6 years ago

rspamd use scheme of sending DMARC Aggregate Reports (XML) for sending to other ISP's.

sknight80 commented 6 years ago

Thank you, @denji !

jaguar1975cn commented 6 years ago

@hardware After a few days' investigation, I have found the reason why Google complains the DKIM signature is incorrect. As I mentioned earlier, there is a python program which is using this SMTP server to send out emails to Gmail, however, this python program didn't set the email envelope properly. Through the spam.domain.tld, I can see a lots of records show:

HFILTER_URL_ONLY(2.099083)[0.95412844036697]
MISSING_MID(2.5)
HTML_SHORT_LINK_IMG_1(2)
MISSING_DATE(1)
...

which cause rspamd gave some quite high spam score like 7.58 / 20, it then executed action "add header" or even "rewrite subject" on those emails.

Somehow, after those actions were executed on those emails, the DKIM signatures were incorrect. After we corrected the python program, now rspamd shows "no action" on all outgoing emails. Since then, the reports sent back by Google shows 100% <dkim>pass</dkim>.

My question here is, after rspamd modified the email header or subject, it will not re-calculate the DKIM signature?

denji commented 6 years ago

My question here is, after rspamd modified the email header or subject, it will not re-calculate the DKIM signature?

@jaguar1975cn The best way send questions to rspamd: https://github.com/vstakhov/rspamd/issues/new

jaguar1975cn commented 6 years ago

@denji Thanks!

hardware commented 6 years ago

As I mentioned earlier, there is a python program which is using this SMTP server to send out emails to Gmail, however, this python program didn't set the email envelope properly

Messages with empty envelope from are not signed by rspamd in current configuration of my docker image :

https://github.com/hardware/mailserver/blob/b58372bac82683eea7fcf0c0a8cc9273acdf718d/rootfs/etc/rspamd/local.d/dkim_signing.conf#L1

But the default configuration allows it :

https://github.com/vstakhov/rspamd/blob/e3e15e12d0bd14a907b406feb870f4e4870f3cb8/conf/modules.d/dkim_signing.conf#L30

Documentation :

https://rspamd.com/doc/modules/dkim_signing.html

My question here is, after rspamd modified the email header or subject, it will not re-calculate the DKIM signature ?

Rspamd can't sign them due to allow_envfrom_empty value.

jaguar1975cn commented 6 years ago

The envelope problem I mentioned was MISSING_MID and MISSING_DATE, the "from" field was correct.