search

jeboehm / docker-mailserver

Docker Mailserver based on the famous ISPMail guide
MIT License
356 stars 93 forks source link

DKIM example please #97

Closed kklepper closed 4 years ago

kklepper commented 4 years ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I read in https://github.com/jeboehm/docker-mailserver/wiki/Feature:-DKIM

After generating the private key,

Well, I hit this key and asked myself where to find this private key?

Describe the solution you'd like A clear and concise description of what you want to happen.

I'd like to see a simple step-by-step instruction like the one you did on your main page.

You see, is very fashionable to be brief to the extent that nobody who doesn't know the solution can make sense of the statement.

This doesn't help. If you take the pain to maintain a wiki, which is very honorable, please be verbose.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

jeboehm commented 4 years ago
Bildschirmfoto 2020-06-09 um 12 33 54

You're right that this is a unprecise explaination. mailserver-admin will assist you in setting everything up. You can find all information below the form where you can enable/disable DKIM. Create a subdomain with the exact name as stated unter "Domain:", set the record type to TXT, and copy the "expected value".

Bildschirmfoto 2020-06-09 um 12 39 52

If the subdomain was set up properly (existing as a TXT record), "Domain Key found" will change to Yes, "Key valid" will change to Yes when the private key, which is saved in the database, matches the public key in your DKIM TXT record.

As always: You're very welcome to make improvements to the documentation and code.

kklepper commented 4 years ago

Thank you, I found out not much later, and if you know which button to click in which sequence, it's really easy.

I'll try to improve the wiki text.

kklepper commented 4 years ago

Well, it just took a few seconds until I was stuck again. I had a look at your wiki text, switched to the admin interface, clicked on DKIM and saw that all buttons were red.

I'm sure that my DNS entry is correct -- it is pretty clear from the admin interface how to define it, and I checked with other resources to see that there is no doubt about it.

I made the addition 12 hours ago and expected that the admin interface could read it by now, if I understood you correctly. Obviously I was wrong. I will wait for another extended time and try again.

jeboehm commented 4 years ago

dig TXT 2020._domainkey.example.com

This tool might help you to debug a little bit. You should see something in the answer section:

;; ANSWER SECTION:
2020._domainkey.example.com. 86400  IN TXT  "v=DKIM1; h=sha256; t=s; p=MIIBIjANBgkqhkiG9w0BAQEFAxxxxxAzReFRcxTFUla/szx3/hREwepT1oIBR0b7eUoqtvga200xxxxxxx" ""

You can even check the cached results of different nameservers: dig @8.8.8.8 TXT 2020._domainkey.example.com

kklepper commented 4 years ago

Thank you. I will resume shortly.

Next my entry from yesterday which I did not send yet; I found something which might explain the difficulties.

kklepper commented 4 years ago

https://www.dmarcanalyzer.com/ says there is no DKIM record on the given domain ...

https://www.appmaildev.com/de/dkim works by sending a mail to them -- they detect my helper domain, which indeed has no DKIM yet. But it says also that DMARC fails, whereas dmarcanalyzer.com says ok for my helper domain DMARC as well.

So I added that helper domain to webmail manager using the admin interface, and got 

Oops! An Error Occurred
The server returned a "500 Internal Server Error".
Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.

I thought it may be because I did not add a user first. But adding a user did not help.

I stopped and restarted docker-mailserver , to no avail. What can I do?

I made sure I can send and receive from this new user account on the helper domain. I know that I will get an error, if I send to it... Well, I did not get an error, but the mail did not arrive.

After a while it worked again, I could add DKIM data to the helper domain... not really, as they truncate the name part. Ok, we'll see if it works nevertheless. Freenom.com capitalizes all names, too.

Now I found an obvious error in manager:

image

I removed this backslash manually in all 3 domain records and we'll see in a while if it was the reason for the failure.

kklepper commented 4 years ago

Just a short notice: this whole stuff drives me crazy.

SPF is not found, DKIM and DMARC work with https://www.appmaildev.com/de/dkim (SPF: None, DKIM: pass, DMARC: pass, see below). That is mostly good.

DKIM delivers an error with https://mxtoolbox.com/SuperTool.aspx as well as with webmail admin (Oops! An Error Occurred The server returned a "500 Internal Server Error". Something is broken. Please let us know what you were doing when this error occurred. We will fix it as soon as possible. Sorry for any inconvenience caused.)

That is outright bad.

DMARC works with mxtoolbox.com as well, but at the same time the nameserver used (which is the original nameserver) says:

root@IONOS_2: /root # nslookup -type=TXT _dmarc ns1.nameserverservice.de
Server:         ns1.nameserverservice.de
Address:        217.172.176.222#53

Non-authoritative answer:
*** Can't find _dmarc: No answer

Mind you, it did work before.

Oh my, I forgot the trick... You must add the domain name.

I found that in https://www.godaddy.com/community/Managing-Domains/TXT-record-not-found/m-p/129225#M25042

Now nslookup works:

root@IONOS_2: /root # nslookup -type=TXT _dmarc.uuu.tld ns1.nameserverservice.de
Server:         ns1.nameserverservice.de
Address:        217.172.176.222#53

_dmarc.uuu.tld       text = "v=DMARC1;p=quarantine;pct=100;rua=mailto:wp@uuu.tld;ruf=mailto:wp@uuu.tld;adkim=s;aspf=r"

and likewise

spf.uuu.tld text = "v=spf1 mx a ip4:217.160.242.13/32 a:mail.zzz.tld -all"

- DKIM

root@IONOS_2: /root # nslookup -type=TXT 2020._domainkey.uuu.tld ns1.nameserverservice.de Server: ns1.nameserverservice.de Address: 217.172.176.222#53

2020._domainkey.uuu.tld text = "v=DKIM1;h=sha256;t=s;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAnDV/ssJv41Ce+8GU1q4706A47UMGBk/ZDq8BBUBQJ3DEy+wjJJJ8LvXEqFkQ" "uCbH+Foh9zhEG6kflXovg33nBihicFPeJO6tJq/cPZNI64t6d9OdbYfGHFy4DOVzTieriNpcgEu9dHOInfVvziMU6PzouH3haFqAuCO6f/Nl0MX57rykLvgjH9gET9z" "M7P0fLfVezgqeLzXXVo0tCJ65FpiTb/ugCwCZJ3h8PaX3LfM1FCQlIXiTcgqANfU3JX+d6lfNWENfdJ6a94jahcP38XlefZkKon81OEXFqegGZv9YWvfTa8L1Xx+mQ0" "V3ArmZh1DNpVWRH3NQf0EAi3bgIQIDAQAB"

So here we are... `nslookup` sees all SPF, DKIM, DMARC.

-----------------------------------------

But SPF cannot be found no matter what except `nslookup` and directly testing with https://www.kitterman.com/spf/test5.py:

`syntax ok: SPF record passed validation test with pySPF (Python SPF library)!`

![image](https://user-images.githubusercontent.com/6130319/84418794-f69e0d80-ac17-11ea-92c9-26f4f041f49d.png)

With most test programs, I cannot verify DKIM, but https://www.appmaildev.com/de/dkim says:

_dmarc.uuu.tld: v=DMARC1;p=quarantine;pct=100;rua=mailto:wp@uuu.tld;ruf=mailto:wp@uuu.tld;adkim=s;aspf=r
Received-SPF: none (appmaildev.com: domain of wp@uuu.tld does not designates permitted sender) client-ip=217.160.241.84
Authentication-Results: appmaildev.com;
    dkim=pass header.d=uuu.tld;
    spf=none (appmaildev.com: domain of wp@uuu.tld does not designates permitted sender) client-ip=217.160.241.84;
    dmarc=pass (adkim=s aspf=r p=quarantine) header.from=uuu.tld;

which is a part of the email header, so DKIM should work, right? But again `spf none`.

I tried to introduce `\` like roundcube suggests, and with it I got a match between `Expected value` and `Current value`, but then a fat warning:

`DKIM is enabled but not properly set up. Your mails may be rejected on the receivers side. Check your DNS settings.`

Do you understand why I grow desparate?

As for DKIM, some programs look for the signature (`2020` here), some for signature plus `._domainkey`, others for the whole string as given by roundcube `2020._domainkey.uuu.tld`. 

Can't they agree about the syntax? And what about SPF -- what name to choose here? I tried 

- spf
- _spf
- domain
- domain.
- @
- even no value at all

I run out of ideas and googling for hours gave me no answer.

----------------------------------------

Ok, I think I finally got it. Having made every mistake thinkable of and every combination thereof, the right syntax finally must appear.

Tomorrow I will try to sum up my experiences.
kklepper commented 4 years ago

https://www.appmaildev.com/de/dkim

_dmarc.uuu.tld: 
v=DMARC1;p=quarantine;pct=100;rua=mailto:wp@uuu.tld;ruf=mailto:wp@uuu.tld;adkim=s;aspf=r
Received-SPF: pass (appmaildev.com: domain of wp@uuu.tld designates 217.160.241.84 as permitted sender) client-ip=217.160.241.84
Authentication-Results: appmaildev.com;
    dkim=pass header.d=uuu.tld;
    spf=pass (appmaildev.com: domain of wp@uuu.tld designates 217.160.241.84 as permitted sender) client-ip=217.160.241.84;
    dmarc=pass (adkim=s aspf=r p=quarantine) header.from=uuu.tld;
jeboehm commented 4 years ago

Nice to see that things are starting to work well.

Do you know what the error in mailserver-admin was about?

kklepper commented 4 years ago

Still searching. Opened a ticket with provider. Their DNS does not seem to deliver TXT records.

Their phone support told me your syntax is wrong referring to dig syntax -- but it works, see below.

Their syntax does not, which may be the reason why mxtoolbox doesn't see a SPF record, if they use something along the lines.

root@IONOS_2: /root # dig google.com TXT @ns1.nameserverservice.de

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> google.com TXT @ns1.nameserverservice.de
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46582
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;google.com.                    IN      TXT

;; AUTHORITY SECTION:
.                       259200  IN      NS      a.root-servers.net.

;; Query time: 7 msec
;; SERVER: 217.172.176.222#53(217.172.176.222)
;; WHEN: Fri Jun 12 14:07:14 CEST 2020
;; MSG SIZE  rcvd: 59
root@IONOS_2: /root # dig google.com TXT @8.8.8.8

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> google.com TXT @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43758
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.                    IN      TXT

;; ANSWER SECTION:
google.com.             3599    IN      TXT     "globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8="
google.com.             299     IN      TXT     "docusign=05958488-4752-4ef2-95eb-aa7ba8a3bd0e"
google.com.             3599    IN      TXT     "facebook-domain-verification=22rm551cu4k0ab0bxsw536tlds4h95"
google.com.             3599    IN      TXT     "v=spf1 include:_spf.google.com ~all"
google.com.             299     IN      TXT     "docusign=1b0a6754-49b1-4db5-8540-d2c12664b289"

;; Query time: 14 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jun 12 14:07:26 CEST 2020
;; MSG SIZE  rcvd: 352

Their phone support told me your syntax is wrong -- but it works

root@IONOS_2: /root # dig @8.8.8.8 TXT spf.uuu.tld

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> @8.8.8.8 TXT spf.uuu.tld
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61603
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;spf.uuu.tld.                        IN      TXT

;; ANSWER SECTION:
spf.uuu.tld.         21599   IN      TXT     "v=spf1 mx a ip4:217.160.241.84/32 a:mail.zzz.tld a:mail.uuu.tld a:mail.xxx.tld -all"

;; Query time: 22 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jun 12 14:50:22 CEST 2020
;; MSG SIZE  rcvd: 147
root@IONOS_2: /root # dig @ns1.nameserverservice.de TXT spf.uuu.tld

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> @ns1.nameserverservice.de TXT spf.uuu.tld
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40406
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;spf.uuu.tld.                        IN      TXT

;; ANSWER SECTION:
spf.uuu.tld.         86400   IN      TXT     "v=spf1 mx a ip4:217.160.241.84/32 a:mail.zzz.tld a:mail.uuu.tld a:mail.xxx.tld -all"

;; AUTHORITY SECTION:
uuu.tld.             86400   IN      NS      ns1.nameserverservice.de.
uuu.tld.             86400   IN      NS      ns2.nameserverservice.de.

;; ADDITIONAL SECTION:
ns1.nameserverservice.de. 86400 IN      A       217.172.176.222
ns2.nameserverservice.de. 86400 IN      A       217.172.164.64

;; Query time: 7 msec
;; SERVER: 217.172.176.222#53(217.172.176.222)
;; WHEN: Fri Jun 12 14:50:47 CEST 2020
;; MSG SIZE  rcvd: 222
kklepper commented 4 years ago

Do you know what the error in mailserver-admin was about?

That's all I know:

image

I tried again:

kklepper commented 4 years ago

I suspected the error might be in the data, so I looked for the volume docker-mailserver_data-dkim

root@IONOS_2: /root/2proxy # docker volume inspect docker-mailserver_data-dkim
[
    {
        "CreatedAt": "2020-06-14T00:51:22+02:00",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "docker-mailserver",
            "com.docker.compose.version": "1.25.5",
            "com.docker.compose.volume": "data-dkim"
        },
        "Mountpoint": "/var/lib/docker/volumes/docker-mailserver_data-dkim/_data",
        "Name": "docker-mailserver_data-dkim",
        "Options": null,
        "Scope": "local"
    }
]

root@IONOS_2: /root/2proxy # ls -latr /var/lib/docker/volumes/docker-mailserver_data-dkim/_data
total 8
drwxr-xr-x 3 root root   19 Jun  8 12:18 ..
-rw-rw-rw- 1 root root 1704 Jun 14 00:51 example.com.2020.key
drwxrwxrwx 2   82 root   59 Jun 14 00:51 .
-rw-rw-rw- 1 root root   16 Jun 14 00:51 dkim_selectors.map

I deleted these files -- no cure. I restarted docker-mailserver -- these files were recreated automatically.

Well, these are just what you would expect. Who recreates them? I had set 3 of them, for each of the 3 domains, but only 1 is there.

kklepper commented 4 years ago

I was wrong!

Now no error, but 3 entries, one of which all green (example.com).

image

Good start! This is the domain I tested with and got all pass.

kklepper commented 4 years ago

I think I got it.

Checking again, I got error 500:

http://kklepper.tk:81/?entity=DKIM&action=list&menuIndex=4&submenuIndex=-1
The server returned a "500 Internal Server Error".

Fix: rm -f /var/lib/docker/volumes/docker-mailserver_data-dkim/_data/*; 2md;2mu (wipe data, docker-mailserver down, docker-mailserver up)

Surprisingly, although the files in the docker volume are wiped out, the data is again present after restart. How come?

image

So DKIM is ok with roundcube.

And with https://dmarcian.com/, but not with mxtoolbox:

https://mxtoolbox.com/SuperTool.aspx?action=dkim%3azzz.tld&run=toolpage

An error has occurred with your lookup. Please try again.

The support of my nameserver provider told me :

Our DNS system cannot handle long DNS entries and splits them. Currently we cannot tell you when this problem can be solved.

Also, all 3 e-mail accounts pass the test at https://www.appmaildev.com/de/dkim with SPF, DMARC, DKIM.

So that's great. How did I do it?

Actually I don't know. For example with the last problem, SPF, 7 days ago I wrote (https://github.com/jeboehm/docker-mailserver/issues/97#issuecomment-642967296):

Can't they agree about the syntax? And what about SPF -- what name to choose here? I tried

  • spf
  • _spf
  • domain
  • domain.
  • @
  • even no value at all

Finally I contacted the support of my nameserver provider and they told me that no value at all is correct for ´SPF`, and indeed, when I tried once more, it worked. Why didn't it work before?

Now for the record, as far as I remember, the following applies:

My DNS provider has a form with 4 fields:

Subdomain type priority target
TXT v=spf1 mx a ip4:123.234.123.234/32 a:kkk.tld a:zzz.tld a:uuu.tld -all
2020._domainkey TXT v=DKIM1;h=sha256;t=s;p=MIIB...the whole public key as provided by roundcube...DAQAB
_dmarc TXT v=DMARC1;p=quarantine;pct=100;rua=mailto:rua@uuu.tld;ruf=mailto:ruf@uuu.tld;adkim=s;aspf=r

Adding the domain name to the token was the reason why I couldn't verify SPF with dig, but with nslookup. Usually the domain name is added automatically, as mentioned in https://github.com/jeboehm/docker-mailserver/issues/97#issuecomment-642967296.

Inspecting a domain I didn't transfer yet which was configured with Plesk, I saw that they give visual evidence of this fact:

image

Next I will try to put this information to the appropriate docker-mailserver wiki section.

kklepper commented 4 years ago

I think I can close this now.