docker-mailserver / docker-mailserver

Production-ready fullstack but simple mail server (SMTP, IMAP, LDAP, Antispam, Antivirus, etc.) running inside a container.
https://docker-mailserver.github.io/docker-mailserver/latest/
MIT License
14.69k stars 1.83k forks source link

other: Accepting incoming mail for `[domain-list]` #3602

Closed BAAAASS closed 1 year ago

BAAAASS commented 1 year ago

Subject

I would like some feedback concerning a use case

Description

I have a question about the configuration of my mail server. It seems that I am unable to receive e-mail for a legitimate internal user.

This user, let's call him xyz@domain.tld can connect via IMAP and send e-mail to abc@google.com. abc@google.com receives the e-mail and tries to reply, except it never gets received by xyz@domain.tld.

Troubleshooting:

The port is open, and the correct service is listening on the port, confirmed using: https://canyouseeme.org.

When I use https://smtpserver.com/smtptest to test the actual mail response when abc@google.com sends to a legitimate user xyz@domain.tld, I get the following response:

554 5.7.1 : Client host rejected: Access denied

Config:

PERMIT_DOCKER=none
hostname: mail.domain.tld

Logs:

12:07:55 mail postfix/submission/smtpd[3341]: connect from unknown[77.73.32.213]
12:07:55 mail postfix/submission/smtpd[3341]: Anonymous TLS connection established from unknown[77.73.32.213]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384
26 12:07:55 mail postfix/trivial-rewrite[3348]: warning: do not list domain domain.tld in BOTH mydestination and virtual_mailbox_domains
12:07:55 mail postfix/submission/smtpd[3341]: NOQUEUE: reject: RCPT from unknown[77.73.32.213]: 554 5.7.1 <unknown[77.73.32.213]>: Client host rejected: Access denied; from=<abc@google.com> to=<xyz@domain.tld> proto=ESMTP helo=<77-73-32-213.32.73.77.in-addr.arpa>
12:07:56 mail postfix/submission/smtpd[3341]: lost connection after RCPT from unknown[77.73.32.213]
12:07:56 mail postfix/submission/smtpd[3341]: disconnect from unknown[77.73.32.213] ehlo=2 starttls=1 mail=1 rcpt=0/1 commands=4/5

Logically there should be list of domains to "accept", or least accept the existing users by default. However, after countless searches and reading through all of the documentation, it is still not clear what I am doing wrong.

Any help is greatly appreciated.

georglauterbach commented 1 year ago

26 12:07:55 mail postfix/trivial-rewrite[3348]: warning: do not list domain domain.tld in BOTH mydestination and virtual_mailbox_domains

Are you using LDAP? Are you manually adjusting Postfix's configuration?

I would like you to open a new issue and report this is a possible bug; this gives maintainers more information about your setup. Right now, I'd need to ask a lot :)

BAAAASS commented 1 year ago

I am not using LDAP. It seems that this was my isssue after-all: https://github.com/docker-mailserver/docker-mailserver/issues/1278#issuecomment-537258675 I kept on trying to ADD the correct hosts (which didn't work for some strange reason, but that is a different story). However, it seems I simply need to REMOVE the first one.

I think it is a good idea that a 'make or break' setting like that should be better controllable through environmental variables; especially for new deployments.

polarathene commented 1 year ago

It seems that this was my isssue after-all: #1278 (comment) I kept on trying to ADD the correct hosts (which didn't work for some strange reason, but that is a different story). However, it seems I simply need to REMOVE the first one.

That is why it's important to convey correct information with us.

If you use the hostname of mail.example.com, your mail address is compatible with user@example.com, but not user@mail.example.com as this is reserved for the internal system mail.

Your hostname for DMS is the mail server address and is not related to the mail accounts DMS will manage. It is common that you will manage user@example.com addresses and use that same domain for DMS to be reached via a subdomain such as mail.example.com for DNS MX record. However, you could also have user@whatever.com and many other mail domains for DMS to manage on a single DMS instance running at mail.example.com.

The error you encountered is due to a conflict in user accounts with that internal usage. Normally this will happen if you configure accounts to use the hostname, which is rare for user@mail.example.com but we do see it. Usually this error is for users that don't understand what I mentioned in the previous paragraph, and they assign DMS a hostname of example.com thinking that is required for DMS to support user@example.com, or for other reasons like sharing the TLS certificate.

When you overlap the hostname for DMS with a mail account address, you have to apply the workaround that you did. This will prevent system mail being delivered properly (you should get these failures logged, but the mail won't be delivered to know what it's content was). This is a non-issue if you've configured these accounts to deliver to the postmaster, but it's not uncommon for users to skip that step too.


However, after countless searches and reading through all of the documentation, it is still not clear what I am doing wrong.

It doesn't seem like you read all the documentation. There is a FAQ entry with the error you encountered, with the same advice that you found in the referenced workaround issue.

Perhaps we could better clarify / document the information in my reply for users? Is my information useful to you? If so what was most helpful and where do you think the best location in our docs would be for discovering that information when you run into this problem, or better avoid it (there's already lots to read however, so I think most users gloss over it until they hit a problem)

You have stated hostname: mail.domain.tld with an account xyz@domain.tld, was that actually the case?


I kept on trying to ADD the correct hosts (which didn't work for some strange reason, but that is a different story). However, it seems I simply need to REMOVE the first one.

Could you clarify? You shouldn't need to add/remove any hosts? (/etc/hosts?)

I think it is a good idea that a 'make or break' setting like that should be better controllable through environmental variables; especially for new deployments.

In the docs, if you visit our ENV vars page, the very first ENV is OVERRIDE_HOSTNAME.

This does as it implies, but is often misused/misunderstood due to above explanation with hostname for DMS vs mail account domains.

How would you expect such a setting to work differently than it already does? If I am correct, and your example account and hostname misrepresented your DMS config, then the problem was misconfiguration elsewhere and a setting to disable internal system mail support is disregarding that as a "fix".

I don't think we should encourage that.

gpapin commented 1 year ago

If you use the hostname of mail.example.com, your mail address is compatible with user@example.com, but not user@mail.example.com as this is reserved for the internal system mail.

Wow I've been struggling for hours and this right here is what made it tick. I think I shoot myself on the foot by adding more confusion because I chose my domain to be mail.domain.tld and therefore created user like so mailbox1@mail.domain.tld

polarathene commented 1 year ago

I think I shoot myself on the foot by adding more confusion because I chose my domain to be mail.domain.tld and therefore created user like so mailbox1@mail.domain.tld

Yes I thought your example didn't match the error log shared haha (EDIT: Whoops different user, sorry! 😅 )


How could we better communicate this to users in our docs? As you are new to them, do you remember where you were starting and if it'd have helped to mention this somewhere you would have seen it?

Or where you looked when you ran into the error and what would we could have done better with our documentation for you to find the solution quickly? Or do you think this information would have been accidentally missed due to the inexperience (skipping over the explanation like the similar FAQ entry I linked, because it didn't seem related to your problem at the time?).

It is a common issue, just a tricky one to improve our docs for :)

gpapin commented 11 months ago

Hi @polarathene sorry for the delay... To give some context to where I'm coming from.

I have limited knowledge of mail server. I had a mail module at uni where I had to setup a Postfix server but is was pretty basic and a long time (+10y) ago - hence why I went for this docker install that simplify things quite a bit 🙏 !

That being said I've had to setup the DNS config quite frequently so I'm familiar with DKIM, SPF, MX etc. I don't know exactly where would be the best place to add more details, but I know that after a quick look at the getting started guide, I went to [the docker-compose example](hostname: mail.example.com) and use it as a starting point.

Your comment that I highlighted in my previous reply explains it very good imo. I would add it in a the compose.yml file in the Basic Example above the hostname: mail.example.com I think that would really help people. Another helpful thing could be to have a tutorial where we see the server setup to send email for 2 completely different domains. This could also highlights, the differences between the internal email and the external ones.

polarathene commented 11 months ago

I know that after a quick look at the getting started guide, I went to [the docker-compose example](hostname: mail.example.com) and use it as a starting point.

Looks like you didn't have the link copied at the time you pasted here 😅 Which example? Was it the one in the main repo files?


I think that would really help people. Another helpful thing could be to have a tutorial where we see the server setup to send email for 2 completely different domains. This could also highlights, the differences between the internal email and the external ones.

Yeah that sounds good, I've been meaning to contribute a fully offline/local example that is a bit more complex but would run a local DNS container with two DMS instances with docker compose up, then the user can try stuff like that.

gpapin commented 11 months ago

Looks like you didn't have the link copied at the time you pasted here 😅 Which example? Was it the one in the main repo files?

Oops, yes that's the basic one I linked further down: https://docker-mailserver.github.io/docker-mailserver/latest/examples/tutorials/basic-installation/

Yeah that sounds good, I've been meaning to contribute a fully offline/local example that is a bit more complex but would run a local DNS container with two DMS instances with docker compose up, then the user can try stuff like that.

Sounds exciting !