huan / docker-simple-mail-forwarder

Simplest and Smallest Email Forward Service based on Docker.
https://hub.docker.com/r/zixia/simple-mail-forwarder/
Apache License 2.0
544 stars 88 forks source link

Google and other Mail Services do not receive my mail #128

Closed m-schoder closed 8 months ago

m-schoder commented 8 months ago

Google and other Services (like www.fhooe.at) do not deliver my forwarded mails, but services like www.temp-mail.org do.

I replaced my real domain with my-domain and my real gmail address with my.mail@gmail.com

My docker compose:

version: "3.5"

services:
  smf:
    container_name: mail-forwarder
    image: zixia/simple-mail-forwarder
    restart: always
    ports:
      - "25:25"
    environment:
      - SMF_CONFIG=@my-domain.at:my.mail@gmail.com
    volumes:
      - /opt/projects/mail_forwarder/volumes/logs:/var/log/postfix
      - /var/db/dkim:/var/db/dkim

docker compose up output:

root@netcup:~/mail-test# docker compose up
[+] Running 1/0
 ✔ Container mail-forwarder  Created                                                                                                                                  0.0s 
Attaching to mail-forwarder
mail-forwarder  | >> Chdir to /app...
mail-forwarder  |  ____  _                 _         __  __       _ _ 
mail-forwarder  | / ___|(_)_ __ ___  _ __ | | ___   |  \/  | __ _(_) |
mail-forwarder  | \___ \| | '_ ` _ \| '_ \| |/ _ \  | |\/| |/ _` | | |
mail-forwarder  |  ___) | | | | | | | |_) | |  __/  | |  | | (_| | | |
mail-forwarder  | |____/|_|_| |_| |_| .__/|_|\___|  |_|  |_|\__,_|_|_|
mail-forwarder  |                   |_|                               
mail-forwarder  |   _____                                _           
mail-forwarder  |  |  ___|__  _ ____      ____ _ _ __ __| | ___ _ __ 
mail-forwarder  |  | |_ / _ \| '__\ \ /\ / / _` | '__/ _` |/ _ \ '__|
mail-forwarder  |  |  _| (_) | |   \ V  V / (_| | | | (_| |  __/ |   
mail-forwarder  |  |_|  \___/|_|    \_/\_/ \__,_|_|  \__,_|\___|_|   
mail-forwarder  |                                                   
mail-forwarder  | 
mail-forwarder  | v1.4.7
mail-forwarder  | Source#3cd9076 Sat Oct 21 22:43:18 2023 +0000 * master
mail-forwarder  | Built on Sat Oct 21 22:43:27 UTC 2023 by buildkitsandbox
mail-forwarder  | 
mail-forwarder  | >> ENV SMF_DOMAIN not set.
mail-forwarder  | >> ENV SMF_CONFIG found. value:[@my-domain.at:my.mail@gmail.com]
mail-forwarder  | >> ARGV arguments found. value:[start]
mail-forwarder  | >> SMF_CONFIG found in ENV. use this settings for forward maps.
mail-forwarder  | >> Setting password[zw6jt5zf] for user @my-domain.at ...
mail-forwarder  | postmap: warning: /etc/postfix/virtual.db: duplicate entry: "@my-domain.at"
mail-forwarder  | >> Set hostname to my-domain.at
mail-forwarder  | Postfix logging configuration
mail-forwarder  | Postfix will use the default logging configuration: /dev/stdout
mail-forwarder  | Inserting my-domain.at data to /etc/opendkim/{KeyTable, SigningTable, TrustedHosts}
mail-forwarder  | OpenDKIM: this TXT record for my-domain.at should be present:
mail-forwarder  | default._domainkey
mail-forwarder  | 
mail-forwarder  | TXT
mail-forwarder  | 
mail-forwarder  | v=DKIM1;t=s;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAv76qvZfEcfjrvd1eADkPidUtkILwC2tJFcITtdB6M3abxGNI15rCA8mYEcO07jIe3lZkjNgM1osQjygX0UeTh9R8ufwC2DY75NweP0TDf13u+mcdgr0Gy2PwinDfX4aJ3xwT/PrJpLyws7bWFs21miNnsC2/2JiWDBs4Y8lXzxl0IBKmanqTrQ7MudOjMtC9uNy1NqTsTFecEE1ffseG7P+FJgrSx3iJ/z61bA6aGnCmfxo5gvP5HQGHRs6a7JOJB9IpSupN2qcR44DMgWA4VHYCAeHFjQFfLUwXO3eubyM1Km/IzsRJ9hI3s5ObdS8w1Oc8i26v3etDV0xRqCmz2QIDAQABConfiguring DKIM key settings in /etc/opendkim/opendkim.conf
mail-forwarder  | Postfix main.cf custom entries from SMF_POSTFIXMAIN_
mail-forwarder  | Postfix master.cf custom entries from SMF_POSTFIXMASTER_
mail-forwarder  | >> Start self-testing...
mail-forwarder  | 1..24
mail-forwarder  | ok 1 SMF_CONFIG exist
mail-forwarder  | ok 2 SMF_DOMAIN exist
mail-forwarder  | ok 3 virtual maping source is set
mail-forwarder  | ok 4 virtual maping data is set
mail-forwarder  | ok 5 virtual maping db is set
mail-forwarder  | ok 6 system hostname FQDN resolvable
mail-forwarder  | ok 7 postfix myhostname FQDN & resolvable
mail-forwarder  | ok 8 check other hostname setting
mail-forwarder  | ok 9 confirm postfix is running
mail-forwarder  | ok 10 confirm port 25 is open
mail-forwarder  | ok 11 crond is running # skip skip this for 0.3.0 -> 0.4.0
mail-forwarder  | ok 12 ESMTP STATTLS supported
mail-forwarder  | ok 13 ESMTP AUTH supported
mail-forwarder  | ok 14 ESMTP STARTTLS connect ok
mail-forwarder  | ok 15 create user testi@testo.com by password test
mail-forwarder  | ok 16 ESMTP AUTH by testi@testo.com/test
mail-forwarder  | ok 17 ESMTP TLS AUTH by testi@testo.com/test
mail-forwarder  | ok 18 deleting test user testi@testo.com
mail-forwarder  | ok 19 test DKIM keys
mail-forwarder  | ok 20 test custom main.cf entries
mail-forwarder  | ok 21 test custom master.cf entries
mail-forwarder  | ok 22 test default postfix logging configuration
mail-forwarder  | ok 23 test custom postfix logging configuration with an error
mail-forwarder  | ok 24 test custom postfix logging configuration
mail-forwarder  | >> Test PASSED
mail-forwarder  | 
mail-forwarder  | 
mail-forwarder  | >> CONGRATULATIONS! System is UP and You are SET!
mail-forwarder  | >> Powered by SMF - a Simple Mail Forwarder
mail-forwarder  | >> View in DockerHub: https://hub.docker.com/r/zixia/simple-mail-forwarder
mail-forwarder  | 
mail-forwarder  | 
mail-forwarder  | >> Init System for Servicing...
mail-forwarder  | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
mail-forwarder  | [s6-init] ensuring user provided files have correct perms...exited 0.
mail-forwarder  | [fix-attrs.d] applying ownership & permissions fixes...
mail-forwarder  | [fix-attrs.d] done.
mail-forwarder  | [cont-init.d] executing container initialization scripts...
mail-forwarder  | [cont-init.d] done.
mail-forwarder  | [services.d] starting services
mail-forwarder  | [2023-12-27T11:05:51.413716] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode. Please update it to use the syslog-ng 3.27 format at your time of convenience. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file; config-version='3.9'
mail-forwarder  | [services.d] done.
mail-forwarder  | Dec 27 11:05:51 b0d4029bbafd syslog-ng[898]: syslog-ng starting up; version='3.27.1'
mail-forwarder  | Dec 27 11:05:51 b0d4029bbafd syslog-ng[898]: WARNING: log-fifo-size() works differently starting with syslog-ng 3.22 to avoid dropping flow-controlled messages when log-fifo-size() is misconfigured. From now on, log-fifo-size() only affects messages that are not flow-controlled. (Flow-controlled log paths have the flags(flow-control) option set.) To enable the new behaviour, update the @version string in your configuration and consider lowering the value of log-fifo-size().;

postfix log after container start:

Dec 27 11:09:14 my-domain postfix/postfix-script[695]: starting the Postfix mail system
Dec 27 11:09:14 my-domain postfix/master[697]: daemon started -- version 3.5.16, configuration /etc/postfix
Dec 27 11:09:14 my-domain postfix/postfix-script[928]: stopping the Postfix mail system
Dec 27 11:09:14 my-domain postfix/master[996]: daemon started -- version 3.5.16, configuration /etc/postfix

postfix log when a mail is forwarded:

Dec 27 11:13:22 my-domain postfix/postfix-script[699]: starting the Postfix mail system
Dec 27 11:13:22 my-domain postfix/master[701]: daemon started -- version 3.5.16, configuration /etc/postfix
Dec 27 11:13:22 my-domain postfix/postfix-script[929]: stopping the Postfix mail system
Dec 27 11:13:22 my-domain postfix/master[997]: daemon started -- version 3.5.16, configuration /etc/postfix
Dec 27 11:14:27 my-domain postfix/smtpd[1001]: connect from mail-pg1-f174.google.com[209.85.215.174]
Dec 27 11:14:28 my-domain postfix/smtpd[1001]: 06BBA4013C3: client=mail-pg1-f174.google.com[209.85.215.174]
Dec 27 11:14:28 my-domain postfix/cleanup[1007]: 06BBA4013C3: message-id=<CAL-5vhE3SrWeiMHd4URtnsRDT9bO7kVR8uLF8mRAZeOOXV4Vkg@mail.gmail.com>
Dec 27 11:14:28 my-domain postfix/qmgr[999]: 06BBA4013C3: from=<my.mail@gmail.com>, size=2831, nrcpt=1 (queue active)
Dec 27 11:14:28 my-domain postfix/smtpd[1001]: disconnect from mail-pg1-f174.google.com[209.85.215.174] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=1 quit=1 commands=7
Dec 27 11:14:28 my-domain postfix/smtp[1009]: 06BBA4013C3: to=<my.mail@gmail.com>, orig_to=<test@my-domain.at>, relay=gmail-smtp-in.l.google.com[108.177.119.26]:25, delay=0.78, delays=0.1/0.02/0.3/0.37, dsn=2.0.0, status=sent (250 2.0.0 OK  1703675668 q17-20020a056402519100b00553fbff96a9si6011954edd.548 - gsmtp)
Dec 27 11:14:28 my-domain postfix/qmgr[999]: 06BBA4013C3: removed

Furthermore, after the container runs for a while I get the following warnings:

Dec 27 12:09:53 my-domain postfix/smtpd[299]: connect from localhost[127.0.0.1]
Dec 27 12:09:53 my-domain postfix/smtpd[299]: warning: connect to Milter service unix:/run/opendkim/opendkim.sock: No such file or directory
Dec 27 12:09:53 my-domain postfix/smtpd[299]: warning: SASL authentication failure: Password verification failed
Dec 27 12:09:53 my-domain postfix/smtpd[299]: warning: localhost[127.0.0.1]: SASL PLAIN authentication failed: authentication failure
Dec 27 12:09:53 my-domain postfix/smtpd[299]: lost connection after AUTH from localhost[127.0.0.1]
Dec 27 12:09:53 my-domain postfix/smtpd[299]: disconnect from localhost[127.0.0.1] auth=0/1 commands=0/1

On my domain I have an SPF, MX and DKIM Record setup as well as rDNS.

Any help would be greatly appreciated.

m-schoder commented 8 months ago

After almost abandoning SMF and looking for paid solutions I found out, that Google just silently drops your email if it is forwarded to the same address as it originiated. 😠 Thanks Google 🖕

Everything works now.

Here is my docker-compose file:

version: "3.5"

services:
  smf:
    container_name: mail-forwarder
    image: zixia/simple-mail-forwarder
    restart: always
    ports:
      - "25:25"
    environment:
      - SMF_DOMAIN=${SMF_DOMAIN}
      - SMF_CONFIG=${SMF_CONFIG}
      - SMF_SRS=true
      - SMF_POSTFIXMAIN_sender_canonical_maps=tcp:localhost:10001
      - SMF_POSTFIXMAIN_sender_canonical_classes=envelope_sender
      - SMF_POSTFIXMAIN_recipient_canonical_maps=tcp:localhost:10002
      - SMF_POSTFIXMAIN_recipient_canonical_classes=envelope_recipient,header_recipient
    volumes:
      - /opt/projects/logs/mail_forwarder:/var/log/postfix
      - ${MAIL_FORWARDER_DKIM}:/var/db/dkim

Here is the documentation I wrote for future me:

How To Setup SMF (for Google and Microsoft 365)

Docker Image Documentation

https://github.com/huan/docker-simple-mail-forwarder

Important

Domain Settings

Records

MX

Setup a MX Record like so:

Host Type MX Destination
@ MX 0 your-domain.at

Setup a SPF Record like so:

Host Type MX Destination
your-domain.at TXT - v=spf1 ip4:\ -all

Setup a DKIM Record like so:

Host Type MX Destination
default._domainkey TXT - v=DKIM1;t=s;p=\

You can generate a key pair at easydmarc.com.

Setup a DMARC Record like so:

Host Type MX Destination
_dmarc TXT - v=DMARC1; p=reject; rua=mailto:your.email@example.com

Docker compose

DKIM Volume

Store the generated DKIM key pair somewhere on the host machine and export the directory to the environment variable MAIL_FORWARDER_DKIM, so the mounted volume ${MAIL_FORWARDER_DKIM}:/var/db/dkim will be set up correctly. Make sure that your private key is named default.private.

rDNS

Setup rDNS to point to mail.your-domain.at (rDNS should be configured as 3rd level domain as per RFC) in your Server Settings. For servers hosted by netcup this can be done in the Server Control Panel. For servers hosted by lansol the Customer Support has to be contacted. So this is dependent on your server provider.

huan commented 8 months ago

Hi @m-schoder ,

Thank you very much for posting your solution with the detailed problem!

I'd love to invite you to add your documentation to the SMF README so that later users will quickly solve problems like yours and read your clear setup guide.

Please feel free to create a PR for it if you would like to.

Have a great new year ahead!