Emails from certain senders are not being forwarded correctly

ririko5834 commented 4 months ago

Support guidelines

[X] I've read the support guidelines

I've found a bug and checked that ...

[X] ... the documentation does not mention anything about my problem
[X] ... there are no open or closed issues that are related to my problem

Description

I selfhost anonaddy using docker. But anonaddy seems to not forward some emails. For example, I don't get emails from the courier company that my package will be delivered the next day. Other emails I usually normally have. Can you check it?

I don't receive emails from serv@intime.sk (they are not forwarded on anonaddy). I receive emails from that mail address normally on my gmail.

After running postqueue -p, I can see there the undelivered emails in queue

Later, there are more emails:

None of them are listed in failed delivery tab, why?

Logs:

Why was it deferred? There are many of those logs

{"log":"\n","stream":"stdout","time":"2023-07-18T14:40:01.420379427Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: connect from unknown[172.25.0.1]\n","stream":"stdout","time":"2023-07-18T14:40:01.841606347Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: D266910A021B: client=unknown[172.25.0.1]\n","stream":"stdout","time":"2023-07-18T14:40:01.86416435Z"}
{"log":"Jul 18 14:40:01 mail postfix/cleanup[365841]: D266910A021B: message-id=\[u003c202307181440.36IEe1tK1112957@node1.myotherdomain.com](mailto:u003c202307181440.36IEe1tK1112957@node1.myotherdomain.com)\u003e\n","stream":"stdout","time":"2023-07-18T14:40:01.865085534Z"}
{"log":"Jul 18 14:40:01 mail postfix/qmgr[1290]: D266910A021B: from=\[u003csmmsp@node1.myotherdomain.com](mailto:u003csmmsp@node1.myotherdomain.com)\u003e, size=1062, nrcpt=1 (queue active)\n","stream":"stdout","time":"2023-07-18T14:40:01.867180311Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: disconnect from unknown[172.25.0.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5\n","stream":"stdout","time":"2023-07-18T14:40:01.874239627Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: connect from unknown[172.25.0.1]\n","stream":"stdout","time":"2023-07-18T14:40:01.926272342Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtp[365842]: warning: host [node1.myotherdomain.com](http://node1.myotherdomain.com/)[my_vps_ip_address]:25 greeted me with my own hostname [mail.my_anonaddy_domain.com](http://mail.my_anonaddy_domain.com/)\n","stream":"stdout","time":"2023-07-18T14:40:01.926299102Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtp[365842]: warning: host [node1.myotherdomain.com](http://node1.myotherdomain.com/)[my_vps_ip_address]:25 replied to HELO/EHLO with my own hostname [mail.my_anonaddy_domain.com](http://mail.my_anonaddy_domain.com/)\n","stream":"stdout","time":"2023-07-18T14:40:01.926530312Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtp[365842]: D266910A021B: to=\[u003croot@node1.myotherdomain.com](mailto:u003croot@node1.myotherdomain.com)\u003e, relay=[node1.myotherdomain.com](http://node1.myotherdomain.com/)[my_vps_ip_address]:25, delay=0.07, delays=0.01/0.01/0.05/0, dsn=5.4.6, status=bounced (mail for [node1.myotherdomain.com](http://node1.myotherdomain.com/) loops back to myself)\n","stream":"stdout","time":"2023-07-18T14:40:01.933908931Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: disconnect from unknown[172.25.0.1] ehlo=1 quit=1 commands=2\n","stream":"stdout","time":"2023-07-18T14:40:01.933931834Z"}
{"log":"Jul 18 14:40:01 mail postfix/cleanup[365841]: E459E10A02EC: message-id=\[u003c20230718144001.E459E10A02EC@mail.my_anonaddy_domain.com](mailto:u003c20230718144001.E459E10A02EC@mail.my_anonaddy_domain.com)\u003e\n","stream":"stdout","time":"2023-07-18T14:40:01.9364827Z"}
{"log":"Jul 18 14:40:01 mail postfix/qmgr[1290]: E459E10A02EC: from=\u003c\u003e, size=3091, nrcpt=1 (queue active)\n","stream":"stdout","time":"2023-07-18T14:40:01.939295714Z"}
{"log":"Jul 18 14:40:01 mail postfix/bounce[365843]: D266910A021B: sender non-delivery notification: E459E10A02EC\n","stream":"stdout","time":"2023-07-18T14:40:01.939310912Z"}
{"log":"Jul 18 14:40:01 mail postfix/qmgr[1290]: D266910A021B: removed\n","stream":"stdout","time":"2023-07-18T14:40:01.939314078Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: connect from unknown[172.25.0.1]\n","stream":"stdout","time":"2023-07-18T14:40:01.979248302Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtp[365842]: warning: host [node1.myotherdomain.com](http://node1.myotherdomain.com/)[my_vps_ip_address]:25 greeted me with my own hostname [mail.my_anonaddy_domain.com](http://mail.my_anonaddy_domain.com/)\n","stream":"stdout","time":"2023-07-18T14:40:01.979319675Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtp[365842]: warning: host [node1.myotherdomain.com](http://node1.myotherdomain.com/)[my_vps_ip_address]:25 replied to HELO/EHLO with my own hostname [mail.my_anonaddy_domain.com](http://mail.my_anonaddy_domain.com/)\n","stream":"stdout","time":"2023-07-18T14:40:01.979577174Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtp[365842]: E459E10A02EC: to=\[u003csmmsp@node1.myotherdomain.com](mailto:u003csmmsp@node1.myotherdomain.com)\u003e, relay=[node1.myotherdomain.com](http://node1.myotherdomain.com/)[my_vps_ip_address]:25, delay=0.05, delays=0/0/0.04/0, dsn=5.4.6, status=bounced (mail for [node1.myotherdomain.com](http://node1.myotherdomain.com/) loops back to myself)\n","stream":"stdout","time":"2023-07-18T14:40:01.98116576Z"}
{"log":"Jul 18 14:40:01 mail postfix/smtpd[365838]: disconnect from unknown[172.25.0.1] ehlo=1 quit=1 commands=2\n","stream":"stdout","time":"2023-07-18T14:40:01.981494181Z"}
{"log":"Jul 18 14:40:01 mail postfix/qmgr[1290]: E459E10A02EC: removed\n","stream":"stdout","time":"2023-07-18T14:40:01.981856666Z"}
{"log":"Jul 18 14:40:24 mail postfix/qmgr[1290]: 7F5EB10A036B: from=\[u003cserv@intime.sk](mailto:u003cserv@intime.sk)\u003e, size=6904, nrcpt=1 (queue active)\n","stream":"stdout","time":"2023-07-18T14:40:24.99979179Z"}
{"log":"Jul 18 14:40:25 mail postfix/qmgr[1290]: CBCC110A0369: from=\[u003cserv@intime.sk](mailto:u003cserv@intime.sk)\u003e, size=6867, nrcpt=1 (queue active)\n","stream":"stdout","time":"2023-07-18T14:40:25.005908912Z"}
{"log":"Jul 18 14:40:25 mail postfix/pipe[365844]: 7F5EB10A036B: to=\[u003cnappy.sage401@wolfoone.my_anonaddy_domain.com](mailto:u003cnappy.sage401@wolfoone.my_anonaddy_domain.com)\u003e, relay=anonaddy, delay=429304, delays=429304/0.01/0/0.56, dsn=4.3.0, status=deferred (An error has occurred, please try again later. )\n","stream":"stdout","time":"2023-07-18T14:40:25.573641348Z"}
{"log":"Jul 18 14:40:25 mail postfix/pipe[365845]: CBCC110A0369: to=\[u003cmanned.overlord368@anopost.net](mailto:u003cmanned.overlord368@anopost.net)\u003e, relay=anonaddy, delay=430643, delays=430642/0.01/0/0.56, dsn=4.3.0, status=deferred (An error has occurred, please try again later. )\n","stream":"stdout","time":"2023-07-18T14:40:25.574978576Z"}
{"log":"crond: USER anonaddy pid 365854 cmd php /var/www/anonaddy/artisan schedule:run --no-ansi --no-interaction\n","stream":"stdout","time":"2023-07-18T14:41:00.968479041Z"}
{"log":"\n","stream":"stdout","time":"2023-07-18T14:41:01.321617025Z"}

None of those emails were saved in failed deliveries, how does failed delivery work, it saves there only if the alias recipient email address refused the forwarded email.

I would appreciate any help. Do you think that it's caused because the sender serv@intime.sk did not pass all anonaddy anti-spam checks? If yes, is there a way to whitelist that domain so it will skip all the checks and forward it straight?

Expected behaviour

Forward them normally

Actual behaviour

Emails from serv@intime.sk get stuck in the queue

Steps to reproduce

.

Docker info

Client: Docker Engine - Community
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 12
  Running: 11
  Paused: 0
  Stopped: 1
 Images: 73
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc version: v1.1.9-0-gccaecfc
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.4.0-165-generic
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.759GiB
 Name: myhostname
 ID: M6DG:A64P:F2XO:Z7MR:T5KM:4KVS:SDVX:XFTF:QGCR:VGXB:WMC6:MFHU
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Docker Compose config

No response

Logs

11.7 - https://pastebin.com/raw/4EgthhbX
12.7 - https://pastebin.com/raw/mhpZNSZb
13.7 - https://pastebin.com/raw/SwwzhEnF

Additional info

No response

ririko5834 commented 4 months ago

I already made issue on anonaddy repo last year https://github.com/anonaddy/anonaddy/issues/485 But even after a year and rebrand to addy.io I still cant receive emails from serv@intime.sk which sucks because idk when courier will deliver me a package.

I host anonaddy using this container on portainer panel. willbrowningme can't reproduce the issue, but he tested only locally without using docker, I hope you can help me to fix it

buxm commented 4 months ago

I'm not sure I can help... for me postfix is akin to black magic. However in the past I realised I misconfigured stuff that prevented delivery by logging in to the rspamd web interface and going to the History tab. If you go there, do you see any errors?

ririko5834 commented 4 months ago

those emails from serv@intime.sk passed

buxm commented 4 months ago

I can see there are quite a few "cannot connect upstream" errors down below in the Errors list. When those errors repeatedly happened to me in the past, they were due to the fact that rspamd could not communicate with the redis server. And that led to undelivered emails, despite the fact rspamd "passed" the emails. Now, I'm not sure whether your issue is the same as mine. For me rspamd is even more esoteric than postfix, and the errors logged don't help. But maybe you can try to figure out why the redis server might not be reachable (wrong container name? wrong port? redis is requiring a password that wasn't set in REDIS_PASSWORD?...) and see whether it helps fixing things.

ririko5834 commented 4 months ago

It doesnt show any logs, I host both anonaddy, mysql and redis using docker compose https://github.com/anonaddy/docker/blob/master/examples/rspamd/compose.yml I didn't set any password for it

ririko5834 commented 4 months ago

redis logs: redis env:

anonaddy env:

trying to ping redis db from anonaddy container:

It seems to work, so redis should be reachable

buxm commented 4 months ago

From what I can see, the redis setup should be fine.

However ping redis:6379 only pings the redis host (container), not its actual port 6379. If you want a more thorough test, run this command from the shell in the anonaddy container:

db42ab424180:/var/www/anonaddy# nc redis 6379

then you should get a prompt on an empty line. Type:

PING

you should get some kind of response back. I get:

-NOAUTH Authentication required.

because I set up redis with a password.

I expect it works, because you didn't set up a custom redis port. In which case, unfortunately I don't know where to look further to investigate your issue.

ririko5834 commented 4 months ago

buxm commented 4 months ago

yes, that works. so the issue clearly is somewhere else I have no clue of... :-(

ririko5834 commented 4 months ago

@willbrowningme can you take a look at it and investigate it running it using docker

ririko5834 commented 4 months ago

What other steps to investigate it?

ririko5834 commented 4 months ago

Cant anyone help me to write some script which will run regularly and will forward/send all emails from serv@intime.sk that got stuck in postqueue to different email address or to discord webhoook? This would work as a tempfix. Its really annoying that I cant receive delivery notifications for my parcels from courier company

ririko5834 commented 4 months ago

?

ririko5834 commented 2 months ago

@buxm

buxm commented 2 months ago

@ririko5834 unfortunately I can't help with your script: as said, I'm totally unfamiliar with postfix.

anonaddy / docker