search

mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
8.99k stars 1.18k forks source link

Watchdog notifications: Webhook to Pushover #5760

Closed mrclschstr closed 6 months ago

mrclschstr commented 9 months ago

Contribution guidelines

I've found a bug and checked that ...

Description

I have seen that you can now also use webhooks for the watchdog notifications:

I have therefore made the following settings in mailcow.conf (inspired by: https://community.mailcow.email/d/2234-watchdog-notification-how-to-enable-pushover-notification/7):

USE_WATCHDOG=y
WATCHDOG_NOTIFY_BAN=y
WATCHDOG_NOTIFY_START=y
WATCHDOG_NOTIFY_WEBHOOK=https://api.pushover.net/1/messages.json
WATCHDOG_NOTIFY_WEBHOOK_BODY='{"token":"XXX", "user":"XXX", "message":"[${SUBJECT}] ${BODY}"}'

With the setting WATCHDOG_NOTIFY_BAN=y I wanted to get an overview of current IP bans. However, I noticed in the Docker logs that an error is generated when a ban notification is sent (see logs below).

I am sure that the token is not the problem as the notification about the watchdog start (WATCHDOG_NOTIFY_START=y) is delivered normally. I have already contacted Pushover support to get details about the request. The short answer was simply:

I don't show any parameters sent with that request, so they are probably not encoded properly.

I therefore suspect that the body of the webhook is causing a syntax error in the event of an IP ban. Do a few special characters need to be escaped here?

Logs:

mailcowdockerized-watchdog-mailcow-1  | {"token":"invalid","errors":["application token must be supplied, see https://pushover.net/api"],"status":0,"request":"abd9b9d4-040d-4621-afd5-xxxxxxxxxxxx"}Sun Feb 18 22:23:35 CET 2024 Sent notification using webhook

Steps to reproduce:

  1. Setup webhook to Pushover as described above
  2. Wait until an IP is banned (or you provoke it)
  3. Inspect Docker logs docker compose logs watchdog-mailcow | grep -i "webhook"

Which branch are you using?

master

Which architecture are you using?

x86

Operating System:

Ubuntu 22.04.3 LTS

Server/VM specifications:

4 vCPU, 8 GB RAM

Is Apparmor, SELinux or similar active?

No

Virtualization technology:

KVM

Docker version:

24.0.7

docker-compose version or docker compose version:

v2.21.0

mailcow version:

2024-01e

Reverse proxy:

No

Logs of git diff:

[Omitted cert.pem and key.pem]

diff --git a/data/conf/postfix/main.cf b/data/conf/postfix/main.cf
index 572300db..2158fecc 100644
--- a/data/conf/postfix/main.cf
+++ b/data/conf/postfix/main.cf
@@ -173,3 +173,36 @@ parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks

 # DO NOT EDIT ANYTHING BELOW #
 # Overrides #
+
+postscreen_dnsbl_sites = wl.mailspike.net=127.0.0.[18;19;20]*-2
+  hostkarma.junkemailfilter.com=127.0.0.1*-2
+  list.dnswl.org=127.0.[0..255].0*-2
+  list.dnswl.org=127.0.[0..255].1*-4
+  list.dnswl.org=127.0.[0..255].2*-6
+  list.dnswl.org=127.0.[0..255].3*-8
+  ix.dnsbl.manitu.net*2
+  bl.spamcop.net*2
+  bl.suomispam.net*2
+  hostkarma.junkemailfilter.com=127.0.0.2*3
+  hostkarma.junkemailfilter.com=127.0.0.4*2
+  hostkarma.junkemailfilter.com=127.0.1.2*1
+  backscatter.spameatingmonkey.net*2
+  bl.ipv6.spameatingmonkey.net*2
+  bl.spameatingmonkey.net*2
+  b.barracudacentral.org=127.0.0.2*7
+  bl.mailspike.net=127.0.0.2*5
+  bl.mailspike.net=127.0.0.[10;11;12]*4
+  dnsbl.sorbs.net=127.0.0.10*8
+  dnsbl.sorbs.net=127.0.0.5*6
+  dnsbl.sorbs.net=127.0.0.7*3
+  dnsbl.sorbs.net=127.0.0.8*2
+  dnsbl.sorbs.net=127.0.0.6*2
+  dnsbl.sorbs.net=127.0.0.9*2
+  zen.spamhaus.org=127.0.0.[10;11]*8
+  zen.spamhaus.org=127.0.0.[4..7]*6
+  zen.spamhaus.org=127.0.0.3*4
+  zen.spamhaus.org=127.0.0.2*3
+
+# User Overrides
+myhostname = mail.[XXX].com

Logs of iptables -L -vn:

Relevant? I can provide more information if it is important.

Logs of ip6tables -L -vn:

Relevant? I can provide more information if it is important.

Logs of iptables -L -vn -t nat:

Relevant? I can provide more information if it is important.

Logs of ip6tables -L -vn -t nat:

Relevant? I can provide more information if it is important.

DNS check:

104.18.32.7
172.64.155.249
mrclschstr commented 9 months ago

Maybe jq could be used to escape the ${BODY} variable for webhooks? Example:

mrclschstr commented 8 months ago

Issue is still present in version 2024-02.

mrclschstr commented 8 months ago

I have rummaged through the watchdog.sh script and found the error. It is in this line:

https://github.com/mailcow/mailcow-dockerized/blob/8d4ef147d2d7d8eda9575c0c3069fc6944fb6bd0/data/Dockerfiles/watchdog/watchdog.sh#L173

The problem is the message subject during an IP ban that looks like IP ban: fe80::7645:6de2:ff:1/128 or IP ban: 127.0.0.1/24, for example. Specifically, the problem lies in the CIDR notation of the IP address with the slash / contained in it, which is also used as a separator in sed. The following error message is displayed in bash:

sed: -e expression #1, char 105: unknown option to `s'

A simple fix could be to replace the separator to # for the subject, such as (inspired by https://unix.stackexchange.com/a/378991):

WEBHOOK_BODY=$(echo ${WATCHDOG_NOTIFY_WEBHOOK_BODY} | sed "s#\$SUBJECT\|\${SUBJECT}#$SUBJECT#g" | sed "s/\$BODY\|\${BODY}/$BODY/g")

Unfortunately, I cannot judge whether changing the separator provokes another error.

FreddleSpl0it commented 8 months ago

Already did it. Originally i used | as a separator, but it was reverted with this PR as it didn't fully work. https://github.com/mailcow/mailcow-dockerized/pull/5647

mrclschstr commented 8 months ago

The only other possibility I can think of off the top of my head would be sanitizing the replacement string.

You could use bash variable expansion, for example (inspired by https://stackoverflow.com/a/13210909):

sed "s/\$SUBJECT\|\${SUBJECT}/${SUBJECT//\//\\/}/g"

Caution: Not fully tested!

[EDIT] See also:

milkmaker commented 6 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.