search

mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
8.32k stars 1.13k forks source link

Sieve scripts stops working on dovecot start, but works when dovecot is already started #4770

Open pztrn opened 1 year ago

pztrn commented 1 year ago

Contribution guidelines

I've found a bug and checked that ...

Description

Sieve scripts with personal includes stopped working on dovecot container restart, but works fine after it successfully started. I've made this script to fix them "automagically":

#!/bin/bash

cd /var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/domain.tld/user/sieve
mv default.sieve ..
cd /opt/mailcow-dockerized
docker compose restart dovecot-mailcow
sleep 10
cd /var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data/domain.tld/user/sieve
mv ../default.sieve .

Logs

dovecot-mailcow_1  | default: line 6: error: include: personal location for included script `spam' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 8: error: include: personal location for included script `archlinux' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 9: error: include: personal location for included script `autotrash' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 10: error: include: personal location for included script `chats_and_socials' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 11: error: include: personal location for included script `communications' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 12: error: include: personal location for included script `cron' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 13: error: include: personal location for included script `development' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 14: error: include: personal location for included script `finances' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 15: error: include: personal location for included script `food' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 16: error: include: personal location for included script `forums' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 17: error: include: personal location for included script `games' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 18: error: include: personal location for included script `hostings' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 19: error: include: personal location for included script `hr' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 20: error: include: personal location for included script `important' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 21: error: include: personal location for included script `learnings' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 22: error: include: personal location for included script `lostfilm' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 23: error: include: personal location for included script `mailing_lists' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 24: error: include: personal location for included script `shops' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 25: error: include: personal location for included script `soft' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: line 26: error: include: personal location for included script `subscriptions' is unavailable (contact system administrator for more information).
dovecot-mailcow_1  | default: error: validation failed.
dovecot-mailcow_1  | sievec: Fatal: failed to compile sieve script '/var/vmail/domain.tld/username/sieve/default.sieve'
dovecot-mailcow_1  | Uptime: 179  Threads: 9  Questions: 468  Slow queries: 0  Opens: 73  Open tables: 64  Queries per second avg: 2.614

Steps to reproduce

  1. Install mailcow.
  2. Add some scripts and include them with :personal in active script.
  3. Restart dovecot.

System information

Question Answer
My operating system Debian 11
Is Apparmor, SELinux or similar active? AppArmor, default settings
Virtualization technology (KVM, VMware, Xen, etc - LXC and OpenVZ are not supported KVM
Server/VM specifications (Memory, CPU Cores) 4 cores, 6 GB RAM, 60 GB HDD (45 GB free, 99% inodes free)
Docker version (docker version) 20.10.17
docker-compose version (docker-compose version) Docker Compose version v2.6.0
mailcow version (git describe --tags `git rev-list --tags --max-count=1`) 2022-09
Reverse proxy (custom solution) SMTP, IMAP, Sieve - directly to VM, web proxied via Caddy on another VM

Output of git diff origin/master, any other changes to the code? If so, please post them:

diff --git a/data/conf/dovecot/dovecot.conf b/data/conf/dovecot/dovecot.conf
index b7aca757..ac122c05 100644
--- a/data/conf/dovecot/dovecot.conf
+++ b/data/conf/dovecot/dovecot.conf
@@ -194,7 +194,7 @@ plugin {
   fts_solr = url=http://solr:8983/solr/dovecot-fts/
   quota = dict:Userquota::proxy::sqlquota
   quota_rule2 = Trash:storage=+100%%
-  sieve = /var/vmail/sieve/%u.sieve
+  sieve = file:%h/sieve;active=/var/vmail/sieve/%u.sieve
   sieve_plugins = sieve_imapsieve sieve_extprograms
   sieve_vacation_send_from_recipient = yes
   sieve_redirect_envelope_from = recipient
diff --git a/data/conf/postfix/main.cf b/data/conf/postfix/main.cf
index a445b60c..61b1c0ce 100644
--- a/data/conf/postfix/main.cf
+++ b/data/conf/postfix/main.cf
@@ -198,3 +198,31 @@ parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks

 # DO NOT EDIT ANYTHING BELOW #
 # User overrides #
+
+myhostname = aurora.pztrn.name
+
+#mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 [fe80::]/10 172.16.0.0/12 192.168.0.0/16 [fc00::]/7
+
+recipient_delimiter = +
+unknown_local_recipient_reject_code = 550
+owner_request_special = no
+
+local_recipient_maps =
+  regexp:/opt/mailman/core/var/data/postfix_lmtp,
+  proxy:unix:passwd.byname,
+  $alias_maps
+virtual_mailbox_maps =
+  proxy:mysql:/opt/postfix/conf/sql/mysql_virtual_mailbox_maps.cf,
+  regexp:/opt/mailman/core/var/data/postfix_lmtp
+transport_maps =
+  pcre:/opt/postfix/conf/custom_transport.pcre,
+  pcre:/opt/postfix/conf/local_transport,
+  proxy:mysql:/opt/postfix/conf/sql/mysql_relay_ne.cf,
+  proxy:mysql:/opt/postfix/conf/sql/mysql_transport_maps.cf,
+  regexp:/opt/mailman/core/var/data/postfix_lmtp
+relay_domains =
+  proxy:mysql:/opt/postfix/conf/sql/mysql_virtual_relay_domain_maps.cf,
+  regexp:/opt/mailman/core/var/data/postfix_domains
+relay_recipient_maps =
+  proxy:mysql:/opt/postfix/conf/sql/mysql_relay_recipient_maps.cf,
+  regexp:/opt/mailman/core/var/data/postfix_lmtp
diff --git a/data/conf/sogo/sogo.conf b/data/conf/sogo/sogo.conf
index 97a34e9e..e0809474 100644
--- a/data/conf/sogo/sogo.conf
+++ b/data/conf/sogo/sogo.conf
@@ -5,7 +5,7 @@
         PrivateDAndTViewer
     );

-    WOWorkersCount = "20";
+    WOWorkersCount = "3";
     SOGoACLsSendEMailNotifications = YES;
     SOGoAppointmentSendEMailNotifications = YES;
     SOGoDraftsFolderName = "Drafts";

(note that I've tried to configure sieve directive according to configuration just to show that this isn't working also. If I rollback this change it doesn't work either.)

All third-party firewalls and custom iptables rules are unsupported. Please check the Docker docs about how to use Docker with your own ruleset. Nevertheless, iptabels output can help us to help you: iptables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 195K   64M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
 195K   64M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
59528   37M ACCEPT     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
 5051  320K DOCKER     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
18813 7258K ACCEPT     all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
 4790  304K ACCEPT     all  --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
  34M 6124M ACCEPT     all  --  *      br-b2d828911f8b  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
18246 1095K DOCKER     all  --  *      br-b2d828911f8b  0.0.0.0/0            0.0.0.0/0
 118K 9912K ACCEPT     all  --  br-b2d828911f8b !br-b2d828911f8b  0.0.0.0/0            0.0.0.0/0
18117 1087K ACCEPT     all  --  br-b2d828911f8b br-b2d828911f8b  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (3 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !br-b2d828911f8b br-b2d828911f8b  0.0.0.0/0            172.19.199.2         tcp dpt:8024
    0     0 ACCEPT     tcp  --  !br-b2d828911f8b br-b2d828911f8b  0.0.0.0/0            172.19.199.3         tcp dpt:8080
    0     0 ACCEPT     tcp  --  !br-b2d828911f8b br-b2d828911f8b  0.0.0.0/0            172.19.199.2         tcp dpt:8001
  128  7680 ACCEPT     tcp  --  !br-b2d828911f8b br-b2d828911f8b  0.0.0.0/0            172.19.199.3         tcp dpt:8000
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.4           tcp dpt:8983
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.249         tcp dpt:6379
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.9           tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.12          tcp dpt:443
   22  1320 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.12          tcp dpt:80
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:587
   60  3600 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:465
    7   412 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:25
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:12345
    1    64 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:4190
  158  9480 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:995
    8   508 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:993
    4   252 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:143
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:110

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
18813 7258K DOCKER-ISOLATION-STAGE-2  all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
 118K 9912K DOCKER-ISOLATION-STAGE-2  all  --  br-b2d828911f8b !br-b2d828911f8b  0.0.0.0/0            0.0.0.0/0
  61M   19G RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (3 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      br-b2d828911f8b  0.0.0.0/0            0.0.0.0/0
5622K 1168M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  61M   19G RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

ip6tables -L -vn:

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
7836K   11G RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:587
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:25
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::f  tcp dpt:465
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::11  tcp dpt:443
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::11  tcp dpt:80
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::e  tcp dpt:143
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::e  tcp dpt:4190
  899 99455 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::e  tcp dpt:993
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::e  tcp dpt:995
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::e  tcp dpt:110

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
 1933  390K DOCKER-ISOLATION-STAGE-2  all      br-mailcow !br-mailcow  ::/0                 ::/0
31059   50M RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all      *      br-mailcow  ::/0                 ::/0
 1933  390K RETURN     all      *      *       ::/0                 ::/0

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
31059   50M DOCKER-USER  all      *      *       ::/0                 ::/0
7836K   11G DOCKER-ISOLATION-STAGE-1  all      *      *       ::/0                 ::/0
7302K   11G DOCKER     all      *      br-mailcow  ::/0                 ::/0
5785K   11G ACCEPT     all      *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
 533K  103M ACCEPT     all      br-mailcow !br-mailcow  ::/0                 ::/0
1381K  100M ACCEPT     all      br-mailcow br-mailcow  ::/0                 ::/0

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
# Warning: ip6tables-legacy tables present, use ip6tables-legacy to see them

iptables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 135K 7992K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 3998  301K MASQUERADE  all  --  *      !br-mailcow  172.22.1.0/24        0.0.0.0/0
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
 7394  444K MASQUERADE  all  --  *      !br-b2d828911f8b  172.19.199.0/24      0.0.0.0/0
    0     0 MASQUERADE  tcp  --  *      *       172.19.199.2         172.19.199.2         tcp dpt:8024
    0     0 MASQUERADE  tcp  --  *      *       172.19.199.3         172.19.199.3         tcp dpt:8080
    0     0 MASQUERADE  tcp  --  *      *       172.19.199.2         172.19.199.2         tcp dpt:8001
    0     0 MASQUERADE  tcp  --  *      *       172.19.199.3         172.19.199.3         tcp dpt:8000
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.4           172.22.1.4           tcp dpt:8983
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.249         172.22.1.249         tcp dpt:6379
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.9           172.22.1.9           tcp dpt:3306
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.12          172.22.1.12          tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.12          172.22.1.12          tcp dpt:80
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:587
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:465
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.253         172.22.1.253         tcp dpt:25
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:12345
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:4190
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:995
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:993
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:143
    0     0 MASQUERADE  tcp  --  *      *       172.22.1.250         172.22.1.250         tcp dpt:110

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  br-mailcow *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
  108  6480 RETURN     all  --  br-b2d828911f8b *       0.0.0.0/0            0.0.0.0/0
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            192.168.8.3          tcp dpt:8024 to:172.19.199.2:8024
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            127.0.0.1            tcp dpt:8024 to:172.19.199.2:8024
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            192.168.8.3          tcp dpt:8080 to:172.19.199.3:8080
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            192.168.8.3          tcp dpt:8001 to:172.19.199.2:8001
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            127.0.0.1            tcp dpt:8080 to:172.19.199.3:8080
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            127.0.0.1            tcp dpt:8001 to:172.19.199.2:8001
  128  7680 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            192.168.8.3          tcp dpt:8000 to:172.19.199.3:8000
    0     0 DNAT       tcp  --  !br-b2d828911f8b *       0.0.0.0/0            127.0.0.1            tcp dpt:8000 to:172.19.199.3:8000
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:18983 to:172.22.1.4:8983
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:7654 to:172.22.1.249:6379
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            192.168.8.3          tcp dpt:13306 to:172.22.1.9:3306
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 to:172.22.1.12:443
   22  1320 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:172.22.1.12:80
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:587 to:172.22.1.253:587
   60  3600 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:465 to:172.22.1.253:465
    7   412 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25 to:172.22.1.253:25
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:19991 to:172.22.1.250:12345
    1    64 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4190 to:172.22.1.250:4190
  159  9540 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:995 to:172.22.1.250:995
    8   508 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993 to:172.22.1.250:993
    4   252 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:143 to:172.22.1.250:143
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:110 to:172.22.1.250:110

ip6tables -L -vn -t nat:

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all      br-mailcow *       ::/0                 ::/0
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:587 to:[fd4d:6169:6c63:6f77::f]:587
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:25 to:[fd4d:6169:6c63:6f77::f]:25
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:465 to:[fd4d:6169:6c63:6f77::f]:465
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:443 to:[fd4d:6169:6c63:6f77::11]:443
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:80 to:[fd4d:6169:6c63:6f77::11]:80
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:143 to:[fd4d:6169:6c63:6f77::e]:143
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:4190 to:[fd4d:6169:6c63:6f77::e]:4190
   35  2940 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:993 to:[fd4d:6169:6c63:6f77::e]:993
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:995 to:[fd4d:6169:6c63:6f77::e]:995
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:110 to:[fd4d:6169:6c63:6f77::e]:110

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 6133  514K DOCKER     all      *      *       ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all      *      *       ::/0                !::1                  ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  all      *      br-mailcow  ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL
   36  3126 MASQUERADE  all      *      !br-mailcow  fd4d:6169:6c63:6f77::  ::/0
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::10  fd4d:6169:6c63:6f77::10  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::10  fd4d:6169:6c63:6f77::10  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::11  fd4d:6169:6c63:6f77::11  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::11  fd4d:6169:6c63:6f77::11  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:143
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:4190
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:993
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:995
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:110
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:25
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:993
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:995
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:110
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:143
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:4190
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::e  fd4d:6169:6c63:6f77::e  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::12  fd4d:6169:6c63:6f77::12  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::12  fd4d:6169:6c63:6f77::12  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:25
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::f  fd4d:6169:6c63:6f77::f  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:25
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::d  fd4d:6169:6c63:6f77::d  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::5  fd4d:6169:6c63:6f77::5  tcp dpt:80
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::5  fd4d:6169:6c63:6f77::5  tcp dpt:443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::12  fd4d:6169:6c63:6f77::12  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::12  fd4d:6169:6c63:6f77::12  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::12  fd4d:6169:6c63:6f77::12  tcp dpt:25

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
# Warning: ip6tables-legacy tables present, use ip6tables-legacy to see them

DNS problems? Please run docker exec -it $(docker ps -qf name=acme-mailcow) dig +short stackoverflow.com @172.22.1.254 (set the IP accordingly, if you changed the internal mailcow network) and post the output:

151.101.129.69
151.101.193.69
151.101.65.69
151.101.1.69
waliwdenko commented 1 year ago

i have the same problem, this error appears, because during startup docker-entrypoint.sh tries compile sieve files. sievec cannot locate files for include instructions. For temporary solution i commeted out in files data/Dockerfiles/dovecot/docker-entrypoint.sh lines 353-359:

#for file in /var/vmail/*/*/sieve/*.sieve ; do
#  if [[ "$file" == "/var/vmail/*/*/sieve/*.sieve" ]]; then
#    continue
#  fi
#  sievec "$file" "$(dirname "$file")/../.dovecot.svbin"
#  chown vmail:vmail "$(dirname "$file")/../.dovecot.svbin"
#done

and put in docker-compose.yml in volumes for dovecot-mailcow:

        - ./data/Dockerfiles/dovecot/docker-entrypoint.sh:/docker-entrypoint.sh

but now dovecot using text file every time when filtering e-mail, so it can be slower.

Anybody knows better soluton?

guiand888 commented 1 year ago

I also have this issue when upgrading and had to stay one version behind because that meant disabling sieve scripts otherwise. I didn't see any mention in the telegram group either. It's really curious that there aren't more people affected by this. I didn't see any mention in the telegram group either.

pztrn commented 1 year ago

Workaround for me: everything works okay when using only one sieve script (e.g. managing filters via SOGo).

waliwdenko commented 1 year ago

Workaround for me: everything works okay when using only one sieve script (e.g. managing filters via SOGo).

it isnt solutions, it works only because in these scripts you dont have "include" instruction

guiand888 commented 1 year ago

Workaround for me: everything works okay when using only one sieve script (e.g. managing filters via SOGo).

Sorry, I don't get your point. In my case, it is precisely the sieve scripts managed by SOGo that get passed through the include instruction and cause the bug.

pztrn commented 1 year ago

It isn't a solution, yes, but workaround to get sieve working. As well as one from @waliwdenko. I decided to go this way and not tamper with entrypoint script to avoid possible problems when updating mailcow.

Also, my SOGo installation doesn't allow to select sieve scripts to activate (like standalone sieve app which I was using previously).

waja commented 1 year ago

I'm affected as well. I'd like to use include :personal to add different roles when I'm on "vacation" for example, but this seems to be not possible anymore since some time. The command has no effect (but is failing to start the container, when used).

milkmaker commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

waja commented 1 year ago

This issue still exists:

mailcowdockerized-dovecot-mailcow-1    | default: line 886: error: include: personal location for included script `vacation' is unavailable (contact system administrator for more information).
mailcowdockerized-dovecot-mailcow-1    | default: error: validation failed.
waja commented 1 year ago

@milkmaker reopen

th-joerger commented 1 year ago 
03.04.2023, 12:44:01 | info | A95A623A48:  to=<thor@joerger.dev>, relay=dovecot[fd4d:6168:6c63:6f77::e]:24,  delay=27434, delays=27434/0.04/0.03/0.02, dsn=4.2.0, status=deferred  (host dovecot[fd4d:6168:6c63:6f77::e] said: 451 4.2.0  <thor@joerger.dev> Temporarily unable to access necessary Sieve  scripts (in reply to end of DATA command))

03.04.2023, 12:44:01 | info | 6B94A239D7:  host dovecot[172.16.1.250] said: 451 4.2.0 <thor@joerger.dev>  Temporarily unable to access necessary Sieve scripts (in reply to end of  DATA command)

03.04.2023, 12:44:01 | info | 992D32394A:  host dovecot[fd4d:6168:6c63:6f77::e] said: 451 4.2.0  <thor@joerger.dev> Temporarily

Issue still persists. Restarting the containers fixes it. Please reopen @DerLinkman @FreddleSpl0it

FreddleSpl0it commented 1 year ago

@th-joerger could you provide me a detailed steps to reproduce? i think the problem was introduced here https://github.com/mailcow/mailcow-dockerized/commit/ad8b7f08944c05854d8d6bf87ae13cd7ecd92f5a also see https://github.com/mailcow/mailcow-dockerized/issues/4770#issuecomment-1312745953

th-joerger commented 1 year ago

Hey @FreddleSpl0it I don't know how to reproduce the problem, sorry. It is no longer occuring on my production deployment since update to 2023-04a though.

waja commented 1 year ago

Still with latest version:

default: line 886: error: include: personal location for included script `vacation' is unavailable (contact system administrator for more information).                                                                                            
default: error: validation failed.                                                                                         sievec: Fatal: failed to compile sieve script '/var/vmail/<domain>/<user>/sieve/default.sieve'

Just create (for example by managedsieve) a "vacation" script and "default" (I think the 'include' require should fit, but keeped my require statement, just to be sure) script with:

require ["fileinto", "reject", "vacation", "regex", "relational", "comparator-i;ascii-numeric", "envelope", "include", "mailbox"];

include :personal "vacation";

Now restart the stack (maybe restarting the dovecot container might also trigger it) and you should face the issue.

milkmaker commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

waja commented 1 year ago

This is still an issue

waja commented 11 months ago

I can report: still with 2023-07 present

999eagle commented 10 months ago

I'm also affected by this issue. After investigating for a bit, it seems like sievec currently doesn't support include :personal at all as it doesn't have an option to set the current user and thus can't set the personal storage location. libsieve then tries to resolve this location with NULL as the user (see https://github.com/dovecot/pigeonhole/blob/11ada0e23c7b9f6cd4af17d3f9b07b26f51452af/src/lib-sieve/plugins/include/ext-include-common.c#L147 and https://github.com/dovecot/pigeonhole/blob/11ada0e23c7b9f6cd4af17d3f9b07b26f51452af/src/lib-sieve/sieve-storage.c#L475).

milkmaker commented 8 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

waja commented 8 months ago

And still ... an issue

DerLinkman commented 8 months ago

Could you guys please send us a example sieve script we can test with? I haven't encountered this issue at all. It would be pretty helpful for us.

999eagle commented 8 months ago

@DerLinkman any script using include :personal "script"; will trigger this. For example looking at the files in my /var/vmail/$domain/$localpart/sieve directory: USER.sieve just contains require ["include"]; include :personal "sort"; (and some comment lines and more similar includes) and sort.sieve contains the actual script. dovecot fails to start because the command sievec "/var/vmail/$domain/$localpart/sieve/USER.sieve" "/var/vmail/$domain/$localpart/sieve/../.dovecot.svbin" (see the entrypoint.sh) fails with this output:

USER: line 12: error: include: personal location for included script `sort' in unavailable (contact system administrator for more information).
USER: error: validation failed.
sievec: Fatal: failed to compile sieve script 'USER.sieve'
judemille commented 7 months ago

I have this exact same problem. Are there any workarounds I can use so that I can conditionally avoid use of things like imapsieve except when in the correct conditions?

waja commented 7 months ago

Could you guys please send us a example sieve script we can test with? I haven't encountered this issue at all. It would be pretty helpful for us.

@DerLinkman do you need more examples or is the one fitting to reproduce the issue?

999eagle commented 7 months ago

@judemille The workaround has been mentioned in this issue already, see here: https://github.com/mailcow/mailcow-dockerized/issues/4770#issuecomment-1312745953

guiand888 commented 7 months ago

Could you guys please send us a example sieve script we can test with? I haven't encountered this issue at all. It would be pretty helpful for us.

Sorry, a bit late to the party but here's an example of a config that will trigger this issue.

Sieve config:

$ user @ server: /m/m/d/v/m/_/g/m/sieve# cat USER.sieve

# USER Management Script
#
# This script includes the various active sieve scripts
# it is AUTOMATICALLY GENERATED. DO NOT EDIT MANUALLY!
# 
# For more information, see http://wiki.kolab.org/KEP:14#USER
#

require ["include"];

include :personal "sogo";

Example of a personal script that breaks things up - in this case, it is the default sieve script that was generated by SOGo when I add rule from the webui:

$ user @ server: m/m/d/v/m/_/g/m/sieve# cat sogo.sieve 

require ["fileinto"];
if anyof (header :contains "subject" "mainbox", header :contains "subject" "cron FAILURE") {
    fileinto "INBOX";
    stop;
}

In the past, I've used KMail to edit Sieve rules in sogo.sieve - without issues back then - and I wonder if this might have to do with the config breaking down the line? For the time being, the workaround I've found is it too comment our include :personal "sogo"; when restarting mailcow or doing updates.

aukfood commented 5 months ago

Hello, I have this same workout after latest update

 Error: sieve: msgid=<MC.TRS.791949486.1885011705913262160@******>: redirect action: failed to redirect message to <*****@******>: smtp(postfix:588): RCPT TO failed: Failed to connect to remote server (temporary failure)
waja commented 1 month ago

Hi there,

any news on this issue?

DerLinkman commented 1 week ago

Hi,

yes actually!

I've looked at the code and found out what is causing this. it is the way the sieve compiler is compiling scripts right now...

We need to rewrite and rethink the sieve script compiling from https://github.com/mailcow/mailcow-dockerized/blob/70126e1f0cb0f78d99446453521b8d665fe07db9/data/Dockerfiles/dovecot/docker-entrypoint.sh#L410C1-L416C5 to make this working.

We're on it but no ETA here.