search

mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
8.92k stars 1.17k forks source link

update.sh script breaks docker-compose.override.yml #4566

Open fbermel opened 2 years ago

fbermel commented 2 years ago

Contribution guidelines

I've found a bug and checked that ...

Description

Hi,

this probably relates somewhat to #4212

In the current master the update.sh script does the following:

# Removing legacy container
sed -i '/ipv6nat-mailcow:$/,/^$/d' docker-compose.yml
if [ -s docker-compose.override.yml ]; then
    sed -i '/ipv6nat-mailcow:$/,/^$/d' docker-compose.override.yml
    if [[ "$(cat docker-compose.override.yml | sed '/^\s*$/d' | wc -l)" == "2" ]]; then
        mv docker-compose.override.yml docker-compose.override.yml_backup
    fi
fi

If it finds a docker-compose.override.yml it will attempt to save it as a backup and remove the ipv6nat-mailcow service from it. This leaves me with the following docker-compose.override.yml which is not valid anymore because I have a comment inside that file. So checking if it's 2 lines long doesn't work anymore and the file is now just empty.

root@mail /opt/mailcow-dockerized # cat docker-compose.override.yml 
# This file is managed by Ansible! Do not manually edit!

version: '2.1'
services:
    ipv6nat-mailcow:
      image: bash:latest
      restart: "no"
      entrypoint: ["echo", "ipv6nat disabled in compose.override.yml"]
root@mail /opt/mailcow-dockerized # docker-compose config -q
root@mail /opt/mailcow-dockerized # echo $?
0
root@mail /opt/mailcow-dockerized # ./update.sh 
Checking internet connection... OK
Checking for newer update script...
Updated 0 paths from a6befecb
Are you sure you want to update mailcow: dockerized? All containers will be stopped. [y/N] y
Great! Native IPv6 NAT is active.
Validating docker-compose stack configuration...
services must be a mapping

Oh no, something went wrong. Please check the error message above.
root@mail /opt/mailcow-dockerized # cat docker-compose.override.yml 
# This file is managed by Ansible! Do not manually edit!

version: '2.1'
services:
root@mail /opt/mailcow-dockerized # docker-compose config -q
services must be a mapping
root@mail /opt/mailcow-dockerized # echo $?
15

If the file is empty should probably be determined some other way.

Also the script should probably not only be checking for docker-compose.override.yml but also for docker-compose.override.yaml which is also a valid name for a compose file.

I think this is a bug in the update.shscript, unless I'm doing something wrong but I've followed the procedure in the docs so I'm not sure.

Best regards

Logs

This is a configuration/upgrade issue. I will provide logs if really needed but currently there is absolutely nothing useful in there.

Steps to reproduce

  1. Have working installation from master with docker-compose.override.yml present for disabling ipv6
  2. run update.sh
  3. Have a broken installation

System information

Question Answer
My operating system Ubuntu 20.04
Is Apparmor, SELinux or similar active? No
Virtualization technlogy (KVM, VMware, Xen, etc - LXC and OpenVZ are not supported KVM
Server/VM specifications (Memory, CPU Cores) 4GiB, 2 Cores
Docker Version (docker version) 20.10.14
Docker-Compose Version (docker-compose version) v2.4.1
Reverse proxy (custom solution) nginx with ssl termination on proxy

Output of git diff origin/master, any other changes to the code? If so, please post them:

root@mail /opt/mailcow-dockerized # git diff -w origin/master
diff --git a/data/conf/unbound/unbound.conf b/data/conf/unbound/unbound.conf
index 27110c04..2ad79fc2 100644
--- a/data/conf/unbound/unbound.conf
+++ b/data/conf/unbound/unbound.conf
@@ -1,10 +1,12 @@
+# This file is managed by Ansible!
+
 server:
   verbosity: 1
   interface: 0.0.0.0
   interface: ::0
   logfile: /dev/console
   do-ip4: yes
-  do-ip6: yes
+  do-ip6: no
   do-udp: yes
   do-tcp: yes
   do-daemonize: no
diff --git a/docker-compose.yml b/docker-compose.yml
index 5a32236a..ec68f533 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,3 +1,5 @@
+# This file is managed by Ansible! Do not manually edit!
+
 version: '2.1'
 services:

@@ -578,42 +580,12 @@ services:
           aliases:
             - ofelia

-    ipv6nat-mailcow:
-      depends_on:
-        - unbound-mailcow
-        - mysql-mailcow
-        - redis-mailcow
-        - clamd-mailcow
-        - rspamd-mailcow
-        - php-fpm-mailcow
-        - sogo-mailcow
-        - dovecot-mailcow
-        - postfix-mailcow
-        - memcached-mailcow
-        - nginx-mailcow
-        - acme-mailcow
-        - netfilter-mailcow
-        - watchdog-mailcow
-        - dockerapi-mailcow
-        - solr-mailcow
-      environment:
-        - TZ=${TZ}
-      image: robbertkl/ipv6nat
-      security_opt:
-        - label=disable
-      restart: always
-      privileged: true
-      network_mode: "host"
-      volumes:
-        - /var/run/docker.sock:/var/run/docker.sock:ro
-        - /lib/modules:/lib/modules:ro
-
 networks:
   mailcow-network:
     driver: bridge
     driver_opts:
       com.docker.network.bridge.name: br-mailcow
-    enable_ipv6: true
+    enable_ipv6: false
     ipam:
       driver: default
       config:

All third-party firewalls and custom iptables rules are unsupported. Please check the Docker docs about how to use Docker with your own ruleset. Nevertheless, iptabels output can help us to help you: iptables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 203K  251M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

ip6tables -L -vn:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-1  all      *      *       ::/0                 ::/0                
    0     0 ACCEPT     all      *      docker0  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all      *      docker0  ::/0                 ::/0                
    0     0 ACCEPT     all      docker0 !docker0  ::/0                 ::/0                
    0     0 ACCEPT     all      docker0 docker0  ::/0                 ::/0                
    0     0 DOCKER-USER  all      *      *       ::/0                 ::/0                
    0     0 DOCKER     all      *      br-mailcow  ::/0                 ::/0                
    0     0 ACCEPT     all      *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all      br-mailcow !br-mailcow  ::/0                 ::/0                
    0     0 ACCEPT     all      br-mailcow br-mailcow  ::/0                 ::/0                

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all      docker0 !docker0  ::/0                 ::/0                
    0     0 RETURN     all      *      *       ::/0                 ::/0                

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all      *      docker0  ::/0                 ::/0                
    0     0 RETURN     all      *      *       ::/0                 ::/0                

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all      *      *       ::/0                 ::/0

iptables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   21  1220 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0

ip6tables -L -vn -t nat:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all      *      *       ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all      *      *       ::/0                !::1                  ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all      *      !docker0  fd00:dead:beef:c0::/80  ::/0                

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all      docker0 *       ::/0                 ::/0

DNS problems? Please run docker exec -it $(docker ps -qf name=acme-mailcow) dig +short stackoverflow.com @172.22.1.254 (set the IP accordingly, if you changed the internal mailcow network) and post the output:

151.101.65.69
151.101.193.69
151.101.129.69
151.101.1.69
DerLinkman commented 2 years ago

Hey there,

the update.sh Script calls this mentioned action only if you´ve accepted the native Docker IPv6 NAT Option during the update.sh process.

So that is basically not an issue.

However: It might make sense to check if there is a docker-compose.override.yml or .yaml which have the disabled IPv6 things inside to skip the Docker IPv6 NAT Process in general (since this will appear every time you run the update.sh script and don´t accept the changes from that.

milkmaker commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.