mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
8.94k stars 1.17k forks source link

DNS container starts as 1.17.1 instead of 1.18 in 2023-10a #5476

Closed vigorio closed 1 year ago

vigorio commented 1 year ago

Contribution guidelines

I've found a bug and checked that ...

Description

DNS container starts as 1.17.1 instead of 1.18 in 2023-10a. My update fails due to unbound container being unhealthy. Also confirmed on a clean install.

Logs:

mailcowdockerized-unbound-mailcow-1    | Setting console permissions...
mailcowdockerized-unbound-mailcow-1    | Receiving anchor key...
mailcowdockerized-unbound-mailcow-1    | Receiving root hints...
######################################################################## 100.0%                                        
mailcowdockerized-unbound-mailcow-1    | setup in directory /etc/unbound
mailcowdockerized-unbound-mailcow-1    | Certificate request self-signature ok
mailcowdockerized-unbound-mailcow-1    | subject=CN = unbound-control
mailcowdockerized-unbound-mailcow-1    | removing artifacts
mailcowdockerized-unbound-mailcow-1    | Setup success. Certificates created. Enable in unbound.conf file to use
mailcowdockerized-unbound-mailcow-1    | [1697470871] unbound[1:0] notice: init module 0: validator
mailcowdockerized-unbound-mailcow-1    | [1697470871] unbound[1:0] notice: init module 1: iterator
mailcowdockerized-unbound-mailcow-1    | [1697470871] unbound[1:0] info: start of service (unbound 1.17.1).

Steps to reproduce:

1. Update using ./update.sh or do a clean install.
2. Get "Container mailcowdockerized-unbound-mailcow-1          Error"
3. See the bug

Which branch are you using?

master

Operating System:

Ubuntu 22.04 LTS

Server/VM specifications:

20GB RAM, 4 cores xeon 2699v4

Is Apparmor, SELinux or similar active?

no

Virtualization technology:

esxi

Docker version:

24.0.6

docker-compose version or docker compose version:

v2.21.0

mailcow version:

2023-10a

Reverse proxy:

HAproxy

Logs of git diff:

-

Logs of iptables -L -vn:

# Warning: iptables-legacy tables present, use iptables-legacy to see them
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 346K   68M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 346K   68M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
 310K   63M ACCEPT     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
 9625  599K DOCKER     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0           
26650 3897K ACCEPT     all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0           
 9388  584K ACCEPT     all  --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.2           tcp dpt:8983
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:12345
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:4190
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:995
  165 10556 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:993
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.6           tcp dpt:8443
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.6           tcp dpt:8080
   45  2880 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:143
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.250         tcp dpt:110
    9   540 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:587
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.249         tcp dpt:6379
    7   396 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:465
   11   660 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.253         tcp dpt:25
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.22.1.13          tcp dpt:3306

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
26650 3897K DOCKER-ISOLATION-STAGE-2  all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0           
 346K   68M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0           
26650 3897K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 346K   68M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Logs of ip6tables -L -vn:

-

Logs of iptables -L -vn -t nat:

-

Logs of ip6tables -L -vn -t nat:

-

DNS check:

-
DerLinkman commented 1 year ago

Cannot reproduce that...

Update process works as intended

vigorio commented 1 year ago

Dear DerLinkman, thank you for your answer. The update works but the installation doesn't work resulting in container being unhealthy. There have been no changes since April 2023 to the configs on my side, docker-compose.override.yml etc. I must admit that I use pfsense as resolver and I put the DNS values into the docker-compose.override.yml.

After struggling with my version and new version I found out that the container is unhealthy due to: "DNS resolution is not working correctly...\Maybe check your outbound firewall, as it needs to resolve DNS over TCP AND UDP!"

I don't have any firewall in the mailcow VM.

I tried to change the version of unbound to 1.18 in docker-compose.yml of 2023-09, it pulled 1.18 and started without any issues. I assume that somehow 2023-10a disrespects the DNS settings in docker-compose.override.yml and this is why the container becomes unhealthy.

DerLinkman commented 1 year ago

Could you tell me what you added to the docker compose override dns wise?

You use pfsense (externally I think) as dns resolver?

It needs to be added in the unbound config as well.

Maybe (but I can't say that without seeing what you've added) it is a faulty setup...

vigorio commented 1 year ago

version: '2.1' services:

clamd-mailcow:
  dns:
    - 192.168.3.1

rspamd-mailcow:
  dns:
    - 192.168.3.1

php-fpm-mailcow:
  dns:
    - 192.168.3.1

sogo-mailcow:
  dns:
    - 192.168.3.1
  volumes:
    - ./data/conf/sogo/custom-theme.css:/usr/lib/GNUstep/SOGo/WebServerResources/css/theme-default.css:z

dovecot-mailcow:
  dns:
    - 192.168.3.1

postfix-mailcow:
  dns:
    - 192.168.3.1

nginx-mailcow:
  dns:
    - 192.168.3.1

acme-mailcow:
  dns:
    - 192.168.3.1

watchdog-mailcow:
  environment:
    - CHECK_UNBOUND=0
  dns:
    - 192.168.3.1

dockerapi-mailcow:
  dns:
    - 192.168.3.1
vigorio commented 1 year ago

Tested. Yes, I needed to add following to unbound.conf

forward-zone: name: "." forward-addr: 192.168.3.1

The official documentation is inconsistent, since it mentions to use either Method A or B and not to do both! Please review and edit it. Thank you for your help!

https://docs.mailcow.email/manual-guides/Unbound/u_e-unbound-fwd/

vigorio commented 1 year ago

Closed

DerLinkman commented 1 year ago

Hm. i cannot confirm either that you need todo have both. It worked on my setup doing only once.

vigorio commented 1 year ago

Dear DerLinkman, I have tested it on a fresh VM Ubuntu 22.04 LTS with a fresh setup of mailcow. As soon as I add the unbound.conf section everything works as expected. BTW it works als with unbound.conf only. I assume that the override.yml method/file is being partially "ignored" (in terms of DNS) by the installation.

masterjanic commented 10 months ago

I also encountered this exact issue when restarting my instance of mailcow after updating to the latest version. Same error, using version 1.17.1 instead of 1.18. I also pulled the latest image of unbound and it did not work even after re-installing mailcow itself.

After some search I found out that this issue was caused by the network settings of my VM. I had disabled IPv6 in /etc/sysctl.conf which somehow caused unbound to stop working.

I don't know the exact cause of this error, but I also had IPv6 disabled in docker-compose.yml, which was working fine before...