Mailu / Mailu

Insular email distribution - mail server as Docker images
https://mailu.io
Other
5.94k stars 838 forks source link

front stays on started, admin on unhealthy #2347

Closed j2l closed 2 years ago

j2l commented 2 years ago

Before you open your issue

Environment & Versions

Ubuntu 20.04 VPS with a reverse proxy for other containers, using 80 and 443, so I changed mailu to other ports in docker-compose.yml I've chosen sqlite rainloop, and all the options (clamav, webdav, fetchmail)

Happens when runing for the first time docker-compose -p mailu exec admin flask mailu admin admin ...

Environment

Versions

1.9 $> docker ps -a | grep mailu

f535414c13c5   mailu/admin:1.9                   "/bin/sh -c /start.py"   4 minutes ago       Up 4 minutes (unhealthy)    80/tcp                                                                   mailu_admin_1
85d14e40046e   redis:alpine                      "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes                6379/tcp                                                                 mailu_redis_1
09ffe46bebce   mailu/fetchmail:1.9               "/fetchmail.py"          4 minutes ago       Up 4 minutes                                                                                         mailu_fetchmail_1
15f7bd9f590c   mailu/radicale:1.9                "/bin/sh -c 'radical…"   4 minutes ago       Up 4 minutes (healthy)      5232/tcp                                                                 mailu_webdav_1
98f629e3811f   mailu/nginx:1.9                   "/bin/sh -c /start.py"   4 minutes ago       Created                                                                                              mailu_front_1
e66cf3fcf76b   mailu/clamav:1.9                  "/bin/sh -c /start.py"   4 minutes ago       Up 4 minutes (healthy)      3310/tcp                                                                 mailu_antivirus_1
43bee6dcc89a   mailu/unbound:1.9                 "/bin/sh -c /start.py"   4 minutes ago       Up 4 minutes (healthy)      53/tcp, 53/udp                                                           mailu_resolver_1

$> grep MAILU_VERSION docker-compose.yml mailu.env

docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}nginx:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}unbound:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}admin:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}dovecot:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}postfix:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}rspamd:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}clamav:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}radicale:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}fetchmail:${MAILU_VERSION:-1.9}
docker-compose.yml:    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}rainloop:${MAILU_VERSION:-1.9}

Replication Steps

install mailu using https://setup.mailu.io/1.9/setup

Expected behaviour

healthy containers

Logs

docker-compose -f /mailu/docker-compose.yml logs --tail 1000 admin

f535414c13c5_mailu_admin_1 | Traceback (most recent call last):
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 407, in __call__
f535414c13c5_mailu_admin_1 |     result = fn(*args, **kwargs)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/socrate/system.py", line 12, in resolve_hostname
f535414c13c5_mailu_admin_1 |     return socket.gethostbyname(hostname)
f535414c13c5_mailu_admin_1 | socket.gaierror: [Errno -2] Name does not resolve
f535414c13c5_mailu_admin_1 | 
f535414c13c5_mailu_admin_1 | The above exception was the direct cause of the following exception:
f535414c13c5_mailu_admin_1 | 
f535414c13c5_mailu_admin_1 | Traceback (most recent call last):
f535414c13c5_mailu_admin_1 |   File "/usr/bin/flask", line 8, in <module>
f535414c13c5_mailu_admin_1 |     sys.exit(main())
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 994, in main
f535414c13c5_mailu_admin_1 |     cli.main(args=sys.argv[1:])
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 600, in main
f535414c13c5_mailu_admin_1 |     return super().main(*args, **kwargs)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/click/core.py", line 1053, in main
f535414c13c5_mailu_admin_1 |     rv = self.invoke(ctx)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/click/core.py", line 1653, in invoke
f535414c13c5_mailu_admin_1 |     cmd_name, cmd, args = self.resolve_command(ctx, args)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/click/core.py", line 1700, in resolve_command
f535414c13c5_mailu_admin_1 |     cmd = self.get_command(ctx, cmd_name)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 556, in get_command
f535414c13c5_mailu_admin_1 |     return info.load_app().cli.get_command(ctx, name)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 406, in load_app
f535414c13c5_mailu_admin_1 |     app = locate_app(self, import_name, name)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 276, in locate_app
f535414c13c5_mailu_admin_1 |     return find_best_app(script_info, module)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 68, in find_best_app
f535414c13c5_mailu_admin_1 |     app = call_factory(script_info, app_factory)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/flask/cli.py", line 123, in call_factory
f535414c13c5_mailu_admin_1 |     return app_factory(*args, **kwargs)
f535414c13c5_mailu_admin_1 |   File "/app/mailu/__init__.py", line 81, in create_app
f535414c13c5_mailu_admin_1 |     return create_app_from_config(config)
f535414c13c5_mailu_admin_1 |   File "/app/mailu/__init__.py", line 21, in create_app_from_config
f535414c13c5_mailu_admin_1 |     config.init_app(app)
f535414c13c5_mailu_admin_1 |   File "/app/mailu/configuration.py", line 144, in init_app
f535414c13c5_mailu_admin_1 |     self.resolve_hosts()
f535414c13c5_mailu_admin_1 |   File "/app/mailu/configuration.py", line 116, in resolve_hosts
f535414c13c5_mailu_admin_1 |     self.config[f'{key}_ADDRESS'] = self.get_host_address(key)
f535414c13c5_mailu_admin_1 |   File "/app/mailu/configuration.py", line 112, in get_host_address
f535414c13c5_mailu_admin_1 |     return system.resolve_address(self.config[f'HOST_{name}'])
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/socrate/system.py", line 20, in resolve_address
f535414c13c5_mailu_admin_1 |     ip_address = resolve_hostname(hostname)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 324, in wrapped_f
f535414c13c5_mailu_admin_1 |     return self(f, *args, **kw)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 404, in __call__
f535414c13c5_mailu_admin_1 |     do = self.iter(retry_state=retry_state)
f535414c13c5_mailu_admin_1 |   File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 361, in iter
f535414c13c5_mailu_admin_1 |     raise retry_exc from fut.exception()
f535414c13c5_mailu_admin_1 | tenacity.RetryError: RetryError[<Future at 0x7f60356703a0 state=finished raised gaierror>]
nextgens commented 2 years ago

Check that the value set for DOMAIN and HOSTNAMES in your mailu.env are valid (have DNS entries)

j2l commented 2 years ago

Thanks @nextgens , they do. Domain is my TLD and hostname is mail.TLD DNS is setup to reach this machine (CNAME->A) then a reverse proxy to reach the containers on whatever port I want.

nextgens commented 2 years ago

Please share your docker-compose.yml and mailu.env

j2l commented 2 years ago

Sure, thanks for the help, merci pour l'aide Florent :+1:

# This file is auto-generated by the Mailu configuration wizard.
# Please read the documentation before attempting any change.
# Generated for compose flavor

version: '2.2'

services:

  # External dependencies
  redis:
    image: redis:alpine
    restart: always
    volumes:
      - "/docker/mailu/redis:/data"
    depends_on:
      - resolver
    dns:
      - 192.168.203.254

  # Core services
  front:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}nginx:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    logging:
      driver: json-file
    ports:
      - "myboxexternalip:8081:80"
      - "myboxexternalip:4443:443"
      - "myboxexternalip:25:25"
      - "myboxexternalip:465:465"
      - "myboxexternalip:587:587"
      - "myboxexternalip:110:110"
      - "myboxexternalip:995:995"
      - "myboxexternalip:143:143"
      - "myboxexternalip:993:993"
    volumes:
      - "/docker/mailu/certs:/certs"
      - "/docker/mailu/overrides/nginx:/overrides:ro"
    depends_on:
      - resolver
    dns:
      - 192.168.203.254

  resolver:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}unbound:${MAILU_VERSION:-1.9}
    env_file: mailu.env
    restart: always
    networks:
      default:
        ipv4_address: 192.168.203.254

  admin:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}admin:${MAILU_VERSION:-1.9}
    ports:
      - "8082:80"
      - "5443:443"
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/data:/data"
      - "/docker/mailu/dkim:/dkim"
    depends_on:
      - redis
      - resolver
    dns:
      - 192.168.203.254

  imap:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}dovecot:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/mail:/mail"
      - "/docker/mailu/overrides/dovecot:/overrides:ro"
    depends_on:
      - front
      - resolver
    dns:
      - 192.168.203.254

  smtp:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}postfix:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/mailqueue:/queue"
      - "/docker/mailu/overrides/postfix:/overrides:ro"
    depends_on:
      - front
      - resolver
    dns:
      - 192.168.203.254

  antispam:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}rspamd:${MAILU_VERSION:-1.9}
    hostname: antispam
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/filter:/var/lib/rspamd"
      - "/docker/mailu/overrides/rspamd:/etc/rspamd/override.d:ro"
    depends_on:
      - front
      - resolver
    dns:
      - 192.168.203.254

  # Optional services
  antivirus:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}clamav:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/filter:/data"
    depends_on:
      - resolver
    dns:
      - 192.168.203.254

  webdav:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}radicale:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/dav:/data"
    depends_on:
      - resolver
    dns:
      - 192.168.203.254

  fetchmail:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}fetchmail:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/data/fetchmail:/data"
    depends_on:
      - resolver
    dns:
      - 192.168.203.254

  # Webmail
  webmail:
    image: ${DOCKER_ORG:-mailu}/${DOCKER_PREFIX:-}rainloop:${MAILU_VERSION:-1.9}
    restart: always
    env_file: mailu.env
    volumes:
      - "/docker/mailu/webmail:/data"
      - "/docker/mailu/overrides/rainloop:/overrides:ro"
    depends_on:
      - imap
      - resolver
    dns:
      - 192.168.203.254

networks:
  default:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 192.168.203.0/24

.

# Mailu main configuration file
#
# This file is autogenerated by the configuration management wizard for compose flavor.
# For a detailed list of configuration variables, see the documentation at
# https://mailu.io

###################################
# Common configuration variables
###################################

# Set to a randomly generated 16 bytes string
SECRET_KEY=NONONO

# Subnet of the docker network. This should not conflict with any networks to which your system is connected. (Internal and external!)
SUBNET=192.168.203.0/24

# Main mail domain
DOMAIN=TLD.com

# Hostnames for this server, separated with comas
HOSTNAMES=mail.TLD.com

# Postmaster local part (will append the main mail domain)
POSTMASTER=admin

# Choose how secure connections will behave (value: letsencrypt, cert, notls, mail, mail-letsencrypt)
TLS_FLAVOR=notls

# Authentication rate limit per IP (per /24 on ipv4 and /56 on ipv6)
AUTH_RATELIMIT_IP=60/hour

# Authentication rate limit per user (regardless of the source-IP)
AUTH_RATELIMIT_USER=100/day

# Opt-out of statistics, replace with "True" to opt out
DISABLE_STATISTICS=True

###################################
# Optional features
###################################

# Expose the admin interface (value: true, false)
ADMIN=true

# Choose which webmail to run if any (values: roundcube, rainloop, none)
WEBMAIL=rainloop

# Dav server implementation (value: radicale, none)
WEBDAV=radicale

# Antivirus solution (value: clamav, none)
ANTIVIRUS=clamav

###################################
# Mail settings
###################################

# Message size limit in bytes
# Default: accept messages up to 50MB
# Max attachment size will be 33% smaller
MESSAGE_SIZE_LIMIT=50000000

# Message rate limit (per user)
MESSAGE_RATELIMIT=200/day

# Networks granted relay permissions
# Use this with care, all hosts in this networks will be able to send mail without authentication!
RELAYNETS=

# Will relay all outgoing mails if configured
RELAYHOST=

# Fetchmail delay
FETCHMAIL_DELAY=600

# Recipient delimiter, character used to delimiter localpart from custom address part
RECIPIENT_DELIMITER=+

# DMARC rua and ruf email
DMARC_RUA=admin
DMARC_RUF=admin

# Welcome email, enable and set a topic and body if you wish to send welcome
# emails to all users.
WELCOME=false
WELCOME_SUBJECT=Welcome to your new email account
WELCOME_BODY=Welcome to your new email account, if you can read this, then it is configured properly!

# Maildir Compression
# choose compression-method, default: none (value: gz, bz2, lz4, zstd)
COMPRESSION=
# change compression-level, default: 6 (value: 1-9)
COMPRESSION_LEVEL=

# IMAP full-text search is enabled by default. Set the following variable to off in order to disable the feature.
# FULL_TEXT_SEARCH=off

###################################
# Web settings
###################################

# Path to redirect / to
WEBROOT_REDIRECT=/webmail

# Path to the admin interface if enabled
WEB_ADMIN=/adminette

# Path to the webmail if enabled
WEB_WEBMAIL=/webmail

# Website name
SITENAME=TLD

# Linked Website URL
WEBSITE=https://TLD.com

###################################
# Advanced settings
###################################

# Log driver for front service. Possible values:
# json-file (default)
# journald (On systemd platforms, useful for Fail2Ban integration)
# syslog (Non systemd platforms, Fail2Ban integration. Disables `docker-compose log` for front!)
# LOG_DRIVER=json-file

# Docker-compose project name, this will prepended to containers names.
COMPOSE_PROJECT_NAME=mailu

# Number of rounds used by the password hashing scheme
CREDENTIAL_ROUNDS=12

# Header to take the real ip from
REAL_IP_HEADER=

# IPs for nginx set_real_ip_from (CIDR list separated by commas)
REAL_IP_FROM=

# choose wether mailu bounces (no) or rejects (yes) mail when recipient is unknown (value: yes, no)
REJECT_UNLISTED_RECIPIENT=

# Log level threshold in start.py (value: CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET)
LOG_LEVEL=WARNING

# Timezone for the Mailu containers. See this link for all possible values https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
TZ=Etc/UTC

###################################
# Database settings
###################################
DB_FLAVOR=sqlite

the already used ports are changed because the reverse proxy takes care of the routing with all the containers. and the admin path to prevent bots to reach something and try brute force.

nextgens commented 2 years ago

In your docker-compose.yml you shouldn't override the ports for "admin"; doing it in "front" is enough (keep in mind that front is the only exposed container)

nextgens commented 2 years ago

You also need to configure REAL_IP_FROM and REAL_IP_HEADER if you use a reverse-proxy, see https://mailu.io/1.9/configuration.html#reverse-proxy-headers

j2l commented 2 years ago

In your docker-compose.yml you shouldn't override the ports for "admin"; doing it in "front" is enough (keep in mind that front is the only exposed container)

You're right, I added this after writing this ticket.

The section about the real_ip is not clear enough for me:

The REAL_IP_HEADER (default: unset) and REAL_IP_FROM (default: unset) settings controls whether HTTP headers such as X-Forwarded-For or X-Real-IP should be trusted. The former should be the name of the HTTP header to extract the client IP address from and the later a comma separated list of IP addresses designating which proxies to trust. If you are using Mailu behind a reverse proxy, you should set both. Setting the former without the later introduces a security vulnerability allowing a potential attacker to spoof his source address.

Should I set it to my public ip?

nextgens commented 2 years ago

It depends on how your reverse proxy is configured; It should be obvious from the logs what IP address is seen by Mailu/front

j2l commented 2 years ago

Ok, retrying with the public IP: Before docker-compose -p mailu exec admin flask mailu admin admin ..., there are no logs on admin and front. After docker-compose -p mailu exec admin flask mailu admin admin ... same result for admin (see logs above), and for front: Attaching to mailu_front_1 ... then it exists because it's in created status.

j2l commented 2 years ago

127.0.0.1 made it!

ADmin works from TLD, not from IP. Well, it works