search

maxking / docker-mailman

Dockerfiles for the Mailman suite.
https://asynchronous.in/docker-mailman/
MIT License
235 stars 104 forks source link
containers django docker exim mailman mailman-suite mta postfix

readme

permalink: /

GNU Mailman 3 Deployment with Docker

CircleCI

This repository hosts code for two docker images maxking/mailman-core and maxking/mailman-web both of which are meant to deploy GNU Mailman 3 in a production environment.

Docker is a container ecosystem which can run containers on several platforms. It consists of a tool called docker-compose which can be used to run multi-container applications. This repository consists of a docker-compose.yaml file which is a set of configurations that can be used to deploy the Mailman 3 Suite.

Please see release page for the releases and change log.

Release

The tags for the images are assumed to be release versions for images. This is going to be a somewhat common philosophy of distributing Container images where the images with same tags are usually updated with the new functionality.

Releases will follow the following rules:

Container Registries

The container images are available from multiple container registries. Do specify an explicit version tag (e.g. 0.4.5 , MAJOR.MINOR like 0.4 also works as floating tag pointing to latest patch version) as tag latest is not updated anymore.

Mailman Core

Mailman Web

Postorius

Rolling Releases

Rolling releases are made up of Mailman Components installed from git source. Note that these releases are made up of un-released software and should be assumed to be beta quality.

Every commit is tested with Mailman's CI infrastructure and is included in rolling releases only if they have passed the complete test suite.

$ docker pull docker.io/maxking/mailman-web:rolling
$ docker pull docker.io/maxking/mailman-core:rolling

Rolling releases are built with every commit and also re-generated nightly. You can inspect the images to get which commit it was built using:

$ docker inspect --format '{{json .Config.Labels }}' mailman-core | python -m json.tool
{
    "version.git_commit": "45a4d7805b2b3d0e7c51679f59682d64ba02f05f",
}

$ docker inspect --format '{{json .Config.Labels }}' mailman-web | python -m json.tool
{
    "version.git_commit": "45a4d7805b2b3d0e7c51679f59682d64ba02f05f",
}

Dependencies

To install these on Ubuntu/Debian:

$ sudo apt install docker.io docker-compose-plugin

For other systems, you can read the official Docker documentation to install Docker from here and docker compose from here.

Configuration

Most of the common configuration is handled through environment variables in the docker-compose.yaml. However, there is need for some extra configuration that interacts directly with the application. There are two configuration files on the host that interact directly with Mailman's settings. These files exist on the host running the containers and are imported at runtime in the containers.

Also, note that if you need any other files to be accessible from the host to inside the container, you can place them at certain directories which are mounted inside the containers.

Mailman-web

These are the settings that you MUST change in your docker-compose.yaml before deploying:

Please note here that if you choose to create the admin user using the environment variables mentioned above (MAILMAN_ADMIN_USER & MAILMAN_ADMIN_EMAIL), no password is set for your admin account. To set a password, plese follow the "Forgot Password" link on the "Sign In" page.

Mailman web is already configured to send emails through $SMTP_HOST as the MTA's address. If you want to modify it, you can set the value in under docker-compose.yaml for mailman-web container. By default, SMTP_HOST points to the gateway of the web container, which is the host itself.

You can also use the environment variables SMTP_HOST (defaults to the container's gateway), SMTP_PORT (defaults to 25), SMTP_HOST_USER (defaults to an empty string), SMTP_HOST_PASSWORD (defaults to an empty string), SMTP_USE_TLS (defaults to False) and SMTP_USE_SSL (defaults to False).

This is required in addition to the Setup your MTA section below, which covers email setup for Mailman Core.

For more details on how to configure this image, please look at Mailman-web's Readme

Mailman-Core

These are the variables that you MUST change in your docker-compose.yaml before deploying:

For more details on how to configure this image, please look Mailman-core's Readme

While the above configuration will allow you to run the images and possibly view the Web Frontend, it won't be functional until it is fully configured to to send emails.

To configure the mailman-core container to send emails, see the Setting your MTA section below.

Running

To run the containers, simply run:

$ mkdir -p /opt/mailman/core
$ mkdir -p /opt/mailman/web
$ git clone https://github.com/maxking/docker-mailman
$ cd docker-mailman
# Change some configuration variables as mentioned above.
$ docker compose up -d

Note that the web frontend in the mailman-web container is, by default, only configured to serve dynamic content. Anything static like stylesheets, etc., is expected to be served directly by the web server. The static content exists at /opt/mailman/web/static and should be aliased to /static/ in the web server configuration.

See the nginx configuration as an example.

This command will do several things, most importantly:

Some more details about what the above system achieves is mentioned below. If you are only going to deploy a simple configuration, you don't need to read this. However, these are very easy to understand if you know how docker works.

Setting up your MTA

The provided docker containers do not have an MTA in-built. You can either run your own MTA inside a container and have them relay emails to the mailman-core container or just install an MTA on the host and have them relay emails.

Exim4

To use Exim4, it should be setup to relay emails from mailman-core and mailman-web. The mailman specific configuration is provided in the repository at core/assets/exim. There are three files

Also, the default configuration inside the mailman-core image has the MTA set to Exim, but just for reference, it looks like this:

# mailman.cfg
[mta]
incoming: mailman.mta.exim4.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: $MM_HOSTNAME
lmtp_port: 8024
smtp_host: $SMTP_HOST
smtp_port: $SMTP_PORT
configuration: python:mailman.config.exim4

Postfix

To use Postfix, edit the main.cf configuration file, which is typically at /etc/postfix/main.cf on Debian-based operating systems. Add mailman-core and mailman-web to mynetworks so it will relay emails from the containers and add the following configuration lines:

# main.cf

# Support the default VERP delimiter.
recipient_delimiter = +
unknown_local_recipient_reject_code = 550
owner_request_special = no

transport_maps =
    regexp:/opt/mailman/core/var/data/postfix_lmtp
local_recipient_maps =
    regexp:/opt/mailman/core/var/data/postfix_lmtp
relay_domains =
    regexp:/opt/mailman/core/var/data/postfix_domains

To configure Mailman to use Postfix, add MTA=postfix under mailman-core's environment section in the docker-compose.yaml:

  mailman-core:
    <snip>
    environment:
    - MTA=postfix

This will auto-generate the configuration to talk to Postfix assuming that Postfix is available at the gateway address for the container's bridge network at port 25. The final configuration can be found by executing:

$ docker exec mailman-core cat /etc/mailman.cfg

The postfix configuration that is generated looks like this:

[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: $MM_HOSTNAME
lmtp_port: 8024
smtp_host: $SMTP_HOST
smtp_port: $SMTP_PORT
configuration: /etc/postfix-mailman.cfg

So, if you need to update the values, you can set SMTP_HOST, SMTP_PORT, MM_HOSTNAME environment variables in mailman-core container.

Please verify the output for [mta] section to ensure that it points to the right smtp_host (address to reach postfix from mailman-core container) and lmtp_host (address to reach mailman-core container from postfix).

The configuration file /etc/postfix-mailman.cfg is also generated automatically inside the mailman-core container and contains the configuration specific for Postfix.

Site Owner

Setup site owner address. By default, mailman is setup with the site_owner set to 'changeme@example.com'. This should be pointing to a valid mailbox. Add the following to the '/opt/mailman/core/mailman-extra.cfg'.

[mailman]
# This address is the "site owner" address.  Certain messages which must be
# delivered to a human, but which can't be delivered to a list owner (e.g. a
# bounce from a list owner), will be sent to this address.  It should point to
# a human.
site_owner: changeme@example.com

Setting up search indexing

Hyperkitty in mailman-web image support full-text indexing. The current default indexing engine is Whoosh for historical reasons. It is highly recommended that you instead use Xapian for production use cases. The default will change when the next major version bump happens.

To configure your Mailman-web container to use Xapian, add the following to your settings_local.py:

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'xapian_backend.XapianEngine',
        'PATH': "/opt/mailman-web-data/fulltext_index",
    },
}

If you have been using the default search indexing engine, you might have to re-index emails using the following command:

$ docker compose exec mailman-web ./manage.py rebuild_index

This command can take some time if you a lot of emails, so please be patient!

Setting up your web server

It is advisable to run your Django (interfaced through WSGI server) through an actual webserver in production for better performance.

If you are using v0.1.0, the uwsgi server is configured to listen to requests at port 8000 using the HTTP protocol. Make sure that you preserve the HOST header when you proxy the requests from your Web Server. In Nginx you can do that by adding the following to your configuration:

    # Nginx configuration.
    location /static {
        alias /opt/mailman/web/static;
        autoindex off;
    }

    location / {
          proxy_pass http://127.0.0.1:8000;
          include uwsgi_params;
          uwsgi_read_timeout 300;
          proxy_set_header Host $host;
          proxy_set_header X-Forwarded-For $remote_addr;
    }

Make sure you are using proxy_pass for the HTTP protocol.

uwsgi

Starting from v0.1.1, the uwsgi server is configured to listen to requests at port 8000 with the http protocol and port 8080 for the uwsgi protocol.

Please make sure that you are using port 8080 for uwsgi protocol.

It is advised to use the uwsgi protocol as it has better performance. Both Apache and Nginx have native support for the uwsgi protocol through plugins which are generally included in the distro packages.

To move to uwsgi protocol in the above nginx configuration use this

    # Nginx configuration.
    location /static {
        alias /opt/mailman/web/static;
        autoindex off;
    }

    location / {
          uwsgi_pass localhost:8080;
          include uwsgi_params;
          uwsgi_read_timeout 300;
    }

Please make sure that you are using v0.1.1 or greater if you use this configuration.

Serving static files

UWSGI by default doesn't serve static files so, when running mailman-web using the provided docker-compose.yaml file, you won't see any CSS or JS files being served.

To enable serving of static files using UWSGI, add the following environment variable to your docker-compose.yaml file under mailman-web:

UWSGI_STATIC_MAP=/static=/opt/mailman-web-data/static

It is recommended to use web-server to serve static files instead of UWSGI for better performance. You will have to add an alias rule in your web server to serve the static files. See here for instructions on how to configure your web server. The STATIC_ROOT for you would be /opt/mailman/web/static.

SSL certificates

SSL Certificates from Lets Encrypt need to be renewed every 90 days. You can setup a cron job to do the job. I have this small shell script (certbot-renew.sh) that you can put up in /etc/cron.monthly to get the job done.

#! /bin/bash

cd /opt/letsencrypt/
./certbot-auto --config /etc/letsencrypt/renewal/MY_DOMAIN_NAME.conf certonly

if [ $? -ne 0 ]
 then
        ERRORLOG=`tail /var/log/letsencrypt/letsencrypt.log`
        echo -e "The Let's Encrypt cert has not been renewed! \n \n" \
                 $ERRORLOG
 else
        nginx -s reload
fi

exit 0

Please do not forget to make the script executable (chmod +x certbot-renew.sh).

LICENSE

This repository is licensed under the MIT License. Please see the LICENSE file for more details.