FR: Allow file based communication (sendmail-like/unix socket; no networking/ports)

[RafaelKr]

Use case summary

• Starting up many local projects in parallel
• Hosting many projects on one server

Use case 1: Starting up many local projects in parallel

I'm working at an agency and sometimes we need to spin up multiple projects (and mailpit instances) in parallel.

We're using https://devenv.sh/ to configure our projects, so we just have to set services.mailpit.enable = true in our devenv.nix file and have a running mailpit instance. Data is stored in ./.devenv/state/mailpit/db.sqlite, the path is relative to the project. Also see https://github.com/cachix/devenv/blob/d612b77ff73912cd82e58256ab5e84d5904abef7/src/modules/services/mailpit.nix#L47 and https://devenv.sh/reference/options/#servicesmailpitenable

The mailpit instance listens on 127.0.0.1:1025 and 127.0.0.1:8025 by default, so if we want to spin up more than one project in parallel we always need to temporarily change the ports of at least one project and then change them back to defaults later. I also thought of reserving 2 free ports per project, but I probably worked on 100+ projects by now. Always reserving new ports per project doesn't seem feasible, also sharing the project configuration with colleagues isn't easy then.

Use case 2: Hosting many projects on one server

We're hosting many NixOS servers, they're configured declaratively. Those servers host multiple projects as virtual hosts. Most projects have a dev and staging environment where we're using a mailpit instance per virtual host.

Currently we spin up multiple mailpit instances and always need to search for free ports for the mailpit UI and the SMTP services. Then we need to define the chosen ports in the configuration. The UI is configured in the reverse proxy to be reachable via https://www.example.com/mailpit/ (protected via BasicAuth).

Other Service Examples

For services like the Database or redis I'm able to configure them to listen via unix sockets, so we can specify them as

# one global socket, authentication by system user
DB_SOCKET=/run/mysqld/mysqld.sock

REDIS_PATH=/run/redis-project-1-dev/redis.sock
REDIS_PATH=/run/redis-project-1-staging/redis.sock
REDIS_PATH=/run/redis-project-1-prod/redis.sock

REDIS_PATH=/run/redis-project-2-dev/redis.sock
REDIS_PATH=/run/redis-project-2-staging/redis.sock
REDIS_PATH=/run/redis-project-2-prod/redis.sock

As you can see we can just use a generic project name (project-1, project-2, ...) to automatically reserve sockets. Or in the case of the database, where only one instance is running, the context (accessible databases, database permissions, ...) depends on the system user under which the application is running.

So unix sockets have another advantage from a security perspective: We can set unix permissions on them to control which user has access to them. In our server environments every vhost has an own user. The application is running as that user. And the redis instances (started via systemd) are running as the same user and a socket permission is set to 600, so only the vhost user can read and write to their own socket.

Mailpit Feature Request

It would be nice to be able to configure mailpit in a similar way.

SMTP

For the SMTP configuration it could be a sendmail-compatible wrapper file.

# Wrapper binary (not sure if we could use a unix socket here?)
MAILER_DSN=sendmail://default?command=/run/mailpit/project-1-dev/mailpit-sendmail
MAILER_DSN=sendmail://default?command=/run/mailpit/project-1-staging/mailpit-sendmail

MAILER_DSN=sendmail://default?command=/run/mailpit/project-2-dev/mailpit-sendmail
MAILER_DSN=sendmail://default?command=/run/mailpit/project-2-staging/mailpit-sendmail

# currently
MAILER_DSN=smtp://localhost:1031 # project-1-dev
MAILER_DSN=smtp://localhost:1030 # project-1-staging
MAILER_DSN=smtp://localhost:1041 # project-2-dev
MAILER_DSN=smtp://localhost:1040 # project-2-staging

Configuration examples are for Symfony:

• MAILER_DSN transport syntax: https://symfony.com/doc/current/mailer.html#transport-setup
• sendmail command syntax: https://symfony.com/doc/current/mailer.html#other-options

Mailpit UI

See: https://mailpit.axllent.org/docs/configuration/proxy/ For the reverse proxy (nginx in our case) something like:

server {
    server_name dev.project-1.com;
    auth_basic secured;
    auth_basic_user_file /path/to/nginx-auth/project-1-dev.htpasswd;

    # ...

    location ^~ /mailpit/ {
        # currently: proxy_pass http://127.0.0.1:8031;
        proxy_pass http://unix:/run/mailpit/project-1-dev/mailpit-ui.sock;
        # configure the websocket
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }
}

server {
    server_name staging.project-1.com;
    auth_basic secured;
    auth_basic_user_file /path/to/nginx-auth/project-1-staging.htpasswd;

    # ...

    location ^~ /mailpit/ {
        # currently: proxy_pass http://127.0.0.1:8030;
        proxy_pass http://unix:/run/mailpit/project-1-staging/mailpit-ui.sock;
        # configure the websocket
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }
}

server {
    server_name dev.project-2.com;
    auth_basic secured;
    auth_basic_user_file /path/to/nginx-auth/project-2-staging.htpasswd;

    # ...

    location ^~ /mailpit/ {
        # currently: proxy_pass http://127.0.0.1:8041;
        proxy_pass http://unix:/run/mailpit/project-1-dev/mailpit-ui.sock;
        # configure the websocket
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }
}

server {
    server_name staging.project-2.com;
    auth_basic secured;
    auth_basic_user_file /path/to/nginx-auth/project-2-staging.htpasswd;

    # ...

    location ^~ /mailpit/ {
        # currently: proxy_pass http://127.0.0.1:8040;
        proxy_pass http://unix:/run/mailpit/project-2-staging/mailpit-ui.sock;
        # configure the websocket
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }
}

# ...

Proposed mailpit configuration options

We need to be able to specify a socket path and also need to be able to specify the socket/mailpit-sendmail permissions. My idea would be to have the following configuration options:

• MP_UI_BIND_ADDR: allow configuration of sockets or create an own configuration option (important thing is to be able to disable networking so there will be no EADDRINUSE errors due to port collissions)

• MP_UI_SOCKET_PERM: octal permissions, see redis unixsocketperm (references below), default to 600

• MP_SMTP_BIND_ADDR: see MP_UI_BIND_ADDR

• MP_SMTP_SOCKET_PERM: see MP_UI_SOCKET_PERM, default to 700 if it is a wrapper binary or 600 if it's somehow possible to use a socket

Socket owner (user) and group should be determined from the mailpit instance user and group.

References:

• redis unixsocket and unixsocketperm, also needs port 0 to disable networking: https://redis.io/docs/latest/operate/oss_and_stack/management/config-file/
• MariaDB Remote Client Access: https://mariadb.com/kb/en/configuring-mariadb-for-remote-client-access/
• MariaDB Unix Socket Authentication: https://mariadb.com/kb/en/authentication-plugin-unix-socket/
• Caddy Webserver bind syntax for sockets: https://caddyserver.com/docs/caddyfile/directives/bind#examples
• Dovecot/Postfix LMTP (Local Mail Transfer Protocol): https://doc.dovecot.org/2.3/configuration_manual/howto/postfix_dovecot_lmtp/
• gitea Issue Allow SMTP over unix socket: https://github.com/go-gitea/gitea/issues/18901

Advanded: Abstract Namespace Sockets

Maybe interesting for an implementation of the mailpit-sendmail wrapper if it needs to be a binary instead of a socket. The to communicate with the mailpit instance process could be implemented with that.

• Linux's abstract namespace for Unix domain sockets: https://utcc.utoronto.ca/~cks/space/blog/linux/SocketAbstractNamespace
• The net package of Go already supports Abstract Namespace Sockets out of the box: https://golang-examples.tumblr.com/post/92025745979/pitfall-of-abstract-unix-domain-socket-address-in
• nginx Unit also supports reverse proxying to abstract namespace sockets, so maybe it would also be interesting to support them for the mailpit UI socket: https://unit.nginx.org/news/2022/unit-1.28.0-released/#abstract-unix-domain-sockets
• They even have the best performance: https://github.com/mschwager/socket-speed

[axllent]

Thanks for your detailed feature request @RafaelKr ~ very useful.

This is a very interesting idea as it does provide an alternate approach to the whole "port issue" ~ which has been raised a couple of times before by others implementing Mailpit in large multi-user/customer environments with shared hosting.

Forgetting briefly that Mailpit is not limited to just *nix platforms (ie: I do not believe unix sockets are supported on Windows)..... theoretically this idea could work. I just did a (very crude) test running the Mailpit HTTP server via a unix socket which Nginx connected directly to (as per your example) and it worked great (including the websocket).

But ... I'm not sure about the SMTP server because the mhale/smtpd module (that Mailpit uses) does not currently support sockets (it's hardcoded to use tcp). I would need to fork and test to see if that would even work with a unix socket.

Assuming it all worked, to implement this properly will be quite a lot of work as there are several moving parts, including of course the SMTP server & SMTP client, plus testing & of course documentation. I'm not sure when I can make the time available as I'm scheduled to have shoulder surgery in about 6 weeks time which will put me "out of action" for a couple more months.

To give me some idea of your need for this feature, could you please tell me what kind of scale are you wanting to use this for (ie: how many projects would be using this if it was implemented), and how & if you (or your company) could potentially contribute? Thanks.