Mailtrain-org / mailtrain

Self hosted newsletter app
GNU General Public License v3.0
5.53k stars 692 forks source link

Summary of why MailTrain needs MariaDB + Redis + Mongo? #1280

Closed dm17 closed 8 months ago

dm17 commented 2 years ago

I'm trying to make a lightweight mailtrain config and wondering what each database is required for - and why it can't just use one. Perhaps this should be obvious, but so far it isn't. We already have our own databases running - lots of stuff for members, including their email addresses. What does MailTrain use each DB for?

Also, has anyone integrated an auth or SSO service to manage user access to the mailtrain stack?

Is there a community chat or forum - hopefully this is the right place to ask these questions? Thanks!

bures commented 2 years ago

Mongo is needed only if you use the builtin ZoneMTA. I guess redis is the same case. If you use an external MTA, feel free to drop them.

dm17 commented 2 years ago

@bures Thanks - using the built-in ZoneMTA (not sure what it is) as opposed to what? I'm mostly using it for the Mosaico editor's mailing list compatibility and ease connecting MailTrain with our SMTP relay. In terms of importing/exporting each mailing list from our other database; I've not figured out the best way yet (whether to export a CSV or actually integrate such that MailTrain has DB access).

saper commented 2 years ago

MTA (Message transfer agent) is a program that does the hard work of sending email to maybe thousands of recipients. ZoneMTA which needs those things comes installed by default, but there others, like Sendmail, Postfix, Exim and few more.

saschafoerster commented 2 years ago

I tried to turn off Mongo and Redis in my docker-compose.yml but that alone was not enough. I played a little bit around with the local- and production-config files, but no success. So in the end I have restarted everything with mongodb and redis, even though I am not using ZoneMTA but SMTP. So help or experiences are appreciated. 👍

saper commented 2 years ago

I don't use mailtrain yet, but it is hard to tell what the problem is without specific error messages. And if you are using SMTP, which message transfer agent it is connecting to?

a1ro commented 2 years ago

I use an external SMTP server, so I disabled both the builtin ZoneMTA and MongoDB for my Docker instance. I sent a test campaign with this setup and it's working great so far.

For ZoneMTA, I had to disable it in server's production.yaml along with setting a Docker environment variable. For MongoDB, the service and volume can be commented out in docker-compose.yml.

So, while Mailtrain is running:

# execute a shell in container
docker-compose exec mailtrain /bin/bash

# edit server config
vi server/config/production.yaml
# set existing entry to false
builtinZoneMTA:
  enabled: false
# return to host shell
exit

Then, in the Compose file:

vim docker-compose.yml
# comment out Mongo service and volume
# add a variable to disable ZoneMTA

services:
  # mongo:
  #   image: mongo:4-xenial
  #   volumes:
  #   - mongo-data:/data/db

  mailtrain:
    environment:
    - WITH_ZONE_MTA=false

volumes:
  # mongo-data:
# stop running containers
docker-compose stop

# recreate containers
docker-compose up

# Optional:
## remove Mongo container and image with `docker-compose up --remove-orphans`
## remove Mongo volume with `docker volume rm mailtrain_mongo-data`

From Mailtrain's Admin UI, in Administration > Send configurations, I created a send configuration with Generic SMTP mailer type.

Note that the Built-in ZoneMTA dynamic config is no longer available when selecting the Zone MTA mailer type.

a1ro commented 2 years ago

As for Redis, I get a few warnings after disabling it. Looks like Mailtrain relies on it other than for ZoneMTA, so I guess it should be kept enabled for now.

Mailtrain log with Redis disabled:

Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.

info Express WWW server [trusted] listening on port 3000

Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.

info Express WWW server [sandbox] listening on port 3003

Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.

info Express WWW server [public] listening on port 3004

Changes in production.yaml to disable it:

redis:
  enabled: false

Changes in docker-compose.yml to disable it:

services:
  # redis:
  #   image: redis:5
  #   volumes:
  #   - redis-data:/data

  mailtrain:
    environment:
    - WITH_REDIS=false

volumes:
  # redis-data:
talheim-it commented 8 months ago

We are going to start with the development and testing of mailtrain v3 in the next weeks.

You are welcome to help us with the testing as soon as the first release candidate is available.