More volumes needed?

[pjeby]

I notice that although people's mail is stored in a volume under /var/mail, there looks to be data in at least /var/lib and /var/spool that should be kept from one rebuild of the container to the next. I mean, if you shut down postfix with mail in the spool, then that mail would be quietly lost, no?

At the same time, there is other stuff in /var/lib that probably should not be in a volume, e.g. initscripts, logrotate, etc. Maybe configurations should be changed to move all the mail apps' /var/lib dirs to a /var/mail/lib? Or perhaps thy should be moved and symlinked to something under /var/mail? amavis, clamav, dovecot, fetchmail, gross, postfix, postgrey, and spamassasin all have /var/lib dirs.

Any thoughts? I'm thinking maybe have the Dockerfile move the relevant dirs and replace them with symlinks from the original locations, that way there is still only one data volume for everything that needs to be kept from one run to the next, and isn't certs.

[hardware]

Folder	Persistence	Reason
/var/lib/dovecot	Not needed	Diffie Hellman parameters & mountpoints only
/var/lib/fetchmail	Not needed	Empty
/var/lib/gross	Not needed	DBFile : /var/mail/gross/grossd.state
/var/lib/postgrey	Not needed	DBFile : /var/mail/postgrey/postgrey.db
/var/lib/spamassassin	Not needed	sa-update-keys file only
/var/lib/clamav	Not needed	Virus databases only, these are regenerated by freshclam
/var/lib/postfix	Not needed	Cache & lock file only
/var/lib/amavis/db	Not needed	Databases are only used for monitoring health and performance of amavisd instance, and for a cache of recent check results
/var/lib/amavis/.spamassassin	**Needed***	Spamassassin Bayes database & user score preferences
/var/lib/amavis/virusmails	**Needed***	Virus quarantine storage location
/var/spool/postfix	**Needed?***	Postfix queue directories

*Currently not implemented in this docker image.

• /var/lib/amavis/.spamassassin : Training SpamAssassin's Bayesian classifier implementation is broken in 1.0-stable : https://github.com/hardware/mailserver/issues/120 https://github.com/hardware/mailserver/issues/110 https://github.com/hardware/mailserver/issues/76 but fixed in 1.1-beta with rspamd : https://github.com/hardware/mailserver/issues/110#issuecomment-309279757
• /var/lib/amavis/virusmails : Maybe I will add it for in the next oldstable version when 1.1-stable will be released
• /var/spool/postfix : I should not directly manipulate the postfix queue directories, adding old queue files to a new queues can cause bad consequences. I think for a personal mail server with small load, persistence of queue directory is not needed but for a mail server with high server loads with postfix queue congestion/bottleneck this directory should be restored correctly. Currently, only /var/spool/postfix/etc is restored.

[pjeby]

Wow, that's an impressive table. :) Thanks for doing the research and adding the clarifications.

So, pending a future release, I should probably just set queue_directory in /etc/posfix/main.cf to override the queue location. The virusmails directory can be set as $QUARANTINEDIR from /etc/amavis/conf.d, and the parent directory for .spamassassin directory can be set as $helpers_home there as well. Both can be set from the command line via -Q and -S for amavisd-new, and so might be sufficiently settable via supervisord.conf.

[hardware]

You want to make a pull-request with these changes ?

[pjeby]

Might as well if I'm making the changes anyway, right? :wink: I was planning to try them in the morning anyway, so I'll throw it back as a PR if it works. Against master, right?

[hardware]

I was planning to try them in the morning anyway, so I'll throw it back as a PR if it works

:+1:

For the volumes paths, you can name them like this :

/var/mail/postfix/spool
/var/mail/amavis/quarantine
/var/mail/amavis/bayes

"Against master" ?

[pjeby]

I mean, do you want the PR to be done against the master branch, as opposed to the 1.1 branch?

As for the naming, amavis doesn't control the .spamassassin name -- it merely sets HOME to /var/lib/amavis and spamassassin does the rest. So we can have it set HOME to /var/mail/amavis, and end up with /var/mail/amavis/.spamassassin as the db.

[hardware]

master branch because 1.1 doesn't have amavis and spamassassin.

[pjeby]

The patch was a lot harder than it looked. :smile: The .spamassassin fix was trivial, but the quarantine dir isn't getting made and I'm not sure whether that's normal and if so how to test it.

Postfix, on the other hand, was being a super jerk, what with /etc being full of stuff that references /var/spool/postfix, and there also being a /usr/lib subdir that needed to exist. Making /var/spool/postfix a symlink to the volume fixed those issues.

But the weirdest part of all is where, if I tried to replace /var/spool/postfix too late in the Dockerfile, the build would fail with all kinds of weird problems. From what I could track down, it's an issue with overlayfs and the linux kernel. Moving the commands high up in the RUN block works around it; that's why the new commands aren't just added at the end.