polarathene
commented
1 year ago Relevant changes to consider (dug up from a local branch experiment back in March 2023). I probably have some notes elsewhere laying around on my disk, if I find anything else that's relevant I'll update it here 👍
- Probably want to opt-in to log namespacing.
- Since Aug 2023 the default/encouraged config format changed from TOML to YAML.
Including Vector
While there is no official package in Debian, Vector supports a variety of install methods including their own third-party repo if preferrable.
Since we have packages.sh that may be a good approach. Alternatively, especially for ease of use (by not invalidating the packages.sh layer during implementation), one could copy the binary over like we do with the ClamAV database.
Dockerfile:
# Copy binary over from published DockerHub release:
# - https://hub.docker.com/r/timberio/vector
# - NOTE: Adjust the tag to current release tag (eg `:0.33.0-debian`), otherwise it won't be properly version pinned:
# - Besides `-debian` variant, they also offer `-alpine`, `-distroless-static`, `-distroless-libc`
# - They also now have `0.33.X-variant` and `0.X-variant` which will provide the latest minor or patch release
COPY --link --from=docker.io/timberio/vector:0.X-debian /usr/bin/vector /usr/bin/vectorThe image can be deployed as a separate container for side-car usage with the /var/log/mail directory and file source config, but since metadata is lost (especially for /var/log/mail/mail.log), this isn't as helpful. Vector can configure some parsing to infer log level of some entries based on content, same with service and facility. Vector docs for Docker image deployment.
Stashed changes
Rsyslog
This was probably just experimenting with rsyslog config. It was a stashed change with the only context "rsyslog file source" (EDIT: This snippet is roughly what was also shared here).
target/scripts/build/dms.conf:
module(load="imfile" Mode="inotify") #needs to be done just once
# File 1
input(type="imfile"
File="/var/log/supervisor/postfix.log"
Tag="tag1"
Facility="mail")
input(type="imfile"
File="/var/log/supervisor/postgrey.log"
Facility="mail")Amavis
There was also a similar one where it looks like I was experimenting with Amavis log configuration (applied to this DMS config):
$log_level = 4;
# $LOGDIR = "/var/log";
# $logfile = "/dev/stdout";
# $do_syslog = 0;Test suite adjustment
A change for tests to be more explicit about services being searched via their own log instead of generic /var/log/mail/mail.log output, such as a supervisord stdout log file.
Probably a bit of changes since, but gives a rough overview of approach taken to better focus the scope of log content (helpful for troubleshooting).
Setup
Add these log vars to this setup.bash helper method (_test/test_helper/common.bash is the legacy path as well until all tests are converted over to test/helper/setup.bash_):
# Log files for tests to grep:
export LOGFILE_AMAVIS=/var/log/mail/mail.log
export LOGFILE_POSTFIX=/var/log/supervisor/postfix.log
export LOGFILE_POSTGREY=/var/log/supervisor/postgrey.logHelpers
This is specifically interested in Postfix logs: /var/log/mail.log => "${LOGFILE_POSTFIX}"
Amavis
Replace all 4 /var/log/mail/mail.log instances with ${LOGFILE_AMAVIS}:
Postfix
This would require the following addition to main.cf (_detailed in orignal FR description above regarding info from this PR_):
# Logging (instead of syslog default, output to stdout for supervisor)
maillog_file = /dev/stdoutOnly requires Postfix logs: /var/log/mail/mail.log => "${LOGFILE_POSTFIX}"
virtual_transport alternative support:
All 6 instances here (1st one requires quotes to grep to adjust to double quotes for variable expansion):
And this one:
PERMIT_DOCKER test (also need to adjust quotes):
Postgrey
Log stdout to service specific file (replace these two lines, this is the only service that lacks that):
stdout_logfile=/var/log/supervisor/%(program_name)s.log
stderr_logfile=/var/log/supervisor/%(program_name)s.logReplace these two lines with "${LOGFILE_POSTGREY}"
georglauterbach
Context
Vectorfor easier and better documented configuration, offering more flexible log management than the currentrsyslogpackage supports (syslog focused only):rsyslogfor why to adopt Vector detailed here: https://github.com/docker-mailserver/docker-mailserver/pull/3134#issuecomment-1450935046Another maintainer approving interest in adopting Vector: https://github.com/docker-mailserver/docker-mailserver/pull/3134#issuecomment-1451031308
Description
This has been considered for a while. I just haven't properly documented / discussed an issue for going forward.
It's not likely to be a quick and easy change to implement, tests will likely need to be updated and current logging configuration reviewd/revised. Docs should also be added to better communicate the support to users that want to leverage it.
Alternatives
I think there was some improvements that could be done with
rsyslogconfig, but beyond that there were not many other appealing options within a container that I came across.remote_syslog2but had concerns with the lack of regular maintenance.Applicable Users
Anyone that needs better control of log management than we currently have. I am particularly interested in it as our current docs are lacking in this area, our log support with DMS is rather limited AFAIK.
IIRC, there was a concern with how
supervisordhandled logging which @georglauterbach raised upstream but his fix was rejected. I believe this was related to ANSI-based coloured logs and a workaround was implemented at least fordocker logsstdout. With Vector the log level can be treated as metadata or streams, similar to what syslog supports, but IIRC syslog was less flexibility on "facilities"/tagging for grouping streams, especially when handling multi-line logs.Are you going to implement it?
Yes, because I know the probability of someone else doing it is low and I can learn from it.
What are you going to contribute?
If this is ever contributed, it's probably going to be handled by me 😅
This is a low priority task due to effort required and ROI for users + maintainers vs other tasks in my backlog. I'm raising this FR for the benefit of others that may want to try tackle it if I don't get around to it.
For reference this PR adjusted services to centralize logging to
/var/log/maildirectory (which we support as a volume to persist for logs).supervisordwas integrated and introduced another location that services output log files to (although these often appear empty / minimal as services get configured to log elsewhere instead of stdout?)./dev/stdoutinstead of syslog (_to be handled by thepostlogdservice inmaster.cf_), which would then have supervisord write the log to apostfix.logfile in the supervisord directory in/var/log./var/log/mail/mail.logto grep for log entries across multiple services this resulted in many test failures.syslogoutput to a file.There are some concerns (as noted by the linked context resources).
rsyslogdoes, makingrsyslogeasier to replace).rsyslog, but I recall some issues with that like this issue.Cargo.toml? (seems there is somecargo buildflag support, and building with vendored (external?) deps)Feature is better to delay until
Dockerfilesees some more housekeeping and the testsuite is converted to thecompose.yamlrevision (presently still converting tests tosetup.bashhelper + refactoring tests)