Allow specifying data and config dirs for quickstart

[daftaupe]

Following what was said on the IRC channel, and for ease of packaging, it would be nice to be able to run the quickstart with predetermined data and config dirs that are different from the default dirs.

That would help because people could run mox quickstart by specifying the proper args and end up using what the distribution package expects them to use.

[mjl-]

Indeed!

We could add cli flags to quickstart to set the directories. But we could go one step further: Make sure mox can look in /etc/mox/ (assuming that's the location) for mox.conf (instead of in ./) without any cli flags/env vars. The mox.conf points to the data dir, with the ctl socket over which many cli subcommands are sent. So it's nice when that works out of the box. That currently only works if you run mox in the same directory as the mox.conf file. Otherwise you need to specify the MOXCONF env var or -config flag.

Perhaps we should have per-OS default directories for where to look for mox.conf (it'll be /etc/mox/ for all unix systems, which is essentially all of them). Perhaps there is a good default directory for the data dir too, somewhere in /var, perhaps that will vary more per OS. Packagers could change the default directories by passing new values for some global variables to the linker when building. Quickstart could write to /etc/mox and /var/somewhere by default, so we would instead add a flag for writing mox.conf and domains.conf to the current working directory (instructions would have to be updated). If we write to /etc and /var/ by default, we would have to do something differently for the systemd unit file too, where to write it?

Not entirely sure what the best way forward is. Does the package write its own service file? Should there be an option to prevent it being created by mox?

[lmeunier]

My 2 cts...

I must admit that I'm not a big fan of applications that "force" users to use a command to have a configuration file. I prefer when applications provide an example or default config file (with good comments, as this is a case for mox.conf, thanks for that!) so that I can edit/create my own config file. One step further would be to have the possibility that mox read a default config file and then read my own config file that overrides the default configuration. This way, if the default configuration is ok for me, I've nothing to do. But if, for example, I just want to change the data location, I'll create a config file with only the DataDir parameter. The default config file should be managed by the OS package, so the default configuration is updated with new mox version and I just have to check if my own config file is still usable with the new mox version, instead of creating (again) a complete new config file with mox quickstart (maybe the old config file generated with "quickstart" is obsolete and can't be used with the new mox version, as a regular user how could I know that?). For example, the default config file could be /etc/mox/mox.conf (this file is managed by the OS package and should never be modified by the user) and mox could read additional config files in /etc/mox/conf.d/*.conf (files managed by the user).

About default directories, most linux distributions will use /etc/mox for configuration and /var/lib/mox for data. Those paths should be the defaults, but users should be able to change them. I think this is already possible with the -config flag and the DataDir parameter. For security reason, /etc/mox should be only writable by root and /var/lib/mox should be only writable by the mox user. That being said, domains.conf should be in the data directory (as it's a file handled directly by mox through the web interface, we can consider this file to be data), I'm not sure for the adminpasswd file and the dkim folder. The systemd service file should be provided by the OS package.

That being said, I know that mox quickstart is not just a command that generate a configuration file based on the given parameters. This command will also check a lot of things and give useful informations to the user (like the admin password and DNS records). mox should keep this feature.

It's just my opinion and I'm sorry if my comment is a bit off-topic. Be aware that I'm already happy with the current situation :)

[mjl-]

i'll expand more below, discussion appreciated. i think changes to config file handling will be needed long-term. but it may take some time before the right approach emerges and is implemented.

i think a short-term improvement is to add quickstart cli flags for the destination config files, and a global variable for the default path to look for mox.conf that can be set at link time. diff attached. quickstart.diff.txt

next question is: what to do with the service file and instructions for installing/starting as service. you can't start mox without creating a config file (i think some distro's like starting services with default/empty config files after installing). i suppose the instructions from the package maintainer will be: "run quickstart, then run these commands to start and enable as system service: ...".

we may need a compile-time option to skip writing the .service file in the quickstart so the package maintainer can provide it.

more discussion:

edit/create config files manually default values and overrides in separate files updates & compatible config files

on one hand i'm a fan of simply editing config files. (and having those config file in version control). on the other hand, i find a web interface to make some changes often more convenient. web interfaces can also provide more guidance. part of the goal of mox is to make it easy to run a mail server, i think a web interface helps with that. so i want to keep both manual config file editing, and managing through the web interface. it means we may have to deviate a bit from the standard of managing config files.

having /etc/mox/conf.d/*.conf seems reasonable. an /etc/mox/mox.conf could be simply an include of those files. the quickstart would just generate /etc/mox/conf.d/00-quickstart.conf. however, that would probably make it unfeasible for mox to ever change those files through the web interface or during updates. mox.conf has the "static" config options where a change requires a restart to take effect; but those options are in mox.conf and not in domains.conf (a bit of a misnomer, the internal name "dynamic config" fits better) mostly because it was too hard to handle changes to those options at runtime before getting the first release out. i'm not against making more "static" config options dynamic in the future. but if everything can be changed at runtime, there would be no need for a static mox.conf anymore, it would all be in the dynamic config file. on the other hand, i see the value of having a config file with important options that cannot be changed from within mox, but requiring root user access...

the config files are designed so missing values have reasonable behaviour. to make introducing new config options (for new features/behaviour) behave sanely (the idea aligns with usefulness of zero values in Go). new versions of mox should always work with the previous config files. also the static mox.conf. there isn't always an OS package around to do updates, e.g. in current approach installing in /home/mox but also with docker. i'm not against automatically rewriting the config files (especially the dynamic config file) during an update in the future (to prevent the config file from becoming too awkward with changed/new features/behaviour).

most of the values in the quickstart-generated are necessary or good to make explicit (also serving as documentation).

i'm hoping to get mox to upgrade itself in the future (part of making it easy to run). with a "mox upgrade" command and/or a web interface button to upgrade. config compatibility and/or config file rewriting will be a requirement then.

default directories, /var/lib/mox on linux

it seems bsds use /var/<appname>, though i'm not sure based on openbsd and freebsd hier(7) man pages and other docs. on openbsd some applications are chroot'ed in /var/<appname>, and they have their on hierarchy again with etc/, var/ (probably because software assumes that layout, but might as well not bother with unix hierarchies in such cases).

domains.conf in /var, other files...

i think you're correct that files in /etc/ are typically not expected to be modified by applications themselves. but it feels like /etc/ is the right place for configuration files, and /var for data... i consider most of the config options (like the configured domains, accounts, webserver config) to be "configuration", not "data".

most domains.conf fields can only be changed (through the mox cli commands and web interfaces) by the admin only. but some account fields can be changed by accounts themselves. that means making domains.conf read-only for the mox user isn't a good idea at the moment. i think those account-settable config options are still "configuration options", not data. they could be moved to the per-account database, but i prefer them in files so i can diff and vcs-manage them. perhaps they should be in another config file that is mox-process-writable (allowing you to make domains.conf mox-process read-only). the line between "config" and "data" is blurry. the dkim keys and host keys are currently in config/, the acme tls certificates are in data. in the future, it would be nice if mox can rotate dkim & host keys itself in the future, making data a better fit...

perhaps the ctl socket should be in some "run" directory... but i wonder what the advantage would be, other than adhering to an OS standard. as an admin, i like to have files for 1 app in as few places as possible. and i think we're heading more to per-app isolation (like with chroot, docker or other mechanisms) over time.

mox quickstart does more than creating config files, keep it.

indeed, it should be helping people set up mox, making it easy to run a mail server. (: i'm also envisioning a "mox setup" that provides a guided web interface, doing sanity checks along the way (e.g. check dns the settings after you've adding dns records, checking connectivity again after you've opened outbound port 25, etc). it would also write a mox.conf and domains.conf.

[RobSlgm]

Here's another point regarding the unclear separation between /config and /data directories. As I understand it, the current mox backup only captures the /data directory. This means critical configuration files, especially domains.conf, are not included in the backup.