zone-eu / zone-mta-template

Template application for ZoneMTA
39 stars 25 forks source link

configuration with javascript rather than toml #13

Closed ndmgrphc closed 3 years ago

ndmgrphc commented 3 years ago

I'd like to avoid these toml files and just merge over default config with my own config.js file.

https://github.com/zone-eu/zone-mta#configuration

I see people referencing js config files in zone-mta issues but there's no instruction on how to do this.

While I do not need dynamic config support while zone-mta is running I do need it during bootstrap and deployment (of course) relies on .env files and environmental variables which are very hard to keep track of in these toml files.

https://github.com/nodemailer/wild-config#loading-order

Can I just remove the ./config dir of zone-mta-template and replace it with this at ./config/default.js https://github.com/zone-eu/zone-mta/blob/master/config/default.js ?

I can't seem to get this to work because for some weird reason nodejs "process" isn't defined so I can't access process.env.

Even in my package.json with npm run config

    {
        "start": "node index.js --config=config/default.js",
        "config": "NODE_CONFIG_ONLY=true npm start"
    }

I need access to my env vars. Is this possible? I haven't read through all of this 'wild-config' project yet...

ndmgrphc commented 3 years ago

Sorry this config needs a refactor.

wild-config has 9 "stars" and is the foundation of this package. Is zone-mta even maintained? This needs to fast-forward 4 years. Happy to contribute to a total 2020 config refactor.

In all fairness it bills itself "Opinionated configuration management" and that's just it...opinions expire.

andris9 commented 3 years ago

Using ENV is basically not supported, or it is supported in a limited way, eg. you could use ENV values as arguments like this:

$ cd /path/to/zone-mta
$ export SMTP_PORT=1234
$ node index.js --smtpInterfaces.feeder.port="$SMTP_PORT"

and to check if the values were set properly (prints out active configuration structure):

$ NODE_CONFIG_ONLY=true node index.js --smtpInterfaces.feeder.port="$SMTP_PORT"

In general you would still have to modify the config files to set up a general structure and then use the few basic values that are truly dynamic as command line arguments.

The configuration is like this due to the main target of Zone-MTA and WildDuck Email server. These are run in dedicated hardware (not in VPS or Docker) as SystemD services and the app is structured in a way where all configuration files reside in the /etc folder. There has been no need to use ENV in any of the config files and so the support was never added.

ndmgrphc commented 3 years ago

Okay thank you. I was able to get it working thus far by reading through wild-config. You can actually just replace the entirety of the ./config dir with ./config/default.js and use process.env (as covered in wild-config docs). The wild-config readme covers this BUT you cannot pass --config=config/default.js as it doesn't execute the file as a module hence no process. It's weird, wild indeed.

Still, a very small change to make this (and zonemta) a happily conventional library.

ndmgrphc commented 3 years ago

One thing I'm still confused about is why the NODE_CONFIG_ONLY env in app.js doesn't seem to output a merged final config. I assumed this was the point.

andris9 commented 3 years ago

It does output merged config. This is the exact config that is going to be used. If it is just a small object, then merging either failed or something else went wrong. See my previous example to see that feeder.port is merged into main config.

ndmgrphc commented 3 years ago

Okay it's not for me. just running "node index" per the wild-config docs finds my config/default.js but it's not merged over zone-mta/config/default.js like (I would think) it should be.

It continues at this moment I'm baffled by 'core/dkim' in plugins in the config so now there is namespacing but regardless of what's in my default's js plugins/dkim.js my 'sender:connection' is never called.

plugins['core/dkim'] nor plugins.dkim "enabled" in any way ever execute 'sender:connection' observer as described in numerous issues over the past 3 years.

Every single issue I've had with zone-mta at all is config related; if not the toml files it's the fact that the config wasn't clear until I read the default.js comments.

Furthermore, look at https://github.com/zone-eu/zone-mta/search?q=dkim&type=issues -- it's 2020 DKIM is 100% required to inbox, period. This needs to be the number one documented item.

Sorry to ramble I would love to help I'm just trying to figure out how and where to contribute first.

ndmgrphc commented 3 years ago

Understand your response about config now. I think my advice to anyone finding this repo is to ditch config/.toml and replace that entire directory with a copy* of https://github.com/zone-eu/zone-mta/blob/master/config/default.js

Thus, ./config/**.toml becomes a single file ./config/default.js

This will also require removing the --config= arg pointing to the config dir as it appears that wild-config will look here first but sadly not deep merge over https://github.com/zone-eu/zone-mta/blob/master/config/default.js

What you get:

1) Documentation on every configuration item in detail 2) The ability to inject environmental variables for docker or any sane deployment

What you lose:

1) Compact config. Not the end of the world.

andris9 commented 3 years ago

This repo is an example template, so if you have any specific requirements that do not align with the way this template is structured, you should use zone-mta directly and completely ignore this example template.