mqtt-tools / mqttwarn

A highly configurable MQTT message router, where the routing targets are notification plugins, primarily written in Python.
https://mqttwarn.readthedocs.io/
Eclipse Public License 2.0
955 stars 183 forks source link

Environment variables for sensitive configuration file values #560

Open tobru opened 2 years ago

tobru commented 2 years ago

The configuration file contains sensitive data like passwords. Is it possible to pass values via environment variables into the configuration of mqttwarn? That would allow me to e.g. share my configuration in public, without having to place passwords into the configuration file.

jpmens commented 2 years ago

I don't think this is yet possible, but it is a good idea, and maybe we could take this a step further.

Whenever a password is required in the service code, the service invokes a function pass() with the configured secret. If the secret begins with a special token, we obtain the clear text password appropriately, otherwise, we use the verbatim password.

  1. pass("$ENV:SECRET") would use the value from the environment's $SECRET
  2. pass("$FS:/etc/mysecret") would use the value from the first white-space trimmed line in /etc/mysecret
  3. pass("bla17") would use the password "bla17"

Using these special tokens would mean that in the above example a password must not begin with either $ENV: or with $FS: (regex ^\$[A-Z]+:), but I think we should be able to risk that.

This is also extensible; imagine $ETCD: or $GPG:, etc. Imagination can now run wild.

tobru commented 2 years ago

This proposal sounds great, it would extend the possibilities to have sensitive data stored on another (secure) place than the non-sensitive configuration data.

amotl commented 2 years ago

Hi Tobias and JP,

I also believe this would make a sweet feature, kudos!

Because ConfigParser already has interpolation capabilities [1], with specific examples how to expand environment variables [2,3], I would suggest building this feature on top of this in a generic way instead of having to tweak all service plugins by sprinkling calls to getpass(), or similar.

Attaching to the suggestion by @jpmens to support multiple password backends, I would like to add HashiCorp's Vault to the list. From the perspetive of Salt, where I just happen to have an example at hand, reading secrets from Vault (in this case, a TLS private key) looks like salt['vault'].read_secret('acme/infra/files/www.example.org.key').

With kind regards, Andreas.

[1] https://docs.python.org/3.6/library/configparser.html#interpolation-of-values [2] https://newbedev.com/configparser-and-string-interpolation-with-env-variable [3] https://gist.github.com/malexer/ee2f93b1973120925e8beb3f36b184b8

psyciknz commented 1 year ago

Hello again. Did this function get shipped? I'm trying to push all keys etc to environment vars, so for example my pushover config, Id like to use the following:

'HomeAssistant' : [ '${PUSHOVER_KEY}', 'apjvopwx2jrqbdzaneh6tzvskq1b84', 'none'],

Where PUSHOVER_KEY is an environment variable for the container. Which I can see when I post an env in the container. Odd that for the mqtt client name I have clientid = 'mqttwarn-${HOSTNAME}' Which gets resolved properly.

amotl commented 1 year ago

Dear @psyciknz. Thank you for asking. We did not work on this feature yet.

jhakonen commented 1 year ago

I have a need for this feature as well so I've written a pull request (#681) that implements it.