Configuration Management

[kc-dot-io]

overview

as per issue #24 we want to make this repo more focused on configuration management as a whole. When it comes to reading configs, we'll look to node-config for this rather then re-invent that functionality. We should also build out the repo to have support for more complex production configuration management storage such as vault, consul and etc so that we can not only protect secrets but also eventually support service discovery.

goals

• [ ] create base Configuration class
• [ ] create base Adapterclass
• [ ] Adapters should be able to read and write
• [ ] a configuration will have default Adapters
• [ ] a configuration can be extended with one or more storage Adapters
• [ ] support multiple Adapters
  • [ ] process.env (default)
  • [ ] json files (default)
  • [ ] vault
  • [ ] consul
  • [ ] etcd
• [ ] Adapters will be implemented with loading precedence
• [ ] proposed loading precedence: process.env.feathers, json files, storage (vault, consul, etcd)
• [ ] a value that is already set will not be overridden by subsequent Adapters
• [ ] conflicts will be logged for user transparency
• [ ] processes will manage config via app.get and app.set
• [ ] operators will manage storage configs via feathers config get and feathers config set
• [ ] operators will need to pass explicitly the --storage when managing a configuration
• [ ] configuration will default to always reading from process.env and json files
• [ ] if storage options are passed, then additional configurations will be merged
• [ ] an applications should be able to simply listen for new configurations added at any name space

  examples

  implement

const feathers  = require('feathers');
const configuration  = require('feathers-configuration');
const etcd =  require('feathers-etcd')(['http://127.0.0.1:2379']);
const vault = require('feathers-vault')(['http://127.0.0.1:8200', process.env.VAULT_TOKEN]);

feathers()
  .configure(configuration()) // process.env.feathers + json files by default
  .configure(etcd())
  .configure(vault())

service discovery

// listen to a specific storage adapter directly
vault.on('create', function(key, value) {
  console.log(key, '=>', value); // /databases/mongo/production => mongodb://mongo.stack.local:27017
});

//listen across all storage adapters that support pub sub
configuration.on('create /databases/mongo/*', function(key, value) {
  console.log(key, '=>', value); // /databases/mongo/production => mongodb://mongo.stack.local:27017
});

vault.create({ 
  key:'/databases/mongo/production', 
  value: 'mongodb://mongo.stack.local.27107'
});

workflow

// 1. loads from process.env
// 2. loads from json files and merges
// 3. loads from etcd and merges
// 4. loads from vault and merges
// 5. listens to each storage
// 6. notifies of changes in storage

[marshallswain]

Should configurations be manageable via a public feathers service? (not very secure)

This can be made secure. One way would be to create a second, separate auth endpoint just for configuration tokens. Enabling IP-aware auth or multi-factor auth would help.

Should we bundle adapters or treat them as standalone plugins that extend feathers-configuration?

I would imagine we'd want them separate, just like the other adapters.

[kc-dot-io]

Cool, I think I can work with that. Typical best practice would be to put your configuration storage in a private subnet so there is no outside access, perhaps in a case where there is we can simply have before hook that validates based on IP or MFA like you suggested.

10-4. I think that looks best as well, however it might make it hard to centralize the service discovery events if you use multiple storage engines, unless I add a way to bind them at run time. I'll play with that idea a bit.

[ekryski]

Ya so far I'm really liking that direction. Not much to add after @marshallswain's comments.

[daffl]

Closing since it has been open for a while and I think it may now be out of scope of this little module. Happy to follow up some time in the future.