search

tmaiaroto / discfg

A distributed, serverless, configuration tool using AWS services
Apache License 2.0
75 stars 4 forks source link

Import/Export config #6

Open tmaiaroto opened 8 years ago

tmaiaroto commented 8 years ago

Something not seen in etcd, an export configuration command. One could use the REST API to get the root key with the recursive option to get a full JSON tree structure of the entire config...But from the CLI it should be possible to dump the entire configuration to a JSON file.

This point of this is save, share, and/or "cache" configurations. While it's very critical to have up to date configurations for many situations (especially for systems concerns), it's not always so critical for application concerns. In fact, configurations may not update all too frequently at all for applications. This means if you use discfg for your web app, you'll be unnecessarily slowing it down with remote requests for the data unless the config is cached somehow. This is a concern of the application and maybe discfg helper libraries can be built in the future...But from the ability to save a JSON file from the CLI is a very convenient thing.

tmaiaroto commented 8 years ago

Though this package solves the problem for etcd. https://github.com/mickep76/etcd-export ... Maybe look at that for some ideas. +1 for TOML support.

tmaiaroto commented 8 years ago

Note that values are stored as byte arrays and it is becoming more expected to see binary data in there. Not just strings. So this export still works, but note that can be of little benefit when there's a string representation of some image in the middle of your JSON or TOML export.

Having a content type field may be really important here (so it's known what to skip).

tmaiaroto commented 7 years ago

Have to think about this some more... There are some limits with DynamoDB and also AWS Lambda (the intended host for discfg).

If the configuration is quite large, export/import begins to take some time and then there's the size concerns as well. Depending on the size of the config, there may even need to be some batching going on. So this is definitely a much larger feature than originally anticipated.

It's pretty useful to have, but I think it isn't pressing and shouldn't hold up the boat. Will definitely need feedback here.