search

DataDog / kvexpress

## Auto-archived due to inactivity. ## Go program to move data in and out of Consul's KV store.
Apache License 2.0
129 stars 13 forks source link

Comparison with git2consul and consul-template? #90

Open doublerebel opened 8 years ago

doublerebel commented 8 years ago

Hello,

kvexpress looks like a great tool, and easier than setting up my own workflow using git2consul+consul-template. The watch-and-write-config-file-then-do-command seems to duplicate consul-template, but in the README and blog it seems like this is to be used in conjunction with consul-template? Is consul-template meant for varying data whereas kvexpress uses checksums to verify static config-file contents from input to output?

Thanks for any details or experience you can provide.

darron commented 8 years ago

You've got it pretty much correct in your description - but there's one main thing I'd add:

kvexpress is really about delivery of files - it doesn't do anything to them in transit.

We use kvexpress with Consul Template a couple of ways:

  1. We use Consul Template to help to output complete configuration files - and use kvexpress to deliver it to all of the nodes.
  2. We use git2consul and Consul Template to feed more complete files to kvexpress - which then deliver it to all nodes.

We're looking at removing Consul Template from at least part of the process. For number 1 it's crucial to still keep using it - it fulfils a very useful purpose. For number 2 it's involved in the git2consul to kvexpress translation - but it's a bit cumbersome and I'd like to remove it.

In order to remove it for that, I need to have the abillity to make kvexpress watch a set of git2consul keys and then update the kvexpress keys that change - right now that's handled by some Consul Template watches.

Does that help clarify at all?

doublerebel commented 8 years ago

That's a big help, thanks. It would be great to have all this info in the README about how the different parts work together.

Can I ask why you made the decision to have consul-template render locally and have kvexpress render on the nodes? Was consul-template creating the problems you are guarding against? e.g. zero-length files, files different than expected (checksum), unsorted data

darron commented 8 years ago

The reason we used Consul Template to render on a single node and then distribute the result with kvexpress was because:

  1. The file is the same everywhere - it doesn't need to be created on every node from Consul service data.
  2. At the time, Consul 0.5 had some significant performance issues that meant we couldn't build it on each node.

Here's the bug report we filed:

https://github.com/hashicorp/consul-template/issues/331

And the detailed test case to reproduce it:

https://github.com/darron/consul-bug-report

Consul Template didn't have the issues you described - it just couldn't perform in our environment due to the limitations of Consul 0.5.

I'll look at adding some more documentation when I've got a moment.

doublerebel commented 8 years ago

Thanks again. That consul-template issue is another good link for the docs, now I get it.

Having consul-template render only once and then distribute makes a lot of sense, the more nodes in the cluster. consul-template seems to have reduced consul load by adding max-stale: 1s in 0.11 so that consul-template can talk to followers, but a single render for a homogeneous app deployment would be the best solution.

You're welcome to close this issue whenever you see fit.

darron commented 8 years ago

Consul Template also added a deduplicate mode to help with some of the issues we raised:

https://github.com/hashicorp/consul-template#de-duplication-mode

Sadly it wasn't available at the time we were discovering the limitations.

I discuss some of this in my Scale14x talk - might be helpful.

I'll close once I add some better documentation.