Feature Request: Key-value update history

[JordyMoos]

We would like to have a key-value update history if thats possible. We use the key-value store a lot for updating our parameters.yml files over all the instances (its a symfony configuration file). And there are a lot of squads making the update. It could be that something goes wrong and we need to revert the last change.

We can off course build our own wrapper around the api which stores changes itself but it might be a nice feature for consul.

[F4bwDP6a6W]

upvote

[slackpad]

Hi @JordyMoos and @hot13399 - we've thought about this as well but it adds a lot of complexity to Consul and creates issues around managing an ever-growing history of changes. I'd suggest using something like https://github.com/Cimpress-MCP/git2consul and not having folks change Consul's KV directly. This will get you the benefits you are looking for, and all the tooling that's available for Git to manage users, workflow, etc.

[maxenglander]

This is too bad @slackpad. As to the ever-growing history of changes, what if the amount of history per key were user-tunable, e.g. a user could set a "history_max_entries_per_key" in agent config to ensure no key could have more than 10 items in history?

[slackpad]

Not sure of a time frame on this, but kicking back open for further consideration. It's closely related to https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/consul-tool/5XyxyQgsE1g/Mot-o4VeBAAJ.

[abraithwaite]

To clarify: I would be totally happy with just getting all new updates via a stream after making a call to consul. Doesn't have to be the entire history. Also doesn't have to be "everything after index". Just "everything after this call has been made".

[abraithwaite]

Can we split this into two issues? A streaming watch interface vs getting historical changes?

[jkirschner-hashicorp]

We do not plan to support a mechanism for storing/retrieving the history of key-value data for the reasons originally given by @slackpad, so I'm closing this issue.

As to @abraithwaite 's question about a streaming update interface as an alternative approach, we could review/accept a PR from the community which implements that in the future. Two relevant Github issues already exist for a streaming update interface:

• https://github.com/hashicorp/consul/issues/2791
• https://github.com/hashicorp/consul/issues/6366

Please follow up on / subscribe to the issues linked above if interested in a streaming update interface. Also let us know if you are interested in contributing a PR!

[alashti]

@jkirschner-hashicorp It's not really convincing why this issue is tagged as no-support-planned due to the rollback it provides when an unintentional or problematic change on configs is made. Yes, as @abraithwaite has mentioned all will be happy to get the latest version at the config reading time on their application, however, if the problem arises due to the config changes, the best solution is the rollback feature that the config manager has provided. That's because any member of a group, regardless of being aware of the correct config values, can roll back the configs easily to a previous correct version. Moreover, some other alternatives have implemented this feature. So, adding this feature to Consul will complete the package much better. could anyone depict why this feature is not planned to be supported by Consul?

[mljohns89]

Yeah this should definitely, 100%, be reopened and flagged as a critical feature to have.

[nfekunprdtiunnkge]

Any update on this topic?

[jdmarshall]

also git2consul hasn't been maintained in 7 years.