search

spotify / heroic

The Heroic Time Series Database
https://spotify.github.io/heroic/
Apache License 2.0
848 stars 109 forks source link

Bigtable retention #445

Open hexedpackets opened 5 years ago

hexedpackets commented 5 years ago

Metrics stored in Bigtable should be able to have a retention policy set via a column family garbage collection policy.

hexedpackets commented 5 years ago

One major edge case to consider is if the node is configured with a different policy than the rest of the cluster. How do we decide when to update the Bigtable cluster? One option is to only set a new policy, never override an existing one. If we go that route we should have a shell task for manually adjusting the policy later.

dmichel1 commented 5 years ago

Good point - we could have the default policy be forever (current behavior) and have the GC policy always be set through a shell task. It would be good to have some best practices on setting up a new cluster. e.g. these one-off shell commands should be in SCM and run though a CI process that way this is always repeatable and we have a source of truth for new clusters

stale[bot] commented 5 years ago

⚠ This issue has been automatically marked as stale because it has not been updated for at least two months. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. This issue will automatically be closed in ten days if no further activity occurs. Thank you for all your contributions.