graphite-project / carbonate

Utilities for managing graphite clusters
MIT License
516 stars 80 forks source link

Allow syncing to rename metrics #16

Closed jssjr closed 10 years ago

jssjr commented 10 years ago

Herding metrics often involves renaming metrics. The carbon-sync tool currently supports accepting an arbitrary list of metrics and a source node. It then slurps up a batch of metrics from the source, and uses the local copy to fill or create the metric on the node where carbon-sync is running.

Carbon-sync is intentionally dumb about where metrics should live, as it places all of that responsibility on the operator. The filtering and location gathering for lists of metrics can be handled with the other carbonate tools.

Currently, my strategy for renaming metrics involves making a copy or link of the original metric on the local node as the new name. (Side note: hardlinks work well here.) Then sieving the new newly named metric list and running carbon sync on each node against each node to pick up all the changes. Some additional smarts can be used to reduce overhead, but by and large, the process is gross and overly complicated.

I propose modifying the carbon-sync tool to include two additional parameters; --rename and --rename-separator. When rename is enabled, carbon-sync expects each lines in the supplied input to be a separator delimited pair of metric names. Old and new. It will then sync the old metrics to tmp in its rsync batch run, and fill that data into the new names on the local node. Thus sidestepping all the local file system manipulation.

The final task is to clean up the old metric names, which is a simple task and doesn't yet seem to merit its own tool.

I'll include an example graphite-rename script as an example when it's working.

jssjr commented 10 years ago

Not needed. There are better ways....