search

graphite-project / carbonate

Utilities for managing graphite clusters
MIT License
516 stars 80 forks source link

Allow carbon-sync to delete remote data after successful sync #32

Closed kamaradclimber closed 4 years ago

kamaradclimber commented 10 years ago

After a successful carbon-sync , we eventually want to remove the original metric.

We cannot use last modification time (such as find /opt/graphite/storage/whisper -type f -mtime +2) since it will remove metric not updated (because there was no new point). Any combination with carbon-sieve won't help us distinguish them either because carbon-sync ignore (logs actually) some errors.

It would be nice for carbon-sync to output metric that were successfully updated on its output to be able launch a delete.

We are currently doing this with a checksum of `whisper-fetch.py--until xxx` output (fetching all points up to now-1hour), but this is far from perfect.

jssjr commented 10 years ago

I will be merging in https://github.com/jssjr/carbonate/pull/30 as soon as a few adjustments are made. Does this work for you, or is the problem of infrequently updated whisper data files still an issue?

kamaradclimber commented 10 years ago

30 is a good start but not sufficient.

I'd like carbon-sync to output the list of metric successuly merged (or those with an error). It would be the definitive answer

jssjr commented 10 years ago

Is the current output useful to you?

It might be more unix-like to simply output successfully updated metric names to stdout and failed metric names to stderr. I need to do some more thinking about this problem, since it relates somewhat to the work I was attempting in the https://github.com/jssjr/carbonate/tree/support-rename branch.

kamaradclimber commented 10 years ago

Current output is nice (progress + stats in the end). What I would suggest would be to ouput progress (and stats and all logs such as failed metric) on stderr and updated metrics on stdout.

On Fri, Oct 10, 2014 at 9:08 PM, Scott Sanders notifications@github.com wrote:

Is the current output useful to you?

It might be more unix-like to simply output successfully updated metric names to stdout and failed metric names to stderr. I need to do some more thinking about this problem, since it relates somewhat to the work I was attempting in the https://github.com/jssjr/carbonate/tree/support-rename branch.

— Reply to this email directly or view it on GitHub https://github.com/jssjr/carbonate/issues/32#issuecomment-58702325.

Grégoire

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.