search

raintank / worldping-api

Worldping Backend Service
Other
25 stars 18 forks source link

Delete Endpoint (backend) #4

Closed woodsaj closed 8 years ago

woodsaj commented 8 years ago

Issue by mattttt Friday Jun 19, 2015 at 19:37 GMT Originally opened as https://github.com/raintank/grafana/issues/224

Do the needful to delete the endpoint and all data points in the backend.

woodsaj commented 8 years ago

Comment by mattttt Thursday Jul 23, 2015 at 03:48 GMT

@nopzor1200 this is the backend ticket. What do I need to do here? (front-end ticket is #225)

woodsaj commented 8 years ago

Comment by woodsaj Thursday Jul 23, 2015 at 06:39 GMT

Deleting an endpoint will delete from the DB and emit an event that the endpoint has been deleted.

we need to process that event and do the following:

1) delete from elasticsearch. This should be done after 5minutes to ensure that no lingering metrics from collectors come in as these will cause the elasticsearch data to be re-created. Once the elasticsearch data is deleted, users wont see any trace of the endpoint via graphite. 2) delete the data from cassandra. The data will just get marked for deletion by wont be removed from disk for a while. http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_about_deletes_c.html

woodsaj commented 8 years ago

Comment by Dieterbe Thursday Jul 23, 2015 at 08:13 GMT

1) delete from elasticsearch. This should be done after 5minutes to ensure that no lingering metrics from collectors come in as these will cause the elasticsearch data to be re-created. Once the elasticsearch data is deleted, users wont see any trace of the endpoint via graphite.

is that robust? aren't there scenarios like when the collector can't temporarily send metrics, where it might send metrics more than 5 minutes after the fact? or maybe an other part of the pipeline backs up, like the inbound queue in grafana