search

gnocchixyz / gnocchi

Timeseries database
Apache License 2.0
298 stars 84 forks source link

Automatically detect deleted resources #1386

Open rafaelweingartner opened 1 month ago

rafaelweingartner commented 1 month ago

This patch is created on top of https://github.com/gnocchixyz/gnocchi/pull/1385, https://github.com/gnocchixyz/gnocchi/pull/1381, https://github.com/gnocchixyz/gnocchi/pull/1387, and https://github.com/gnocchixyz/gnocchi/pull/1388. Therefore, we need to merge them first. Then, the extra commits here will disappear.

While executing some Gnocchi optimizations (https://github.com/gnocchixyz/gnocchi/pull/1307), we noticed that some deleted/removed resources do not have the "ended_at" field with a datetime. This can cause slowness with time, as more and more "zombie" resources are left there, and this has a direct impact in the MySQL queries executed with the aggregates API.

This patch introduces a new parameter called metric_inactive_after, which defines for how long a metric can go without receiving new data points until we consider it as inactive. Then, when all metrics of a resource are in inactive state, we can mark/consider the resource as removed.

rafaelweingartner commented 1 month ago

Thanks @pedro-martins !

rafaelweingartner commented 3 weeks ago

@pedro-martins and @chungg I have rebased this PR and it is ready for your reviews

rafaelweingartner commented 3 weeks ago

Thanks @pedro-martins for your review!

rafaelweingartner commented 2 weeks ago

@chungg thanks for your review! I have added the code changes you suggested, and there are some remarks open, which I would like to hear your response before closing them.