Recently we started receiving agent metrics as well and otel go http and instrumentation send up duration and content_length metrics. However, they send up a metric object for each request which leads to a flood of metrics and the collector and prometheus server memory fills up quickly. We have added a filter processor to filter out these metrics.
We should strive to replace our existing metrics remover processor with the filter processor. I'm not sure if the None metric type or the logic we use to remove the Empty type metrics translates to a filter.
Description
Recently we started receiving agent metrics as well and otel go http and instrumentation send up duration and content_length metrics. However, they send up a metric object for each request which leads to a flood of metrics and the collector and prometheus server memory fills up quickly. We have added a filter processor to filter out these metrics.
Testing
Verified that the filter works locally
Checklist: