Add general aggregations into Metric interface

[yantang-msft]

I'm trying to write a output plugin to Azure Application Insights, which supports aggregated value natively in its datapoint. The problem is for an aggregated metric, I won't know which aggregator emitted it. Thus the conversion logic can be funky. It also doesn't scale if a new aggregator is added. So I'd like to have the Metric interface expose those aggregations (count, min, max, quantiles...) along with the IsAggregate() method.

[danielnelson]

Why is it important to know which aggregator emitted the metric?

[yantang-msft]

@danielnelson The App Insights has its own aggregated telemetry defined. https://github.com/Microsoft/ApplicationInsights-Go/blob/master/appinsights/telemetry.go#L194 I'll have to convert the aggregated metric from telegraf to App Insights' format. Otherwise, the backend of App Insights won't be able to understand the metric correctly.

Currently, to convert the aggregated metric from telegraf to App Insights, I'll need to parse each field based on the format of the aggregator plugins. But as an output plugin, I shouldn't need to know what aggregator is used, given count, min, max... are general aggregations.

[danielnelson]

What if you just treat, for example, any field named "*_min" as a AggregateMetricTelemetry with a Min field without checking if it is IsAggregate()? I can imagine there could be some false positives but I think that would be rare, and it would allow the output to work without knowledge of the source.

[yantang-msft]

@danielnelson Yes, I can do that, but as you said, it's kind of unreliable. Also, it requires all aggregator plugins' field follows the same pattern. And in this case, I'd rather have the aggregator plugins conform to the pattern explicitly by having these field strongly typed.

[danielnelson]

Ultimately someone would need to guess in many cases as well, such as when the data originates upstream from a queuing system via a plugin like kafka_consumer.

We currently have something similar in the Type() function which is used by the prometheus output. It has a few shortcomings, primarily that it applies to the entire Metric. I am working on some changes to the Metric interface for 1.6, and I hope to rework this aspect, so I'll see if I can provide what you need as part of this.

[danielnelson]

Adding this support on a per field basis while not taking a performance hit way is proving to be a bit more work and won't make it in for 1.6.

[srebhan]

@yantang-msft are you still interested in this matter?

[telegraf-tiger[bot]]

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Forums or provide additional details in this issue and reqeust that it be re-opened. Thank you!