[telemetry] The metric reader needs the ability to send data to a pipeline

[yutingcaicyt]

Is your feature request related to a problem? Please describe. I find that the metric reader in telemetry can collect metrics directly with otel SDK. But the periodic_reader can only send data to an otlp exporter so I can't process the telemetry metric in the agent. The metric reader needs the ability to send the data to a pipeline.

Describe the solution you'd like The metric reader can collect telemetry metrics and send them to a pipeline.

[mx-psi]

Why not use the OTel SDK OTLP exporter to send the metrics to a Collector OTLP receiver? That way you can send them to a pipeline

[yutingcaicyt]

Yeah, that's a solution, but an additional OTLP receiver is needed. I just want to know whether it is the best practice in this situation.

[mx-psi]

I just want to know whether it is the best practice in this situation.

I think the main point of contention with doing something like this is that a Collector monitoring itself can run into some trouble: if the Collector is misbehaving in some way you can't trust that the pipeline through where telemetry will be going, so I think there is some hesitancy to encourage this pattern. IMO that argument makes sense, but still it's a very common thing people want to do, so maybe we should make it easier in some way.

cc @codeboten who maybe has opinions on this.

[codeboten]

Right, this was discussed in a SIG call some time ago and the decision was that the collector shouldn't rely on itself to process telemetry about itself. As @mx-psi mentioned, the problem is that if the collector's dropping datapoints for example, the last thing you want is to send its own telemetry through a pipeline that's already overflowed as you'd then lose the ability to see what's happening w/ the collector

What type of processing were you hoping to do on the collector's internal telemetry? Is this processing capabilities that exist in the go SDK and that just need to be exposed? Or is it more complex processing like transforming the data?

[yutingcaicyt]

Right, this was discussed in a SIG call some time ago and the decision was that the collector shouldn't rely on itself to process telemetry about itself. As @mx-psi mentioned, the problem is that if the collector's dropping datapoints for example, the last thing you want is to send its own telemetry through a pipeline that's already overflowed as you'd then lose the ability to see what's happening w/ the collector

What type of processing were you hoping to do on the collector's internal telemetry? Is this processing capabilities that exist in the go SDK and that just need to be exposed? Or is it more complex processing like transforming the data?

For now, since my storage only supports delta format of metric, so I need the ability like the 'culmulativeToDeltaProcessor'.