How to send metrics to OTEL collector

[sap-ray]

Component(s)

exporter/dynatrace

Describe the issue you're reporting

We intend to send metrics to Dynatrace using otel-collector-contrib. Unable to find good example/use-case.

[github-actions[bot]]

Pinging code owners:

• exporter/dynatrace: @dyladan @arminru @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dyladan]

We recommend you use the OTLP metrics ingest endpoint. You can find an example of the configuration for that here: https://www.dynatrace.com/support/help/shortlink/opentelemetry-collector-explanation#configuration.

Please note the supported metric types and aggregation temporalities here: https://www.dynatrace.com/support/help/extend-dynatrace/opentelemetry/opentelemetry-metrics/opentelemetry-metric-types

You can find more information about using the OpenTelemetry collector with Dynatrace at https://www.dynatrace.com/support/help/extend-dynatrace/opentelemetry/basics/collector

[sap-ray]

Thanks for sharing the links, we have reviewed them in past, currently we are in deployment phase. How we can send metrics to the collector from things we can't put an endpoint on? We would like to report response times from portions of code within a synthetic canary. We also need to send metrics to the collector from other things that can't support an endpoint like SaaS vendors and other custom metrics from application programs. We are looking for a use case to cater above requirements.

[sap-ray]

Request to update on above query, do let me know if the tag needs to be changed from "exporter/dynatrace"

[dyladan]

To me it doesn't sound like this is an issue with the dynatrace exporter, but sounds like you're trying to get metrics into the collector so that they can be exported to Dynatrace. Once you have metrics in the collector, you would either use OTLP or the Dynatrace exporter for that.

How to collect metrics into the collector varies by application. Without more specifics I can't really help with your specific application. If you are trying to collect metrics from an application you have source code access to, you will need to look into the OpenTelemetry metrics API/SDK for the appropriate language. If you are trying to export metrics from a SaaS product, you should look and see if there is an appropriate collector receiver or if your SaaS product can export metrics in a well known format like OTLP, OpenMetrics, etc.

[fpunzohig]

Hi @dyladan can you point us to some documentation or instructions that tell us how to export metrics like you describe in the bolded section below?

If you are trying to export metrics from a SaaS product, you should look and see if there is an appropriate collector receiver or if your SaaS product can export metrics in a well known format like OTLP, OpenMetrics, etc. >>>

[dyladan]

Without knowing where you're trying to export metrics from I really can't. There's no documentation that would apply universally.

[fpunzohig]

Examples:

Method Timing from Python program -> Otel Contrib collector -> Dynatrace Method Timing from C++ program -> Otel Contrib collector -> Dynatrace and Splunk and Elastic AJAX interaction in NodeJS Synthetic -> Otel Contrib collector -> Dynatrace

[fpunzohig]

Forgot one other important one

Plutora (SaaS) transaction timing -> Otel Contrib collector -> Dynatrace

[fpunzohig]

Main goal for us is to use OTEL as an abstraction layer so we don't have to support multiple point to point solutions for our metrics. We just need to figure out how to format and send our metrics to the OTEL collector.

[dyladan]

Examples:

Method Timing from Python program -> Otel Contrib collector -> Dynatrace Method Timing from C++ program -> Otel Contrib collector -> Dynatrace and Splunk and Elastic AJAX interaction in NodeJS Synthetic -> Otel Contrib collector -> Dynatrace

For these examples, I would suggest looking into the python, C++, and JavaScript opentelemetry SDK documentations. Likely all three of these particular cases will require at least some code-level modifications in order to track the methods you are interested in.

Plutora (SaaS) transaction timing -> Otel Contrib collector -> Dynatrace

I'm not familiar with Plutora so I can't say for sure. You should reach out to their support to find out which formats they support exporting metrics in. Once you know the ways they can export metrics, then you can see if there is an appropriate collector receiver for that format.

[fpunzohig]

Sorry I thought that OTEL provided a standardized api for reporting metrics to the collector. What we are trying to achieve is a standardized way for our company's application areas to report their metrics to a centralized location. Does it make more sense for us to create an API in Python that sits on the same instances that the OTEL collector sits on and use the python OTEL SDK to publish to the local collector? And then have the collector export the metrics to Dynatrace, Splunk, Elastic etc?

[dyladan]

Sorry I thought that OTEL provided a standardized api for reporting metrics to the collector.

This is true. OTEL provides a standardized cross-programming-language API to describe telemetry, and also a data format to transfer it in (OTLP).

What we are trying to achieve is a standardized way for our company's application areas to report their metrics to a centralized location. Does it make more sense for us to create an API in Python that sits on the same instances that the OTEL collector sits on and use the python OTEL SDK to publish to the local collector? And then have the collector export the metrics to Dynatrace, Splunk, Elastic etc?

It sounds like you mean some sort of remote API here like an HTTP API. In that case I would suggest you don't bother creating your own service and transmit data using OTLP, as that will be the format your python service would export to the collector anyway.

[evan-bradley]

It sounds like the original question has been answered, so I'm closing this issue. Please re-open it if this isn't the case.