SDK Exporter for AWS CloudWatch Embedded Metrics Format (EMF)

[phitoduck]

Before opening a feature request against this repo, consider whether the feature should/could be implemented in the other OpenTelemetry client libraries. If so, please open an issue on opentelemetry-specification first.

consider whether the feature should/could be implemented in the other OpenTelemetry client libraries

It looks like the Zipkin and OTLP metric exporters are in the OTel SDK codebase, so I could see a case for this belonging there, but on the other hand, other AWS-related extensions have been added to this codebase.

Is your feature request related to a problem?

I mention this further down, but TL;DR, it'd be nice to be able to ship metrics and logs to CloudWatch via stdout, e.g. via the awslogs exporter built into the docker daemon.

This can save compute cost in AWS Lambda and AWS ECS (for example) by not having to turn on the OpenTelemetry Extension for Lambda or running an additional ADOT instance with each task (or a single shared instance at a discoverable URL).

Describe the solution you'd like

Forgive me if this already exists and I just couldn't find it.

Would it be possible or desirable to implement an additional metrics exporter (not as part of the collector but as part of the OTel SDK) that ships metrics to the console but in the AWS CloudWatch Embedded Metrics format?

What I imagine is something like:

pip install opentelemetry-exporter-emf

OTEL_METRICS_EXPORTER=emf \
OTEL_SERVICE_NAME=otel-playground-app \
opentelemetry-instrument -- python script_that_produces_metrics.py

# some logs would now show up in stdout in the emf format

Which would print metrics in batches to stdout as EMF-formatted JSON.

Describe alternatives you've considered

I realize the OpenTelemetry collector already can export in this format.

I imagined having an EMF exporter built-into the SDK would be nice in environments where you don't want to spend additional money/resources on running a dedicated collector, and relying on systems that collect logs from stdout, e.g. docker's awslogs driver forwarding logs and metrics directly to CloudWatch.

Which alternative solutions or features have you considered?

I couldn't think of any alternatives that avoid running a dedicated collector 😅.

Additional context

Add any other context about the feature request here.

[b0lle]

This would be very handy because of the following:

• one might want to migrate/start using the otel stack without the overhead of the collector. It would be great to use the concept of metrics and logs anyhow and benefit from the standard logging structure, otel provides.
• Be able to use this exporter in case the collector is temporary not available (use as fall-back)
• the awsemfexporter for the collector currently supports only one log-group and one log-stream. This does not scale well. This way we should be able to define log-streams/log-groups for each resource separately.