Batch Parsing EMF Logs with Large Numeric Array Targets (>100 Members)

[ThePutty]

Description I'm encountering an issue while using the Embedded Metric Format (EMF) with Amazon CloudWatch Logs. I'm sending batches of 10,000+ log records in JSON schema format, each containing metrics with numeric array targets. However, I'm facing a limitation where "Numeric array metric targets MUST NOT have more than 100 members."

This limitation prevents me from ingesting all my metrics in a single batch, as some targets may have more than 100 members.

Steps to Reproduce:

1. Send a batch of 10,000 log records in JSON schema EMF format to CloudWatch Logs.
2. Each record contains a metric with a numeric array target exceeding 100 members.
3. Observe errors or limitations related to exceeding the 100-member limit.

Expected Behavior CloudWatch Logs should successfully ingest and parse the entire batch of logs, regardless of the individual target member count.

Actual Behavior CloudWatch Logs either rejects the entire batch or only ingests a portion of the metrics due to the target member limit.

Additional Information

• Link to relevant documentation: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html

[edergillian-eero]

Is there a configuration to limit the number of metrics on a given log/record? I'm running into a similar issue where metrics are lost because the log sent by the CWAgent contains >100 metrics.

[github-actions[bot]]

This issue was marked stale due to lack of activity.

[github-actions[bot]]

Closing this because it has stalled. Feel free to reopen if this issue is still relevant, or to ping the collaborator who labeled it stalled if you have any questions.