aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.41k stars 3.8k forks source link

(aws-dynamodb): Cannot alarm on metricThrottledRequestsForOperations due to Alarm limits #24509

Open straygar opened 1 year ago

straygar commented 1 year ago

Describe the bug

"metricThrottledRequestsForOperations" uses a math experession behind the hood to get the metric. By default, operations is set to cover everything (which is good). However, since there are more than 10 operations, trying to create a CloudWatch Alarm with all operations results in the following error on sync:

Error: Alarms on math expressions cannot contain more than 10 individual metrics

Would it be possible to change how the metrics are fetched to avoid this?

Expected Behavior

I can create an alarm on table.metricThrottledRequestsForOperations()

Current Behavior

Trying to create a CloudWatch Alarm with all operations results in the following error on sync:

Error: Alarms on math expressions cannot contain more than 10 individual metrics

Reproduction Steps

const app = new App();
const stack = new Stack(app);
const table = new Table(stack, 'Table', {
    partitionKey: {
        name: 'pk',
        type: AttributeType.STRING
    },
});
const alarm = new Alarm(stack, 'Alarm', {
  metric: table.metricThrottledRequestsForOperations(),
  evaluationPeriods: 1,
  threshold: 1,
});

Possible Solution

Use a SEARCH() math expression or SQL style query and aggregate with SUM in sumMetricForOperation, to find all throttled metrics for the table.

Example:

new MathExpression({
  expression: `
    SELECT SUM(ThrottledRequests)
    FROM "AWS/DynamoDB"
    WHERE TableName = '${table.name}'
  `
});

Additional Information/Context

No response

CDK CLI Version

2.60.0

Framework Version

No response

Node.js Version

18

OS

MacOS

Language

Typescript

Language Version

No response

Other information

No response

pahud commented 1 year ago

This limit is described here:

(Optional) If you want to add another metric to a metric math expression, you can use the search box to find a specific metric. You can add as many as 10 metrics to a metric math expression.

I'll try reproduce this in my account.

jbm00n commented 1 year ago

Same issue with metricSystemErrorsForOperations. We basically just want to trigger an alarm based on the SystemErrors metric from the AWS/DynamoDB namespace. Is there another way to do this? (other than manipulating the MathExpression)

const alarm = new cloudwatch.Alarm(this, "MyAlarm", {
      evaluationPeriods: 1,      
      metric: table.metricSystemErrorsForOperations({        
        period: cdk.Duration.seconds(60),
        unit: cloudwatch.Unit.COUNT,
        statistic: cloudwatch.Stats.SUM,        
      }),
      threshold: 0,
      comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
    });
juarezzz commented 10 months ago

You can pass an option with the operations you want your alarm to listen to. Like this:

import { Operation } from 'aws-cdk-lib/aws-dynamodb';
import { Alarm } from 'aws-cdk-lib/aws-cloudwatch';

const alarm = new Alarm(stack, 'Alarm', {
  metric: table.metricThrottledRequestsForOperations({
          operations: [
            Operation.PUT_ITEM,
            Operation.BATCH_WRITE_ITEM,
            Operation.QUERY,
          ],
   }),
  evaluationPeriods: 1,
  threshold: 1,
});
pwteneyck commented 10 months ago

You can pass an option with the operations you want your alarm to listen to

You can, but the point here is that the default value that CDK uses for table.metricThrottledRequestsForOperations results in a build error

straygar commented 4 months ago

I think this is solved by: https://github.com/aws/aws-cdk/pull/29341

Should we close this issue?

Edit: actually nevermind, I think that limit bump is for metrics per alarm, and not math expressions.