aws-samples / amazon-cloudwatch-auto-alarms

Automatically create and configure Amazon CloudWatch alarms for EC2 instances, RDS, and AWS Lambda using tags for standard and custom CloudWatch Metrics.
MIT No Attribution
91 stars 85 forks source link

How to add dimensions with - in the value? #7

Closed watsonjm closed 3 years ago

watsonjm commented 3 years ago

I'm trying to work with the following dimensions pulled via AWS CLI on a working alarm that I created manually for this purpose:

user@PC:~# aws cloudwatch describe-alarms --alarm-names "tester"
...
"Dimensions": [
                {
                    "Name": "InstanceId",
                    "Value": "i-111111111111111"
                },
                {
                    "Name": "NodeName",
                    "Value": "ip-123-45-123-45.us-east-1.compute.internal"
                },
                {
                    "Name": "ClusterName",
                    "Value": "some-cluster-name"
                }
            ],
...

I've been working with this in the cw_auto_alarms.py file to rule out any issues with tagging (before I got to dashes likely being the issue) and have it hard coded in there currently like this:

       {
            'Key': 'AutoAlarm-ContainerInsights-node_filesystem_utilization-NodeName-ip-123-45-123-45.us-east-1.compute.internal-ClusterName-some-cluster-name-GreaterThanThreshold-1m-Average',
            'Value': alarm_cpu_high_default_threshold
        }

I have also tried wrapping the values with " and also \" to escape the quotes, but no luck.

The alarm isn't creating, and I have been able to create other alarms without extra dimensions, so I'm assuming this has something to do with the dashes in the values. This is for custom metrics using ContainerInsights if that helps. Any help is appreciated.

knizami commented 3 years ago

Hello, thanks for submitting this. You are correct, the separator is hardcoded. I am working on fixing this to allow you to specify the separator. It should be available by Monday. Also, working on supporting DynamoDB as a persistent store so you don't have to edit the code. DynamoDB input of alarms will be available in a week or two.

knizami commented 3 years ago

Hello, I've added a feature to enable you to specify the separator character. In /src/cw_auto_alarms.py you can update the property, alarm_separator to a different value such as "~" to enable you to use the "-" character in your alarm properties.

knizami commented 3 years ago

Hello, closing this issue. Please let me know if you have any questions on implementing a custom alarm separator. The documentation has also been updated to include this feature.

Erlichooo commented 2 years ago

@watsonjm Hello, I have seen that you use the auto-alarm lambda for Container Insight. I want know is that work. I did some job with EKS, and also use Container Insight, but I got some issue for this that alarm could not been created. If any reply I will really appreciate it.

watsonjm commented 2 years ago

@watsonjm Hello, I have seen that you use the auto-alarm lambda for Container Insight. I want know is that work. I did some job with EKS, and also use Container Insight, but I got some issue for this that alarm could not been created. If any reply I will really appreciate it.

Hi @Erlichooo Yes, I was able to create some custom alarms for EKS Container Insights. It's been a while but I can probably still help, what alarm are you trying to create, and what issues are you having?

Erlichooo commented 2 years ago

@watsonjm Hello, I have seen that you use the auto-alarm lambda for Container Insight. I want know is that work. I did some job with EKS, and also use Container Insight, but I got some issue for this that alarm could not been created. If any reply I will really appreciate it.

Hi @Erlichooo Yes, I was able to create some custom alarms for EKS Container Insights. It's been a while but I can probably still help, what alarm are you trying to create, and what issues are you having?

Tks so much that your reply @watsonjm I have met the problem that when I set Nodename dimension for the Container Insights auto add/reduce alarm when the EKS add/reduce some nodes. The problem is the alarm of auto part could not work. And I show my node tag just like AutoAlarm-ContainerInsights-node_memory_utilization-ClusterName-tmp-GreaterThanThreshold-1m-Average

I saw your tag with specific dimension 'Nodename', but I thought it is not suit for EKS to auto add/reduce alarm when the node up or down, because the 'Nodename' likes ec2 dimension 'InstanceId'.

Again, I really thank for your reply, and if you can have any help for me, may be I can share my screen to show my environment with skype\zoom\teams any convenient for you.

watsonjm commented 2 years ago

Hi @Erlichooo , you mentioned adding/reducing nodes based on memory, and it sounds like you might be trying to do autoscaling, is that what you're working with?