structurely / ecs-autoscale

A framework that runs on AWS Lambda for autoscaling ECS clusters and services
MIT License
71 stars 7 forks source link

handle empty Cloudwatch metrics #34

Open Mortinke opened 5 years ago

Mortinke commented 5 years ago

Cloudwatch doesn't return values for some resource metrics if the resource was not used e.g. TargetResponseTime for AWS/ApplicationELB.

Elastic Load Balancing reports metrics to CloudWatch only when requests are flowing through the load balancer. If there are requests flowing through the load balancer, Elastic Load Balancing measures and sends its metrics in 60-second intervals. If there are no requests flowing through the load balancer or no data for a metric, the metric is not reported.

As a result, CloudWatchErrors are always thrown here if the resource is not used

[ERROR] 2019-09-29T18:50:04.383Z ff7065bb-29de-4073-a4f1-0a48bb5d6ea1 Error retreiving CloudWatch statistics, no datapoints found:
=> Namespace: AWS/ApplicationELB
=> MetricName: TargetResponseTime
=> Dimensions: [{'Name': 'TargetGroup', 'Value': 'targetgroup/ac-re-Targe-13P06UAEX2W9Y/f9002d01ab9053e0'}]
=> Period: 60
=> Statistics: ['Average']
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 94, in lambda_handler
services = gather_services(cluster_name, cluster_def)
File "/var/task/ecsautoscale/services.py", line 248, in gather_services
max_tasks=service["max"]
File "/var/task/ecsautoscale/services.py", line 89, in __init__
res = source.get_data(**item)
File "/var/task/ecsautoscale/metric_sources/cloudwatch.py", line 76, in get_data
namespace, metric_name, dimensions_, period, statistics_)
ecsautoscale.exceptions.CloudWatchError: Error retreiving CloudWatch statistics, no datapoints found:
=> Namespace: AWS/ApplicationELB
=> MetricName: TargetResponseTime
=> Dimensions: [{'Name': 'TargetGroup', 'Value': 'targetgroup/ac-re-Targe-13P06UAEX2W9Y/f9002d01ab9053e0'}]
=> Period: 60
=> Statistics: ['Average']
Mortinke commented 5 years ago

a simple solution for this issue: #35