search

logstash-plugins / logstash-input-cloudwatch

A Logstash input to pull events from the Amazon Web Services CloudWatch API
Apache License 2.0
43 stars 28 forks source link

AWS lambda and AWS EC2 metrics are not being shipped to elastic search #54

Open Ritesh007 opened 3 years ago

Ritesh007 commented 3 years ago

I was trying to use this plugin to collect the Lambda metrics and post it to elastic search. Config - cloudwatch { namespace => "AWS/Lambda" type => "cloudwatch_lambda" metrics => [ "Invocations", "Errors", "Duration", "ConcurrentExecutions" ] tags => [ "lambda-metric-logs" ] filters => { "tag:name" => "tag_name" } interval => 900 region => "$APPLICATION_REGION" }

The logs are as below in logstash (surprisingly they show it's successful without any data). - { "level": "INFO", "loggerName": "logstash.inputs.cloudwatch", "timeMillis": 1606209424153, "thread": "[main]<cloudwatch", "logEvent": { "message": "[Aws::CloudWatch::Client 200 0.076219 0 retries] get_metric_statistics(namespace:\"AWS/Lambda\",metric_name:\"Invocations\",start_time:2020-11-24 09:02:04 UTC,end_time:2020-11-24 09:17:04 UTC,period:300,statistics:[\"SampleCount\",\"Average\",\"Minimum\",\"Maximum\",\"Sum\"],dimensions:[{name:\"tag:sku\",value:\"[FILTERED]\"}]) \n" } }

Logstash version - 7.9.0

Use the config above to reproduce the issue.

Mordano commented 3 years ago

I think the problem is not with the plugin but with your filter field. You should use something like: filters => { "FunctionName" => "YOUR_FUNCTION_NAME" }

For filter options, please refer to "2. Choose a dimension" in : AWS Lambda Monitoring Metrics