Support for Metrics with Custom Namespaces

[mattweyant]

We currently have a number of AWS Lambda functions that are reporting metrics to CloudWatch using their own namespace. I've configured the logstash-cloudwatch-plugin like this:

input {
  cloudwatch 
    namespace => "CustomNamespace"
    metrics => [ "CustomMetric" ]
    filters => {}
    region => "us-east-1"
    type => "cloudwatch"
  }
}
filter {
}
output {
  elasticsearch {
    index => "cloudwatch-%{+YYYY.MM.dd}"
    hosts => ["http://my.ece.elasticsearch.instance.ip.es.io:9200"]
    user => //
    password => //
  }
}

I have confirmed that the plugin correctly collects EC2 metrics. It's unclear what the filter statement should be, but it is required.

And here’s what the plugin’s logs show:

[2018-07-23T14:39:53,469][INFO ][logstash.inputs.cloudwatch] Polling CloudWatch API
[2018-07-23T14:39:53,470][DEBUG][logstash.inputs.cloudwatch] Polling metric CustomMetric
[2018-07-23T14:39:53,470][DEBUG][logstash.inputs.cloudwatch] Filters: []

The EC2 instance that hosts logstash has an IAM role that allows the following:

• Cloudwatch:ListMetrics
• Cloudwatch:GetMetricsStatistics

I'm running Logstash 6.3.2 on an Amazon Linux EC2 instance.

[ramramra]

Hi mattweyant Here i struck with the same Use case .I am able to send the Ec2 namespace metrics to logstash and elasticsearch .Coming to RDS,S3,Elasticache,,Lambda namespace metrics are polling in logstash but we are not getting the metrics in Elasticsearch .Could Please help me to resolve this issue . Thanks in Advance

[keithburrows]

Hi, I also have the same use case. I am working with a lot of custom Metrics with and and without dimensions

[premj1]

Even for me the same issue, it's not pulling any info from RDS, no progress after below log in logstash

[INFO ] 2018-11-16 23:32:55.552 [[main]<cloudwatch] cloudwatch - [Aws::CloudWatch::Client 200 0.160806 0 retries] get_metric_statistics(namespace:"AWS/RDS",metric_name:"CPUUtilization",start_time:2018-11-16 23:17:55 UTC,end_time:2018-11-16 23:32:55 UTC,period:300,statistics:["SampleCount","Average","Minimum","Maximum","Sum"],dimensions:[{name:"EngineName",value:"[FILTERED]"}])

[Ritesh007]

I still face this problem with the latest version of the plugin for Lambda metrics