search

logstash-plugins / logstash-input-cloudwatch

A Logstash input to pull events from the Amazon Web Services CloudWatch API
Apache License 2.0
43 stars 28 forks source link

Not able to get metrics in a file and not even elastic search. the cloudwatch_logs one works but this doesnt. #17

Open badaldavda opened 7 years ago

badaldavda commented 7 years ago

input { cloudwatch { namespace => "AWS/EC2" metrics => [ "CPUUtilization" ] filters => { "tag:Name" => "CaseWork" } region => "us-east-1" statistics => "Maximum" period => 300 access_key_id => "xx" secret_access_key => "xx" } } output { elasticsearch { hosts => ["ip-172-31-55-42:9200"] index => "metrics2-%{+YYYY.MM.dd}" } } This is my config and there is not output. Help! I need this as soon as possible.

For all general issues, please provide the following details for fast resolution:

badaldavda commented 7 years ago

I need this plugin to be running.. And there is no output to this in elasticsearch.. I would have added in the ElasticSeach forum but I need to get this working as soon as possible.

alejandrobarrios07 commented 7 years ago

I have the same problem with RDS.

input { cloudwatch { namespace => "AWS/RDS" access_key_id => "xxxxxxxxxxxxxxxxxx" secret_access_key => "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" metrics => [ "FreeStorageSpace", "FreeStorageSpace", "CPUUtilization" ] filters => { "EngineName" => "mysql" } region => "sa-east-1" tags => ["AWS_RDS"] period => 60 } }

output { if "AWS_RDS" in [tags] { elasticsearch{ index => "cloudwatch-%{+YYYY.MM.dd}" hosts => "localhost:9200" } } }

AnushaY1 commented 6 years ago

i am also facing same problem, logstash does n't show any errors & nothing in that index...

input { cloudwatch { access_key_id => "xxxxx" secret_access_key => "xxxxxx" namespace => "AWS/S3" metrics => ["BucketSizeBytes","NumberOfObjects"] filters => { "BucketName" => "awsbillingxxxx" } region => "us-west-2" period => 60 } }

output { stdout { codec => rubydebug } elasticsearch { hosts => ["http://localhost:9200/"] index => "dcloud_watch_aws" } } ~

arocha22 commented 6 years ago

Hello all, Could you help me please? Anyone have a resolution for this issue? Logstash doesn't show any errors and nothing in index (elasticsearch).

[2018-06-25T17:06:48,687][INFO ][logstash.inputs.cloudwatch] [AWS CloudWatch 200 0.156352 0 retries] get_metric_statistics(:dimensions=>[{:name=>"EngineName",:value=>"mysql"}],:end_time=>"2018-06-25T17:06:48+00:00",:metric_name=>"CPUUtilization",:namespace=>"AWS/RDS",:period=>300,:start_time=>"2018-06-25T17:01:48+00:00",:statistics=>["Average"])

[2018-06-25T17:11:45,373][INFO ][logstash.inputs.cloudwatch] Polling CloudWatch API [2018-06-25T17:11:45,778][INFO ][logstash.inputs.cloudwatch] [AWS CloudWatch 200 0.396151 0 retries] get_metric_statistics(:dimensions=>[{:name=>"EngineName",:value=>"mysql"}],:end_time=>"2018-06-25T17:11:45+00:00",:metric_name=>"CPUUtilization",:namespace=>"AWS/RDS",:period=>300,:start_time=>"2018-06-25T17:06:45+00:00",:statistics=>["Average"])

[2018-06-25T17:16:45,370][INFO ][logstash.inputs.cloudwatch] Polling CloudWatch API [2018-06-25T17:16:45,680][INFO ][logstash.inputs.cloudwatch] [AWS CloudWatch 200 0.308614 0 retries] get_metric_statistics(:dimensions=>[{:name=>"EngineName",:value=>"mysql"}],:end_time=>"2018-06-25T17:16:45+00:00",:metric_name=>"CPUUtilization",:namespace=>"AWS/RDS",:period=>300,:start_time=>"2018-06-25T17:11:45+00:00",:statistics=>["Average"])

badaldavda commented 6 years ago

I would suggest that you check the ELB version. I was using older version of ELK due to which the plugin did not work. Change it to the latest and try again.

SoujanyaRama commented 6 years ago

Has anyone been able to find a fix to this issue? I am facing the same. Logstash logs show the polling happens and but nothing in Elastic Index. Only namespace metric data that makes it to Elasticsearch is EC2