logstash-plugins / logstash-input-cloudwatch

A Logstash input to pull events from the Amazon Web Services CloudWatch API
Apache License 2.0
43 stars 27 forks source link

Cannot use role_arn #53

Closed aidan-melen closed 4 years ago

aidan-melen commented 4 years ago

arn:aws:iam::??????????:role/logstash IAM role inline policy:

{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "Stmt1444715676000",
                "Effect": "Allow",
                "Action": [
                    "cloudwatch:GetMetricStatistics",
                    "cloudwatch:ListMetrics"
                ],
                "Resource": "*"
            },
            {
                "Sid": "Stmt1444716576170",
                "Effect": "Allow",
                "Action": [
                    "ec2:DescribeInstances"
                ],
                "Resource": "*"
            }
        ]
    }

/usr/share/logstash/config/logstash.yml

http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline

/usr/share/logstash/pipeline/logstash.conf

input {
    cloudwatch {
        namespace => "AWS/EC2"
        metrics => [ "CPUUtilization" ]
        filters => { "tag:Monitoring" => "Yes" }
        role_arn => "arn:aws:iam::??????????:role/logstash"
        region => "us-west-2"
    }
}

output {
    elasticsearch {
        hosts => ["elastic.com:9200"]
        user => "elastic"
        password => "password"
        ssl => false
        ssl_certificate_verification => false
        index => "cloudwatch-metrics-%{+YYYY.MM.dd}"
    }
}

Dockerfile:

FROM docker.elastic.co/logstash/logstash:7.6.0
RUN bin/logstash-plugin install logstash-input-cloudwatch
RUN bin/logstash-plugin install logstash-output-elasticsearch

Run the container with the following:

docker run -it \
-v $(current_dir)/config:/usr/share/logstash/config \
-v $(current_dir)/pipeline:/usr/share/logstash/pipeline \
logstash-with-plugins:latest

Error:

[ERROR] 2020-05-01 16:20:00.805 [[main]<cloudwatch] javapipeline - A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::CloudWatch namespace=>"AWS/EC2", metrics=>["CPUUtilization"], filters=>{"tag:Monitoring"=>"Yes"}, id=>"3dcfbed65cb898284f8766782e4041abdf2b6e1d085b6bdeca03ddd96ca817ef", role_arn=>"arn:aws:iam::???????????:role/logstash-role", region=>"us-west-2", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_58ed5d50-ec8a-4751-b78c-0e27722fa906", enable_metric=>true, charset=>"UTF-8">, role_session_name=>"logstash", statistics=>["SampleCount", "Average", "Minimum", "Maximum", "Sum"], interval=>900, period=>300, combined=>false>
  Error: unable to sign request without credentials set
  Exception: Aws::Errors::MissingCredentialsError