logstash-plugins / logstash-input-kinesis

Logstash Plugin for AWS Kinesis Input
Apache License 2.0
45 stars 57 forks source link

Role ARN config option isn't working #65

Closed autarchprinceps closed 5 years ago

autarchprinceps commented 5 years ago

I added a role_arn as per the documented configuration option, but it is still trying to use the default credentials. I checked and my version of the plugin 2.0.11 should include that ability.

kinesis { id => "input.kinesis.1" region => "region" kinesis_stream_name => "stream" metrics => "cloudwatch" application_name => "table" role_arn => "arn:aws:iam::account:role/rolename" tags => ["input_kinesis", "codec_cloudwatch_logs"] codec => cloudwatch_logs }

But I get a com.amazonaws.services.kinesis.leases.exceptions.DependencyException: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User: default credentials is not authorized to perform: dynamodb:DescribeTable on resource: arn:aws:dynamodb:eu-central-1:account:table/custom table (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException)

Since I do get the non-standard table name that I overrode in the input config, I am confident, that the configuration applies, but it does switch the role. There are no prior failures or interesting messages in the logs and afterwards it repeats the same access denied error for other dynamodb api calls.

robbavey commented 5 years ago

@autarchprinceps Do you have sufficient permissions for a Kinesis consumer - see https://docs.aws.amazon.com/streams/latest/dev/learning-kinesis-module-one-iam.html

autarchprinceps commented 5 years ago

As I wrote, it is not using the role I configured with the rights. It explicitly states that the instance profile role ARN is the one used. "User: insert instance profile credentials here is not authorized" - That's why I gave it an alternative role via the plugin configuration. Yes, that role has sufficient rights, but even if it didn't the message would be "User: insert override credentials from plugin settings here is not authorized". The fact that it tries to use the wrong role is the reason it is not authorized, which the message explicitly states by telling me what role isn't authorized, therefore it is not assuming that role before or at least not using the assumed role credentials for making the dynamodb calls.

robbavey commented 5 years ago

@autarchprinceps This should have been resolved by a new version of the plugin 2.1.0, which was recently released. Feel free to reopen this issue if it does not resolve your issue