Open JayakumarRamesh opened 4 years ago
s3_role_arn is removed with 2.1 i think... Please use the example in the docs: https://github.com/cherweg/logstash-input-s3-sns-sqs/blob/master/docs/index.asciidoc
s3_options_by_bucket => [
{ bucket_name => "logs-bucket-222222222222-.*"
credentials => { role => "arn:aws:iam::222222222222:role/logging-role" }
},
The docs suggest that you only need to specify the credentials if you are overwriting "the default" ?
If you have credentials per s3 bucket you could overwrite the default.
But it doesn't look like you can put credentials
in the top-level settings?
[2021-09-22T18:58:00,742][ERROR][logstash.inputs.s3snssqs ] Unknown setting 'credentials' for s3snssqs
So you need to specify them separately for each bucket, even if they all use the same setting?
input { s3snssqs { region => "us-east-1" queue => "" queue_owner_aws_account_id => "" access_key_id => "*****" secret_access_key => "**" role_arn => "arn:aws:iam::accountid:role/rolename" role_session_name => "logstash"
sqs_skip_delete => true
}
}
output { stdout { } }
I am using role_arn to fetch data from another aws account but I am getting error though my credentials and assumed role are correct.
Error:
[2020-09-10T21:45:50,635][ERROR][logstash.inputs.s3snssqs ][main] Cannot establish connection to Amazon SQS {:error=># you do not have access to it.>} [2020-09-10T21:45:50,644][DEBUG][logstash.javapipeline ][main] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#"} [2020-09-10T21:45:50,649][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"} [2020-09-10T21:45:50,651][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"} [2020-09-10T21:45:50,731][DEBUG][logstash.javapipeline ][main] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#"} [2020-09-10T21:45:50,733][DEBUG][logstash.javapipeline ][main] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#"} [2020-09-10T21:45:50,734][DEBUG][logstash.javapipeline ][main] Shutdown waiting for worker thread {:pipeline_id=>"main", :thread=>"#"} [2020-09-10T21:45:50,741][DEBUG][logstash.outputs.stdout ][main] Closing {:plugin=>"LogStash::Outputs::Stdout"} [2020-09-10T21:45:50,748][DEBUG][logstash.pluginmetadata ][main] Removing metadata for plugin 6e3e4da5af4891d1493e09316ac7bbe9afccc27748b9230d02cc89b3eebdce81 [2020-09-10T21:45:50,752][ERROR][logstash.javapipeline ][main] Pipeline aborted due to error {:pipeline_id=>"main", :exception=># your credentials>, :backtrace=>["D:/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-s3-sns-sqs-2.1.1/lib/logstash/inputs/sqs/poller.rb:58:in
initi alize'", "D:/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-s3-sns-sqs-2.1.1/lib/logstash/inputs/s3snssqs.rb:251:in
register'", "D:/logstash/sqslo g/logstash-7.8.0/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:216:inblock in register_plugins'", "org/jruby/RubyArray.java:1809:in
each'", "D:/logstash/sqslog/logstash-7.8.0/l ogstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:215:inregister_plugins'", "D:/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:326:in
star t_inputs'", "D:/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:286:instart_workers'", "D:/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/logstash-co re/lib/logstash/java_pipeline.rb:170:in
run'", "D:/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:125:in `block in start'"], "pipeline.sources"=>["D :/logstash/sqslog/logstash-7.8.0/logstash-7.8.0/pipelines/s3snssqs.conf"], :thread=>"#"} [2020-09-10T21:45:50,777][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: Pip elineAction::Create, action_result: false", :backtrace=>nil}Please help me to resolve this issue.