cherweg / logstash-input-s3-sns-sqs

logstash input downloading files from s3 Bucket by OjectKey from SNS/SQS
Other
29 stars 35 forks source link

Possible to reconfigure SqSPoller slightly to accept a queue URL via config instead of sqs_client.get_queue_url() #45

Closed hkelley closed 4 years ago

hkelley commented 4 years ago

We consume messages from a queue provided to us by a vendor, but we are not the owners of the queue and do not have permission to call get_queue_url. This results in the following exception:

`[INFO ][logstash.inputs.s3snssqs ][main] Registering SQS input {:queue=>"ZZZZZZZZZZZZ"}

[ERROR][logstash.inputs.s3snssqs ][main] Cannot establish connection to Amazon SQS {:error=>#} `

Would it be relatively simple to do the following:

1) Accept a queue_url client option queue_url = client_options[:sqs_queue_url]

2) Restructure this block of initialize to skip the get_queue_url and build the poller directly from the config-supplied URL? Apologies if there are Ruby syntax errors below, no experience there.

    begin
      @logger.info("Registering SQS input", :queue => @queue)
      sqs_client = Aws::SQS::Client.new(aws_options_hash)
      if queue_url.nil?
        queue_url = sqs_client.get_queue_url({
          queue_name: @queue,
          queue_owner_aws_account_id: client_options[:queue_owner_aws_account_id]
        }).queue_url # is a method according to docs. Was [:queue_url].
     end  
     @poller = Aws::SQS::QueuePoller.new(queue_url,
        :client => sqs_client
      )