ex_aws_sqs no longer constructs request correctly

[chris-brace]

The following worker (c&p from the docs) fails to connect to the queue.

defmodule MyBroadway.Worker do
  use Broadway

  def start_link(_) do
    Broadway.start_link(__MODULE__,
      name: __MODULE__,
      producers: [
        default: [
          module:
            {BroadwaySQS.Producer,
             queue_name: "second",
             config: [
               access_key_id: "x",
               secret_access_key: "y",
               region: "us-east-2"
             ]}
        ]
      ],
      processors: [
        default: []
      ],
      batchers: [
        default: [
          batch_size: 10,
          batch_timeout: 2000
        ]
      ]
    )
  end

  def handle_message(_, message, _) do
    receipt = %{
      id: message.metadata.message_id,
      receipt_handle: message.metadata.receipt_handle
    }

    IO.inspect(receipt)
  end
end

It gets the following errors:

18:33:39.373 [debug] ExAws: Request URL: "https://sqs.us-east-2.amazonaws.com/" HEADERS: [{"Authorization", "AWS4-HMAC-SHA256 Credential=x/20190924/us-east-2/sqs/aws4
_request,SignedHeaders=content-encoding;content-type;host;x-amz-date,Signature=z"}, {"host", "sqs.us-east-2.amazonaws.com"
}, {"x-amz-date", "20190924T223339Z"}, {"content-type", "application/x-www-form-urlencoded"}, {"content-encoding", "identity"}] BODY: "Action=ReceiveMessage&MaxNumberOfMessages=10&Queue
Url=second" ATTEMPT: 1

18:33:39.420 [error] GenServer MyBroadway.Worker.Broadway.Producer_default_1 terminating
** (UndefinedFunctionError) function Poison.decode/1 is undefined (module Poison is not available)
    Poison.decode("<?xml version=\"1.0\"?><ErrorResponse xmlns=\"http://queue.amazonaws.com/doc/2012-11-05/\"><Error><Type>Sender</Type><Code>InvalidAddress</Code><Message>The address s
econd is not valid for this endpoint.</Message><Detail/></Error><RequestId>4939ea47-c4a1-5715-934f-bc4358a24cee</RequestId></ErrorResponse>")
    (ex_aws) lib/ex_aws/request.ex:108: ExAws.Request.client_error/2
    (ex_aws) lib/ex_aws/request.ex:57: ExAws.Request.request_and_retry/7
    (ex_aws) lib/ex_aws/operation/query.ex:41: ExAws.Operation.ExAws.Operation.Query.perform/2
    (broadway_sqs) lib/broadway_sqs/ex_aws_client.ex:50: BroadwaySQS.ExAwsClient.receive_messages/2
    (broadway_sqs) lib/broadway_sqs/producer.ex:196: BroadwaySQS.Producer.handle_receive_messages/1
    (broadway) lib/broadway/producer.ex:66: Broadway.Producer.handle_demand/2
    (gen_stage) lib/gen_stage.ex:2099: GenStage.noreply_callback/3
Last message: {:"$gen_producer", {#PID<0.316.0>, #Reference<0.1588154580.3783000066.245470>}, {:ask, 10}}
State: %{consumers: [{#PID<0.316.0>, #Reference<0.1588154580.3783000066.245470>}], module: BroadwaySQS.Producer, module_state: %{demand: 0, receive_interval: 5000, receive_timer: nil, s
qs_client: {BroadwaySQS.ExAwsClient, %{ack_ref: #Reference<0.1588154580.3783000065.247160>, config: [access_key_id: "x", secret_access_key: "y", region: "us-east-2"], queue_name: "second", receive_messages_opts: [max_number_of_messages: 10]}}}, transformer: nil}

These are my deps

  defp deps do
    [
      {:broadway_sqs, "~> 0.3.0"},
      {:hackney, github: "benoitc/hackney", override: true}
    ]
  end

When i downgrade to broadway_sqs 0.2.0 it forms the request url successfully:

18:41:13.192 [debug] ExAws: Request URL: "https://sqs.us-east-2.amazonaws.com/second" HEADERS: [{"Authorization", "AWS4-HMAC-SHA256 Credential=x/20190924/us-east-2/sqs/aws4_request,SignedHeaders=content-encoding;content-type;host;x-amz-date,Signature=z"}, {"host", "sqs.us-east-2.amazonaws.com"}, {"x-amz-date", "20190924T224113Z"}, {"content-type", "application/x-www-form-urlencoded"}, {"content-encoding", "identity"}] BODY: "Action=ReceiveMessage&MaxNumberOfMessages=10" ATTEMPT: 1

You can see that second gets appended to the base sqs url.

[chris-brace]

I believe this might be related to #27 which appears to have changed the tests to check for the url im seeing that's broken.

[chris-brace]

So it appears to work when i pass https://us-east-2.queue.amazonaws.com/838607676327/second as the queue_name. Im not 100% sure this is intended, but at very least it should be mentioned in the docs.

[msaraiva]

Hi @chris-brace! Thanks for the report.

So, as far as I could see, since ex_aws_sqs v3.0.0 they started using the queue URL instead of the queue name because, according to https://github.com/ex-aws/ex_aws_sqs/pull/9#issue-298229693, the SQS docs states that:

In your system, always store the entire queue URL exactly as Amazon SQS returns it to you when you create the queue Don't build the queue URL from its separate components each time you need to specify the queue URL in a request because Amazon SQS can change the components that make up the queue URL.

Sounds fair enough to me. However, in this case, I believe it would be better to rename queue_name to queue_url, update the docs/guides and release a new version. WDYT?

Feel free to send a PR if you want ;) Otherwise, I will address this in the next couple of days.

Thanks again!

[chris-brace]

Btw that's my other github account. Github enforcing global uniqueness for SSH keys made it easier to just do this on my personal one.

[msaraiva]

Closed by #32 and #33