jegesh / python-sqs-listener

A simple wrapper for boto3 for listening, and sending, to an AWS SQS queue
Other
154 stars 72 forks source link

The address https://queue.amazonaws.com/ is not valid for this endpoint #30

Closed theCompanyDream closed 5 years ago

theCompanyDream commented 5 years ago

When running the code down here I'm having trouble with the SQS pulling events from the selected Queue below. The error we get is "The address https://queue.amazonaws.com/ is not valid for this endpoint." It does create the queue for us if we remove the "someQueue" from the stack. This is also being run in pipenv. Let me know if you need anymore info.

from sqs_listener import SqsListener
from src import image_handler,  ApiEndpoints

class ImageLabelingListener(SqsListener):
    def handle_message(self, body, attributes, messages_attributes, access_token):
        image_handler([body], api)

listener = ImageLabelingListener('someQueue', region_name='us-east-1')
listener.listen()
'<?xml version="1.0"?><ErrorResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><Error><Type>Sender</Type><Code>InvalidAddress</Code><Message>The address https://queue.amazonaws.com/ is not valid for this endpoint.</Message><Detail/></Error><RequestId>7b75f49f-f99b-5f81-91ae-a0684f8d1674</RequestId></ErrorResponse>'
I0304 18:25:10.150804 4700857792 hooks.py:210] Event needs-retry.sqs.GetQueueUrl: calling handler <botocore.retryhandler.RetryHandler object at 0x1333e3470>
I0304 18:25:10.150948 4700857792 retryhandler.py:187] No retry needed.
Traceback (most recent call last):
  File "handler.py", line 14, in <module>
    listener = ImageLabelingListener('ImageTagging', region_name='us-east-1')
  File "/Users/timothybrantleyii/.local/share/virtualenvs/imageLabeling-SmPpLJku/lib/python3.7/site-packages/sqs_listener/__init__.py", line 56, in __init__
    self._client = self._initialize_client()
  File "/Users/timothybrantleyii/.local/share/virtualenvs/imageLabeling-SmPpLJku/lib/python3.7/site-packages/sqs_listener/__init__.py", line 116, in _initialize_client
    QueueOwnerAWSAccountId=os.environ.get('AWS_ACCOUNT_ID', None))
  File "/Users/timothybrantleyii/.local/share/virtualenvs/imageLabeling-SmPpLJku/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/timothybrantleyii/.local/share/virtualenvs/imageLabeling-SmPpLJku/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAddress) when calling the GetQueueUrl operation: The address https://queue.amazonaws.com/ is not valid for this endpoint.
jegesh commented 5 years ago

I just ran into this myself recently, for the first time. It seems to be cause by this issue in the boto library, although I'm not clear on how this is resurfacing years after they changed the url format. Maybe check your botocore version

jegesh commented 5 years ago

I just updated the lib. Now there's a simple workaround: use of the queue_url kwarg will override the queue name and skip the problematic step. I hope this solves your problem (and mine)

theCompanyDream commented 5 years ago

wow that worked thank you so much for the speedy response!!

theCompanyDream commented 5 years ago

The delete message function still uses the old queue url rather than the queue_url kwarg that is defined.

I0305 11:26:53.244242 4695688640 endpoint.py:101] Making request for OperationModel(name=DeleteMessage) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.9.107 Python/3.7.2 Darwin/18.2.0 Botocore/1.12.107'}, 'body': {'Action': 'DeleteMessage', 'Version': '2012-11-05', 'QueueUrl': 'https://sqs.us-east-1.amazonaws.com/618135106065/ImageTagging', 'ReceiptHandle': 'AQEBE4v5x+EG9Qm4j7CAnwYvjPhU+FgC00OkNkZNxfEGN1po56Ac+QOTQEQ3yDU3amGwE1f+K7Qz30ifh21UxZ4X0kAkcsSIOX7t9/WA/oSIuy2jT+kUwYi8Kr/GHlIPbSI0T9NGSa545gKaO1bQCczYAmQMS8eyVLLINVJf5XVreq4qePRYoRJJr53aNK6B10BxUV6QM+wSe/YCRL0JwoocsaWR5QTayFAIZofnMh9l89W3KdsUMjxUJqjJMGKAlGfAQLsyuGPwjvjndENxFX/oLPHNVRssaNexyPgBMszwtejp9YUtDQ+e6SelcC+N3TeZvnPaZs2pcpMfYzpF64ONeb2pTJBLt6iRw9kyce2YaRXLh2nBP8JojfRC5q0u8tR/ZNGb9ReIXxE3jIJ+daCCCg=='}, 'url': 'https://queue.amazonaws.com/', 
jegesh commented 5 years ago

Just because you see the dastardly url again, doesn't mean there's a problem. Are you getting the same error as before? Does the message get deleted from the queue?

theCompanyDream commented 5 years ago

The message get's deleted but I'm going to close out this issue because for some reason it resolved itself.