SQS Queue instance from resource API returns empty attributes

chophilip21 commented 4 months ago

Describe the bug

The change in the behavior has been introduced starting boto 1.34.90

Here is a demo code:

local_config = {'VisibilityTimeout': 60,
 'MaximumMessageSize': 262144,
 'MessageRetentionPeriod': 1209600,
 'DelaySeconds': 0,
 'ReceiveMessageWaitTimeSeconds': 20}

resource = boto3.resource("sqs", **credentials._asdict())

queue = resource.create_queue(QueueName=name, Attributes=local_config)
print(queue.attributes)

We are passing in Attributes when creating_queue, so that we can call queue.attributes down the line. I have tried similar approach using the client api (SQS.Client.get_queue_attributes) as well, but the attributes are still not visible, regardless of specifying AttributeNames to all.

Expected Behavior

Up until version 1.34.89, we were getting correct response for queue.attributes like the following:

{'ApproximateNumberOfMessagesDelayed': '0', 'VisibilityTimeout': '90', 'ApproximateNumberOfMessagesNotVisible': '0', 'LastModifiedTimestamp': '1714509875', 'QueueArn': 'arn:aws:sqs:elasticmq:000000000000:dummy_queue', 'CreatedTimestamp': '1714509875', 'ApproximateNumberOfMessages': '0', 'ReceiveMessageWaitTimeSeconds': '20', 'DelaySeconds': '0'} Now the response is just empty dict. Expected behavior is to have all the attributes like the previous versions.

Current Behavior

There is no error message, warning, or anything. Suddenly behaviors have changed to return an empty dictionary.

Reproduction Steps

On the latest version of boto3 and botocore (anything above 1.34.89 actually), run:

local_config = {'VisibilityTimeout': 60,
 'MaximumMessageSize': 262144,
 'MessageRetentionPeriod': 1209600,
 'DelaySeconds': 0,
 'ReceiveMessageWaitTimeSeconds': 20}

resource = boto3.resource("sqs", **credentials._asdict())
queue = resource.create_queue(QueueName=name, Attributes=local_config)
queue.attributes

Possible Solution

Something might be happening with the visibility of the attributes. But I do not see a way to control this under resources api.

Additional Information/Context

No response

SDK version used

1.34.94

Environment details (OS name and version, etc.)

Ubuntu 20.04.6 LTS

RyanFitzSimmonsAK commented 4 months ago

Hi @chophilip21, thanks for reaching out. I'm having some trouble reproducing this behavior. When I run your reproduction script on version 1.34.95, the response includes attributes.

local_config = {
    'VisibilityTimeout': '60',
    'MaximumMessageSize': '262144',
    'MessageRetentionPeriod': '1209600',
    'DelaySeconds': '0',
    'ReceiveMessageWaitTimeSeconds': '20'
}

resource = boto3.resource("sqs")

queue = resource.create_queue(QueueName='myTestQueue', Attributes=local_config)
print(queue.attributes)

{'QueueArn': myQueueArn, 'ApproximateNumberOfMessages': '0', 'ApproximateNumberOfMessagesNotVisible': '0', 'ApproximateNumberOfMessagesDelayed': '0', 'CreatedTimestamp': '1714518237', 'LastModifiedTimestamp': '1714518237', 'VisibilityTimeout': '60', 'MaximumMessageSize': '262144', 'MessageRetentionPeriod': '1209600', 'DelaySeconds': '0', 'ReceiveMessageWaitTimeSeconds': '20', 'SqsManagedSseEnabled': 'true'}

When I use GetQueueAttributes, I similarly have no issues.

client = boto3.client('sqs')

response = client.get_queue_attributes(
    AttributeNames = ['All'],
    QueueUrl='https://sqs.us-west-2.amazonaws.com/604627212573/myTestQueue'
)

print(response)

{'Attributes': {'QueueArn': myQueueArn, 'ApproximateNumberOfMessages': '0', 'ApproximateNumberOfMessagesNotVisible': '0', 'ApproximateNumberOfMessagesDelayed': '0', 'CreatedTimestamp': '1714518237', 'LastModifiedTimestamp': '1714518237', 'VisibilityTimeout': '60', 'MaximumMessageSize': '262144', 'MessageRetentionPeriod': '1209600', 'DelaySeconds': '0', 'ReceiveMessageWaitTimeSeconds': '20', 'SqsManagedSseEnabled': 'true'}...

Am I misunderstanding the issue in some way? If not, could you provide debug logs? You can get debug logs by adding boto3.set_stream_logger('') to the top of your script, and redacting any sensitive information. Thanks!

chophilip21 commented 4 months ago

Apologies, I realized that the behavior changes were introduced because of our local testing involving ElasticMQ. When I tested endpoints against our production SQS service, seems like there is no issues nor any changes. Thank you for looking into this!

github-actions[bot] commented 4 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

boto / boto3