Open kevindixon opened 10 months ago
From what I can see of the Kombu SQS implementation calls to boto3 receive_message and receive_messages don't request any attributes (no AttributeNames parameter). At least in my experiments, receive_message and receive_messages don't return Attributes unless specifically requested to do so.
Looked again, and this assertion isn't true - I've misread the code. asynchronous.aws.sqs.connection
does indeed set attribute names.
Not entirely sure why I'm getting this error (only when setting backoff_policy
)..
can you help to reproduce with celery 5.3.6?
Same behaviour with Celery 5.3.6.
However, I think this is my misunderstanding of the documentation - backoff_policy
does not work when raising Reject
with requeue=True
. This stands to reason, I guess, because requeue=True
presumably creates a new SQS message.
So, my solution at least is to NOT use requeue=True
and backoff_policy
is honoured.
The Celery SQS documentation is pretty unclear on all this (as is the documentation of the requeue
parameter), and the KeyError
isn't particularly helpful.
So, maybe a little doc improvement.
More appropriate behaviour, I would think, would be to ignore backoff_policy
when Reject
is thrown with requeue=True
and log the fact the 'policy has been ignored?
Versions:
Using
acks_late
Setting
backoff_policy
fails with aKeyError
when a message fails:From what I can see of the Kombu SQS implementation calls to boto3
receive_message
andreceive_messages
don't request any attributes (noAttributeNames
parameter). At least in my experiments,receive_message
andreceive_messages
don't returnAttributes
unless specifically requested to do so.Not sure if this is a change in Kombu or boto