tim-finnigan
commented
3 months ago Thanks for the feature request. Here is the Boto3 documentation on retries, which has more information on retry configurations and exponential backoff: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html
Have you tried using different retry configurations or increasing max_attempts? Or can you expand a bit more on what you're trying to do and how the current behavior doesn't meet your use case?
github-actions[bot]
Describe the feature
In the documentation of batch_write_item there is a strong recommendation of using exponential backoff algorithm to retry the next writing. However here it is immediately retried. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/client/batch_write_item.html
_If DynamoDB returns any unprocessed items, you should retry the batch operation on those items. However, we strongly recommend that you use an exponential backoff algorithm. If you retry the batch operation immediately, the underlying read or write requests can still fail due to throttling on the individual tables. If you delay the batch operation using exponential backoff, the individual requests in the batch are much more likely to succeed.
For more information, see Batch Operations and Error Handling in the Amazon DynamoDB Developer Guide._
Also, it could contain a initial waiting time and max_retries parameters to control the maximum waiting time. It would be useful to call it from a Lambda function with a timeout.
Use Case
You can use the context manager batch_writer with the possibility of selecting an exponential backoff algorithm for retries passing the parameters max_retries and the initial waiting time. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/table/batch_writer.html#DynamoDB.Table.batch_writer
Proposed Solution
No response
Other Information
No response
Acknowledgements
SDK version used
1.34.116
Environment details (OS name and version, etc.)
Python 3.12