search

awslabs / amazon-sqs-java-extended-client-lib

An extension to the Amazon SQS client that enables sending and receiving messages up to 2GB via Amazon S3.
Apache License 2.0
208 stars 109 forks source link

Use batch payload delete, and properly chain deletes for async #148

Open richardsmithsfdc opened 3 months ago

richardsmithsfdc commented 3 months ago

Issue #, if available:

Description of changes:

Two issues in the new async support were bothering me:

  1. Order of operations between async and sync versions of the delete message operation were different. Sync deletes from S3 and then from SQS. Async deletes from SQS and then S3. This changes the async version of the code to match the original sync code.
  2. In delete message batch, the delete futures for S3 were not chained into the delete future for SQS. In practice, this might not always cause a problem, but it is poor practice.

This pull request solves both problems by performing the delete-from-S3 and delete-from-SQS operations in the same order for both sync and async code, and by properly chaining them the same way for both.

Note that this new code makes use of a new feature in payload offloading, which deletes a batch of offloaded pointers from the store. This new feature is in this pull request: https://github.com/awslabs/payload-offloading-java-common-lib-for-aws/pull/57

Because that feature is only proposed right now, I am creating this as a draft PR for the time being.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

ziyanli-amazon commented 3 months ago

ty. Will take a look next week