aws / aws-sdk-java-v2

The official AWS SDK for Java - Version 2
Apache License 2.0
2.12k stars 804 forks source link

S3TransferManager - Support download / upload resume in the event of host machine power loss #5343

Open jaychung-aws opened 1 week ago

jaychung-aws commented 1 week ago

Describe the feature

Update S3TransferManager to provide a mechanism to resume download / upload in the event of host machine power loss.

Use Case

My IoT application utilizes the S3TransferManager to download and upload files from Amazon S3. Ensuring the ability to resume interrupted download and upload sessions is a critical requirement to avoid unnecessary data transfer costs, not only during intermittent network disconnections, but also in the event of power loss on the device.

Requesting S3TransferManager to provide a mechanism to resume downloads and uploads from the last byte location after the host device has recovered from abrupt power loss.

Proposed Solution

No response

Other Information

No response

Acknowledgements

AWS Java SDK version used

2.20.79

JDK version used

openjdk version "17.0.11" 2024-04-16 LTS OpenJDK Runtime Environment Corretto-17.0.11.9.1 (build 17.0.11+9-LTS) OpenJDK 64-Bit Server VM Corretto-17.0.11.9.1 (build 17.0.11+9-LTS, mixed mode, sharing)

Operating System and version

Ubuntu 22.04

bhoradc commented 1 week ago

Hi @jaychung-aws,

Thanks for submitting this request. S3TransferManager in V2 does support Pause/Resume on Upload/Download in case of network loss - https://github.com/aws/aws-sdk-java-v2/issues/3120, but don't think it considers power loss events.

We'll review it with the Java SDK team further.

Regards, Chaitanya