Partial Support for S3's UploadPartCopy Functionality

Description

We aim to implement a workaround to provide partial support for Amazon S3's UploadPartCopy functionality. This will primarily cater to customers attempting to move or copy whole files larger than 5GB, by identifying UploadPartCopy requests and triggering a server-side copy when the last range of an entire object comes in.

Problem/Pain Point

Our current architecture doesn't allow us to easily copy files by arbitrary ranges, leading to a lack of support for S3's UploadPartCopy. This has resulted in difficulties for key customers and integrations, especially those using S3 clients and libraries (such as boto3 for Python) that default to using UploadPartCopy for files over 5GB.

Why Now

This issue has been causing errors and dissatisfaction among our users. Given our current ability to perform server-side copy of entire objects, it is both a feasible and immediate solution to alleviate this problem and improve the reliability of our service.

Acceptance Criteria

Identification and handling of UploadPartCopy requests involving entire objects
Triggering of a server-side copy once the last range of an entire object comes in
Return of an error for UploadPartCopy requests involving arbitrary ranges

Out of Scope

Full compatibility with S3's UploadPartCopy
Copying files by arbitrary ranges

Success Metrics

Reduction in the number of errors experienced by customers using S3 clients for files over 5GB
Positive feedback from key customers and integrations about the partial UploadPartCopy support

storj / roadmap