overture-stack / score

Secure Cloud Object REsource: file transfer microservice
https://www.overture.bio/products/score
GNU Affero General Public License v3.0
18 stars 10 forks source link

Object Retrieval: Support for Multipart Uploads #411

Closed Azher2Ali closed 3 weeks ago

Azher2Ali commented 1 month ago

Retrieving Object Specification:

Similar initial check for existing objectSpec. If not found, fetches object metadata. New: Additionally checks for multipart uploads using s3Client.listMultipartUploads and filters for the specific object key. New: If a multipart upload is found, it retrieves individual parts using s3Client.listParts and calculates offset based on part summaries.

Calculating Parts:

Moved: The logic for calculating parts based on offset and length is moved after retrieving the object specification (applicable only when forExternalUse is false). Rest of the logic for part calculation and validation remains similar.

Overall Description:

The changes introduced enhanced the way object information is retrieved. The new code now considers the possibility of multipart uploads. If a multipart upload exists for the object, it retrieves individual parts along with their sizes and calculates the overall offset.