search

storj / gateway-st

Single-tenant, S3-compatible server to interact with the Storj network
Apache License 2.0
71 stars 19 forks source link

ListParts cannot list parts after committing the upload #55

Closed amwolff closed 2 years ago

amwolff commented 2 years ago

AWS S3 allows listing the object's parts even after committing the upload. This behaviour allows compatibility with, e.g. partNumber Request URI parameter for GetObject action.

On our side, after we commit the upload, we can't query that upload ID anymore, unfortunately. It seems like it's not an often-used feature, but preserving the retrieval ability would allow better compatibility with the spec. It would require changes on the metainfo side.

wthorp commented 2 years ago

"AWS S3 allows listing the object's parts even after committing the upload. " Does it @amwolff ?

I tried:

aws s3api create-multipart-upload --bucket=storjs3 --key=test
aws s3api upload-part --upload-id="APV..." --bucket=storjs3 --key=test --part-number 1
aws s3api list-parts --upload-id="APV..." --bucket=storjs3 --key=test
aws s3api complete-multipart-upload --upload-id="APV..." --bucket=storjs3 --key=test --multipart-upload file://parts.txt
aws s3api list-parts --upload-id="APV..." --bucket=storjs3 --key=test
aws s3api list-parts --bucket=storjs3 --key=test

list-parts with upload-id yields An error occurred (NoSuchUpload) when calling the ListParts operation: The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.

list-parts without upload-id yields: aws: error: the following arguments are required: --upload-id

wthorp commented 2 years ago

https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListParts.html isn't clear, but https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html says "You can list the parts of a specific multipart upload or all in-progress multipart uploads." [Emphasis added.]

I suggest we close this issue.

amwolff commented 2 years ago

Oops, you're right! I was so sure about this I didn't test this properly—sorry. Thank you for testing this.

I thought it's possible to list parts of committed objects since the existence of the partNumber parameter for GetObject action (it works only for completed uploads—I just tested it), but it should have been obvious to me that the inability to list parts is not mutually exclusive with the ability to download a specific part of a completed upload.

I agree with closing this issue.