search

minio / docs

MinIO Object Storage Documentation
https://docs.min.io/minio/baremetal
Creative Commons Attribution 4.0 International
541 stars 289 forks source link

Describe MinIO behavior with regards to multi-part uploads #1216

Open ramondeklein opened 1 month ago

ramondeklein commented 1 month ago

MinIO behavior with regards to multi-part uploads deviates from AWS S3 in the following ways:

Issues https://github.com/minio/minio/issues/5613, https://github.com/minio/minio/issues/11686 and https://github.com/minio/minio/issues/13246 proof that ListMultipartUploads can only be used to list multi-part uploads if an exact object-name is specified as a prefix.

Issue https://github.com/minio/minio/issues/16120 confirms that AbortIncompleteMultipartUpload is not supported when using PutBucketLifecycle. It looks like we're scanning every 6 hours and purge non-aborted multi-part uploads after 24 hours (see stale_uploads_expiry and stale_uploads_cleanup_interval in the docs). This behavior affects all APIs that deal with aborting incomplete multipart uploads lifecycle rules.

I think we should mark the APIs in the S3 API compatibility document that have restrictions and/or deviate from the standard AWS S3 APIs and provide a proper explanation why we deviate.

feorlen commented 1 month ago

We can expand the compatibility page to add this type of information. In the process, it would be nice to move the small section on unsupported APIs from the thresholds and limits page to the compatibility page.

djwfyi commented 1 month ago

We can expand the compatibility page to add this type of information. In the process, it would be nice to move the small section on unsupported APIs from the thresholds and limits page to the compatibility page.

All of the APIs referenced on the thresholds and limits page are already listed in the unsupported API bucket endpoint list. Maybe a link between the two pages? Or just delete the section on the limits page?

ramondeklein commented 1 month ago

@djwfyi The ListMultipartUploads limitations are not described on that page.

djwfyi commented 1 month ago

@djwfyi The ListMultipartUploads limitations are not described on that page.

@ramondeklein Right. I was commenting on @feorlen's suggestion of expanding the scope of the request.

I do fear that this is a rabbit hole we don't have enough bandwidth or insight to address at the moment, but I agree that it is a good request. Just if we do it for multipart, we should expand all of them.