search

minio / mc

Simple | Fast tool to manage MinIO clusters :cloud:
https://min.io/download
GNU Affero General Public License v3.0
2.84k stars 543 forks source link

mc rm issues a POST instead of a DELETE for single object removals #5024

Closed cholcombe973 closed 1 month ago

cholcombe973 commented 1 month ago

Expected behavior

mc rm should issue a DELETE request for single object removals.

Actual behavior

What I'm seeing in the logs is mc is issuing a DeleteObjects request with a single object in it. This doesn't follow the S3 spec.

Steps to reproduce the behavior

  1. Create a bucket
  2. Upload a single object
  3. Issue an mc rm mystorage/bucket/object
  4. Incorrect POST request is issued.

mc --version

harshavardhana commented 1 month ago

yes it invokes MULTI object delete for all cases 1 or multiple objects.

harshavardhana commented 1 month ago

What I'm seeing in the logs is mc is issuing a DeleteObjects request with a single object in it. This doesn't follow the S3 spec.

S3 spec has no restriction on how many objects you can delete in multi-object request it can be 1 or 1000 anywhere in between.

cholcombe973 commented 1 month ago

That's correct but mc returns an error saying it expected a single DeleteObject result. If mc sees that only a single object delete is being requested shouldn't it use this https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html instead of the multi delete API? I guess what I'm getting at here is as far as correctness goes issuing this request: https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html for a single object and then expecting the result back for https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html isn't right.

marktheunissen commented 1 month ago

but mc returns an error saying it expected a single DeleteObject result.

@cholcombe973 Please could you paste all the error logs you see and the commands you issued?

harshavardhana commented 1 month ago

That's correct but mc returns an error saying it expected a single DeleteObject result. If mc sees that only a single object delete is being requested shouldn't it use this https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html instead of the multi delete API?

Please share the --debug output. works perfectly fine with AWS S3

mc: <DEBUG> POST /?delete= HTTP/1.1
Host: sinhaminio.s3.dualstack.us-east-1.amazonaws.com
User-Agent: MinIO (linux; amd64) minio-go/v7.0.75 mc/DEVELOPMENT.GOGET
Content-Length: 73
Accept-Encoding: zstd,gzip
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20240822/us-east-1/s3/aws4_request, SignedHeaders=content-md5;host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
Content-Md5: nKeMYgTJHOrxnjLP5s/Qtw==
X-Amz-Content-Sha256: 8669c0dc41dcf8c4555e1a95eb8bce631138385322109b4ac833ee323c358e9d
X-Amz-Date: 20240822T003440Z

mc: <DEBUG> HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 22 Aug 2024 00:34:41 GMT
Server: AmazonS3
X-Amz-Id-2: wPkJDvRJvaMHbxwh4oj8G9jjQIzOdQnQ6DY2RqSdpNZiuoEKac0lV6j/IbNt8vETIDU4UZR6j6I=
X-Amz-Request-Id: BSTFQ4SDB8XPV9H7

mc: <DEBUG> TLS Certificate found: 
mc: <DEBUG>  >> Country: US
mc: <DEBUG>  >> Organization: Amazon
mc: <DEBUG>  >> Expires: 2025-05-02 23:59:59 +0000 UTC
mc: <DEBUG> TLS Certificate found: 
mc: <DEBUG>  >> Country: US
mc: <DEBUG>  >> Organization: Amazon
mc: <DEBUG>  >> Expires: 2030-08-23 22:21:28 +0000 UTC
mc: <DEBUG> TLS Certificate found: 
mc: <DEBUG>  >> Country: US
mc: <DEBUG>  >> Organization: Starfield Technologies, Inc.
mc: <DEBUG>  >> Expires: 2037-12-31 01:00:00 +0000 UTC
mc: <DEBUG> TLS Certificate found: 
mc: <DEBUG>  >> Country: US
mc: <DEBUG>  >> Organization: Starfield Technologies, Inc.
mc: <DEBUG>  >> Expires: 2034-06-28 17:39:16 +0000 UTC
mc: <DEBUG> Response Time:  108.040244ms

Removed `s3/sinhaminio/test.csv`.
harshavardhana commented 1 month ago

From what I sense, your S3 compatible storage does not implement multi-object delete properly.