minio / warp

S3 benchmarking tool
GNU Affero General Public License v3.0
567 stars 105 forks source link

warp multipart test facing return code "405 Method Not Allowed" #275

Closed eveningcafe closed 1 year ago

eveningcafe commented 1 year ago

Warp version latest. When I test the multipart upload functionality, I encounter a bug. I'm unsure whether it is a bug on the server or the client side. Has anyone else experienced the same issue?

warp multipart --bucket warp-benchmark-bucket-multiple --parts=200 --part.size=5MiB --debug

Result

GET /warp-benchmark-bucket-multiple/warp-multipart.bin?partNumber=164 HTTP/1.1
Host: localhost
User-Agent: MinIO (linux; amd64) minio-go/v7.0.57 warp/(dev)
Authorization: AWS4-HMAC-SHA256 Credential=6b9d6f0bf2014ae4b4cabf192a761692/20230626/localregion/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230626T023050Z
Accept-Encoding: gzip

---------START-HTTP---------
HTTP/1.1 405 Method Not Allowed
Transfer-Encoding: chunked
Connection: keep-alive
Content-Type: application/xml
Date: Mon, 26 Jun 2023 02:30:50 GMT
X-Amz-Id-2: txb249f3012ca94666939f4-006498f85a
X-Amz-Request-Id: txb249f3012ca94666939f4-006498f85a
X-Openstack-Request-Id: txb249f3012ca94666939f4-006498f85a
X-Trans-Id: txb249f3012ca94666939f4-006498f85a

10e
<?xml version='1.0' encoding='UTF-8'?>
<Error><Code>MethodNotAllowed</Code><Message>The specified method is not allowed against this resource.</Message><RequestId>txb249f3012ca94666939f4-006498f85a</RequestId><Method>GET</Method><ResourceType>PART</ResourceType></Error>
eveningcafe commented 1 year ago

Okay, I figured it out. The Warp client uses path-style requests, while the server uses domain-style requests. https://github.com/minio/minio/tree/master/docs/config#domain

Can we support for domain-style requests (as an --option ) in Warp? This would be useful for people who want to use Warp to benchmark AWS S3 buckets (or another s3 compatitive system), as AWS plans to deprecate path-style requests in the future (https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/).

harshavardhana commented 1 year ago

Date: Mon, 26 Jun 2023 02:30:50 GMT X-Amz-Id-2: txb249f3012ca94666939f4-006498f85a X-Amz-Request-Id: txb249f3012ca94666939f4-006498f85a X-Openstack-Request-Id: txb249f3012ca94666939f4-006498f85a X-Trans-Id: txb249f3012ca94666939f4-006498f85a

Your server does not implement GetObject with partNumber https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestSyntax

Please fix your server this has nothing to do with pathStyle v/s DNS bucket style.