search

minio / mc

Unix like utilities for object store
https://min.io/download
GNU Affero General Public License v3.0
2.87k stars 549 forks source link

"api": "s3v2" in config is not respected, even though --debug mode says it is #4915

Closed lgyurci closed 6 months ago

lgyurci commented 6 months ago

Expected behavior

"api": "S3v2" in config should cause the mc client to use v2 signature. Moreover, there shouldn't be a difference in the http headers seen with the --debug flag, and in the headers that are actually sent out in the packets.

Actual behavior

"api": "S3v2" in config does not cause the mc client to use v2 signature, even though in debug mode it seems like it is. There is a difference between headers seen with the --debug flag, and in the headers that are actually sent out in the packets.

Steps to reproduce the behavior

  1. Set api to s3v2
  2. Start wireshark or tcpdump with applicable flags
  3. Run mc with --debug flag
  4. Observe, that according to mc, the Authorization header is "AWS REDACTED:REDACTED". This corresponds to aws v2 signature, as it would be expected.
  5. Meanwhile, according to the packet capture, the Authorization header is "AWS4-HMAC-SHA256 Credential=something, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=something". This corresponds to aws v4 signature, which is not expected.

    mc --version

    mc version RELEASE.2024-04-29T09-56-05Z (commit-id=5b7b2223717a32ff01d63d57c2d040a719ca581e) Runtime: go1.21.9 linux/amd64

harshavardhana commented 6 months ago
  1. Set api to s3v2
  2. Start wireshark or tcpdump with applicable flags
  3. Run mc with --debug flag
  4. Observe, that according to mc, the Authorization header is "AWS REDACTED:REDACTED". This corresponds to aws v2 signature, as it would be expected.
  5. Meanwhile, according to the packet capture, the Authorization header is "AWS4-HMAC-SHA256 Credential=something, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=something". This corresponds to aws v4 signature, which is not expected.

can you show me the packet capture?

lgyurci commented 6 months ago

Yes. I anonimized some things in there with "W" letters. Also, this is a single GET request, but the headers in question are the same in every request. s3v2v2.zip

harshavardhana commented 6 months ago

can you share mc ls --debug output for this?

harshavardhana commented 6 months ago

and also mc alias list alias/ ?

lgyurci commented 6 months ago

Sure. Same anonimization rule applies.

root@WWWW:~# mc ls --debug localbucket/WWWWW
mc: <DEBUG> GET /WWWWW/?location= HTTP/1.1
Host: 10.185.0.144:8080
User-Agent: MinIO (linux; amd64) minio-go/v7.0.70 mc/RELEASE.2024-04-29T09-56-05Z
Accept-Encoding: zstd,gzip
Authorization: AWS **REDACTED**:**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240502T112434Z

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 137
Connection: Keep-Alive
Date: Thu, 02 May 2024 11:24:34 GMT
X-Amz-Request-Id: tx0000088fbb1c964bdd8cc-00663377f2-8a54f9-ceph-store

mc: <DEBUG> Response Time: 2.285532ms

mc: <DEBUG> HEAD /WWWWW/ HTTP/1.1
Host: 10.185.0.144:8080
User-Agent: MinIO (linux; amd64) minio-go/v7.0.70 mc/RELEASE.2024-04-29T09-56-05Z
Authorization: AWS **REDACTED**:**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240502T112434Z

mc: <DEBUG> HTTP/1.1 200 OK
Connection: Keep-Alive
Date: Thu, 02 May 2024 11:24:34 GMT
X-Amz-Request-Id: tx000005f6c7ec01a3a8d9f-00663377f2-8a54f9-ceph-store
X-Rgw-Bytes-Used: 37567454
X-Rgw-Object-Count: 79
X-Rgw-Quota-Bucket-Objects: -1
X-Rgw-Quota-Bucket-Size: -1
X-Rgw-Quota-Max-Buckets: 1
X-Rgw-Quota-User-Objects: -1
X-Rgw-Quota-User-Size: 10737418240
Content-Length: 0

mc: <DEBUG> Response Time: 20.674677ms

mc: <DEBUG> GET /WWWWW/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix= HTTP/1.1
Host: 10.185.0.144:8080
User-Agent: MinIO (linux; amd64) minio-go/v7.0.70 mc/RELEASE.2024-04-29T09-56-05Z
Accept-Encoding: zstd,gzip
Authorization: AWS **REDACTED**:**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240502T112434Z

mc: <DEBUG> HTTP/1.1 200 OK
Transfer-Encoding: chunked
Connection: Keep-Alive
Content-Type: application/xml
Date: Thu, 02 May 2024 11:24:34 GMT
X-Amz-Request-Id: tx0000014cc6a071d589201-00663377f2-8a54f9-ceph-store

mc: <DEBUG> Response Time: 2.84579ms

[2024-05-02 11:24:34 UTC]     0B sample1/
[2024-05-02 11:24:34 UTC]     0B sample2/
[2024-05-02 11:24:34 UTC]     0B sample3/

root@WWWW:~# mc alias list localbucket/
localbucket
  URL       : http://10.185.0.144:8080/
  AccessKey : WWWWWW
  SecretKey : WWWWWWW
  API       : S3v2
  Path      :
harshavardhana commented 6 months ago

Looks like a regression sending a fix.

lgyurci commented 6 months ago

That was extremely quick! Thank you