mc stat does not handle serverEncryption info correctly

HolyShitMan commented 4 months ago

Expected behavior

If ~~x-amz-server-side-encryption-enabled~~ x-amz-server-side-encryption-bucket-key-enabled is set to false , mc stat should output, that bucket is not server side encrypted.

Actual behavior

In cmd/stat.go: Since Metadata has serverEncryptionKeyPrefix as Prefix, maxKeyEncrypted becomes len(k) and mc assumes object is server side encrypted.

Steps to reproduce the behavior

S3 server that answers with If ~~x-amz-server-side-encryption-enabled~~ x-amz-server-side-encryption-bucket-key-enabled set to `false. We have an internal one, written with rust-s3

version

RELEASE.2024-06-29T19-08-46Z

harshavardhana commented 4 months ago

If x-amz-server-side-encryption-enabled is set to false , mc stat should output, that data is not server side encrypted.

there is no such AWS S3 Header called as x-amz-server-side-encryption-enabled - can you show me documentation from AWS S3?

HolyShitMan commented 4 months ago

Sorry, i had a confusion their: The correct Header i ment is x-amz-server-side-encryption-bucket-key-enabled ~~and it is only about mc stat of a bucket.~~ https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html#configure-bucket-key

klauspost commented 4 months ago

The link you provide is a request header.

Are you saying this header is returned on HeadObject responses?

HolyShitMan commented 4 months ago

Got my debug setup up running again. And yes: s3s 0.8.1 answers with x-amz-server-side-encryption-bucket-key-enabled = false on object stat calls., which leads to an mc output: Encryption: SSE-S3 And my inbetween thought that this is about bucket stat was wrong. In the following you find the complete output of my debug setup, where I printed all metadata key and value in /cmd/stat.go :

$ ./mc stat s3_proxy/demobucket/README.md
Content-Type: text/markdown
X-Amz-Server-Side-Encryption-Bucket-Key-Enabled: false
Name      : README.md
Date      : 2024-06-13 14:05:01 CEST 
Size      : 3.0 KiB 
ETag      : 7854c7e3fc0e797c68a408e6cc3216f4 
Type      : file 
Encryption: SSE-S3
Metadata  :
  Content-Type: text/markdown

harshavardhana commented 4 months ago

https://github.com/minio/mc/pull/4976 Please try this fix

HolyShitMan commented 4 months ago

Works for me. Now i get:

Name      : README.md
Date      : 2024-06-13 14:05:01 CEST 
Size      : 3.0 KiB 
ETag      : 7854c7e3fc0e797c68a408e6cc3216f4 
Type      : file 
Encryption: SSE-Unknown
Metadata  :
  Content-Type: text/markdown

From my point of view SSE-disabled would be more precise, but i agree that there still could be some unknown SSE method.

harshavardhana commented 4 months ago

Can you send the output for --debug ?

harshavardhana commented 4 months ago

Encryption: SSE-Unknown

yeah this won't even be printed in the new revision of the PR.

HolyShitMan commented 4 months ago

Can you send the output for --debug ?

Could be to late, but for sake of completeness

./mc stat s3_proxy/demobucket/README.md --debug
mc: <DEBUG> GET /demobucket/?location= HTTP/1.1
Host: localhost:8000
User-Agent: MinIO (linux; amd64) minio-go/v7.0.72 mc/DEVELOPMENT.2024-07-03T07-29-50Z
Accept-Encoding: zstd,gzip
Authorization: AWS4-HMAC-SHA256 Credential=[...]/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240704T055915Z

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 91
Content-Type: application/xml
Date: Thu, 04 Jul 2024 05:59:15 GMT
X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
X-Amz-Request-Id: 17DEED0EE185CEB9

mc: <DEBUG> Response Time: 99.954474ms

mc: <DEBUG> HEAD /demobucket/README.md HTTP/1.1
Host: localhost:8000
User-Agent: MinIO (linux; amd64) minio-go/v7.0.72 mc/DEVELOPMENT.2024-07-03T07-29-50Z
Authorization: AWS4-HMAC-SHA256 Credential=[...]/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240704T055915Z

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 3079
Accept-Ranges: bytes
Content-Type: text/markdown
Date: Thu, 04 Jul 2024 05:59:15 GMT
Etag: "7854c7e3fc0e797c68a408e6cc3216f4"
Last-Modified: Thu, 13 Jun 2024 12:05:01 GMT
X-Amz-Delete-Marker: false
X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
X-Amz-Missing-Meta: 0
X-Amz-Mp-Parts-Count: 0
X-Amz-Request-Id: 17DEED0EE431EFF3
X-Amz-Server-Side-Encryption-Bucket-Key-Enabled: false

mc: <DEBUG> Response Time: 44.966577ms

mc: <DEBUG> GET /demobucket/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=README.md HTTP/1.1
Host: localhost:8000
User-Agent: MinIO (linux; amd64) minio-go/v7.0.72 mc/DEVELOPMENT.2024-07-03T07-29-50Z
Accept-Encoding: zstd,gzip
Authorization: AWS4-HMAC-SHA256 Credential=[...]/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240704T055915Z

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 582
Content-Type: application/xml
Date: Thu, 04 Jul 2024 05:59:15 GMT
X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
X-Amz-Request-Id: 17DEED0EE68077F9

mc: <DEBUG> Response Time: 38.899855ms

mc: <DEBUG> HEAD /demobucket/README.md HTTP/1.1
Host: localhost:8000
User-Agent: MinIO (linux; amd64) minio-go/v7.0.72 mc/DEVELOPMENT.2024-07-03T07-29-50Z
Authorization: AWS4-HMAC-SHA256 Credential=[...]/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20240704T055915Z

mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 3079
Accept-Ranges: bytes
Content-Type: text/markdown
Date: Thu, 04 Jul 2024 05:59:15 GMT
Etag: "7854c7e3fc0e797c68a408e6cc3216f4"
Last-Modified: Thu, 13 Jun 2024 12:05:01 GMT
X-Amz-Delete-Marker: false
X-Amz-Id-2: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8
X-Amz-Missing-Meta: 0
X-Amz-Mp-Parts-Count: 0
X-Amz-Request-Id: 17DEED0EE8D4F1C0
X-Amz-Server-Side-Encryption-Bucket-Key-Enabled: false

mc: <DEBUG> Response Time: 37.94728ms

Name      : README.md
Date      : 2024-06-13 14:05:01 CEST 
Size      : 3.0 KiB 
ETag      : 7854c7e3fc0e797c68a408e6cc3216f4 
Type      : file 
Encryption: SSE-Unknown
Metadata  :
  Content-Type: text/markdown

minio / mc