`mc cp` does not respect `--enc-c` when copying from one file to another

NiklasRosenstein commented 1 month ago

Expected behavior

The command mc cp minio/bucket/ minio/bucket2/ --enc-c minio/bucket2=... should copy the unencrypted files from bucket1 to bucket2 but encrypt them at the destination.

Actual behavior

The files end up unencrypted in the destination.

Steps to reproduce the behavior

All good copying from a local file:

$ echo "Hello, World" > hello.txt
$ mc cp hello.txt minio/bucket/hello.txt
$ mc cat minio/bucket/hello.txt
Hello, World
$ mc cp hello.txt minio/bucket/hello-encrypted.txt --enc-c minio/=cd19f357a2788152a6be3fa483c9a2088137bdcd133a0789d8f55faf888cb07b  # openssl rand -hex 32
$ mc cat minio/bucket/hello-encrypted.txt
mc: <ERROR> Unable to read from `hcloud/csi-s3/hello-encrypted.txt`. 400 Bad Request.
$ mc cat minio/bucket/hello-encrypted.txt --enc-c minio/=cd19f357a2788152a6be3fa483c9a2088137bdcd133a0789d8f55faf888cb07b
Hello, World

But copying from the file already in the bucket, the copy is not encrypted.

$ mc cp minio/bucket/hello.txt minio/bucket/hello-encrypted2.txt --enc-c minio/=cd19f357a2788152a6be3fa483c9a2088137bdcd133a0789d8f55faf888cb07b
$ mc cat minio/bucket/hello-encrypted2.txt
Hello, World

I expect that last command to fail with 400 Bad Request because no SSE-C was provided.

mc --version

mc version RELEASE.2024-09-16T17-43-14Z (commit-id=11ebe952ea30e426e564f66e78d178465ae7c432)
Runtime: go1.22.6 linux/amd64
Copyright (c) 2015-2024 MinIO, Inc.
License GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>

System information

I've been testing this against Hetzner's Object Storage which to my knowledge uses MinIO; in case that bears any relevance.

It also seems worth noting that mc mirror appears to be ignoring the --enc-c flag as well. My use case is copying from a bucket without SSE-C to a new bucket and this time using SSE-C.

zveinn commented 1 month ago

The fact that it's not throwing errors has already been dealt with on the master branch.

However..

When moving from one bucket to another, we would suggest using different names, because the --enc-c parameter is applied to anything that fits the given path before the key. Which means it will try to decrypt the original object, which is not encrypted and it'll fail. If it's not a possibility to use different bucket names, then you could use a different sub path:

bucket/path1/object.txt
bucket/path2/object.txt

There is a possibility for us to make this work, but it would require changes in the core minio code. Since it is minio itself which is returning a 400 response for the head request since it's expecting it to be encrypted based on the fact you included an encryption key for that path.

2024-10-01T12:05:17.772 [400 Bad Request] s3.HeadObject 127.0.0.1:9000/bb1/meow 127.0.0.1        722µs       ⇣  684.277µs  ↑ 213 B ↓ 0 B

I will start a small discussion internally to see if this is something we'd want to do .. but since it would change core functionality I am doubtful it'll happen.

For now I'm closing this issue as completed. If you find anything new related to this issue feel free to comment again and we can re-open if needed.

zveinn commented 1 month ago

Just did a little test to compare against aws s3 and they are returning 400 on the header as well. We want to be in-line with how aws s3 functions so we will not be changing how this works.

minio / mc