Content-Encoding is blank in S3 Console when `aws-chunked` is passed

aws / aws-sdk

Landing page for the AWS SDKs on GitHub

https://aws.amazon.com/tools/

Other

71 stars 14 forks source link

Content-Encoding is blank in S3 Console when `aws-chunked` is passed #498

Open tim-finnigan opened 1 year ago

tim-finnigan commented 1 year ago

Original issue: https://github.com/boto/boto3/issues/3629

tim-finnigan commented 1 year ago

P83760338 / D119267049

BroMattMiller commented 6 months ago

This blank header is breaking my website when I turn on CloudFront compression. The user's web browser ends up getting two Content-Encoding headers, the blank one from S3 and the "br" one added by CloudFront. Firefox is able to deal with this and correctly respects the "br" one, but Chrome and others don't recognize the content as compressed. All I did was use aws s3 sync ... to upload these files, which I guess decides to use chunking on some files.

So, just adding this common use case to this bug.

tmarcu commented 3 months ago

I see this issue is popping up in various language sdk's, so I'll add my findings here: I am using the aws-sdk-go-v2 and experiencing the same problem - empty (no value) Content-Encoding header automatically added to my objects even when it is not specified. Debugging further, this happens specifically when uploading objects that do not need multiple chunks to upload. If I create an object that requires at least 2 chunks (e.g 6MB+), the header is not added anymore. On the go sdk side, this splits off at https://github.com/aws/aws-sdk-go-v2/blob/main/feature/s3/manager/upload.go#L355.

ben6reul commented 1 month ago

I'm seeing this issue as well when upgrading from 1.12.157 -> 2.25.47 with the java SDK for small files under the chunking threshold.