aws / aws-sdk-php

Official repository of the AWS SDK for PHP (@awsforphp)
http://aws.amazon.com/sdkforphp
Apache License 2.0
6.04k stars 1.23k forks source link

Client Side Encryption with S3EncryptionMultipartUploaderV2 fails on large files. #2512

Open elt4n0 opened 2 years ago

elt4n0 commented 2 years ago

Describe the bug

When using S3EncryptionMultipartUploaderV2 with a file larger than 4GB a warning is thrown during encryption, and an exception is thrown during upload. This does not happen with smaller files.

Expected Behavior

Success on S3EncryptionMultipartUploaderV2::upload()

Current Behavior

When using S3EncryptionMultipartUploaderV2 with a file larger than 4GB this message appears:

PHP Warning: openssl_encrypt(): data is too long in libs\aws\aws-sdk-php\src\Crypto\AesGcmEncryptingStream.php on line 108

After a while, exception is thrown:

An exception occurred while completing a multipart upload: Error executing "CompleteMultipartUpload" on "encryption-test/lfmb_TESNICO.mp4?uploadId=345dd8b5-b470-4c92-8525-86298794e9e4"; AWS HTTP error: Client error: `POST encryption-test/lfmb_TESNICO.mp4?uploadId=345dd8b5-b470-4c92-8525-86298794e9e4` resulted in a `400 Bad Request` response:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed (truncated...)
 MalformedXML (client): The XML you provided was not well-formed or did not validate against our published schema. - <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate against our published schema.</Message><Key>lfmb_TESNICO.mp4</Key><BucketName>encryption-test</BucketName><Resource>/encryption-test/lfmb_TESNICO.mp4</Resource><RequestId>170FDB5FFCA88848</RequestId><HostId>4bf9e65b-bbb0-488d-9da9-d54f60abb5c0</HostId></Error>

Reproduction Steps

$uploader = new S3EncryptionMultipartUploaderV2( 
         new Aws\S3\S3Client([
            'version' => 'latest',
            'region'  => 'us-east-1',
            'use_path_style_endpoint' => true,
            'credentials' => [
                    'key'    => $options['credentials']['key'],
                    'secret' => $options['credentials']['secret'],
                ]
        ]), ,fopen('path to a file larger than 4GB', 'r'),
            [
                '@MaterialsProvider' => $materialsProvider,
                '@CipherOptions' => [ 'Cipher' => 'gcm', 'KeySize' => 128 ],
                '@KmsEncryptionContext' => [],
                'bucket' => $this->bucket,
                'key' => 'thenamefile',
                'concurrency' => 5,
                'part_size' => null,
            ]
        );

$uploader->upload();

Possible Solution

openssl_encrypt is know have trouble with large files. Maybe include a mechanism to encode by chunks.

Additional Information/Context

No response

SDK version used

"aws/aws-sdk-php": "^3.147"

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 7.4.30 (cli) (built: Jun 7 2022 15:36:03) ( NTS Visual C++ 2017 x64 )

yenfryherrerafeliz commented 2 years ago

Hi @elt4n0, thanks for opening this issue. I have reproduced the reported behavior and indeed the issue is coming from openssl library. I am going to leave this issue under review so we can address this further, and will provide updates accordingly.

Thanks!