S3 putObject is generating incorrect signatures in some cases

[n-johnson]

When a multibyte utf-8 character is included in the Metadata object, the sdk currently generates an invalid signature causing upload to fail if the Body is a string. However if a buffer is used, the correct signature is generated.

Simple example:

const AWS = require('aws-sdk');

AWS.config.update({
  accessKeyId: process.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});

AWS.config.update({ region: 'us-east-1' });

const bucketName = 'some-s3-bucket';
const bucket = new AWS.S3({ params: { Bucket: bucketName } });

// Will fail
bucket.putObject({
  Body: 'actual string content',
  Key: '/some/file/path',
  Bucket: bucketName,
  Metadata: { 'key': 'multibyte char ç' }
}, (err, res) => {
  console.log('String', err, res);
});

// Will succeed
bucket.putObject({
  Body: new Buffer('not actual string content'),
  Key: '/some/file/path2',
  Bucket: bucketName,
  Metadata: { 'key': 'multibyte char ç' }
}, (err, res) => {
  console.log('Buffer', err, res);
});

Output:

String { [SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.]
  message: 'The request signature we calculated does not match the signature you provided. Check your key and signing method.',
  code: 'SignatureDoesNotMatch',
  region: null,
  time: Thu Jan 12 2017 17:39:53 GMT-0800 (PST),
  requestId: '45955EF049944D58',
  extendedRequestId: 'TkUKuDhpxI5lYz6uENPSA2udwUUAMgqVT6QSKiGYNcvUYDSGgNj2Dr1Ds7dPhHKmRnwcXQDvyZU=',
  cfId: undefined,
  statusCode: 403,
  retryable: false,
  retryDelay: 76.59807160962373 } null
Buffer null { ETag: '"3690727fb26da2f5984deccf3319c7be"' }

[jeskew]

I reverted the change that addressed this, as it does not appear that sending the body as a buffer has the intended effect in all clients. According to S3's documentation, metadata keys and values should only use characters from the US-ASCII character set. If S3 receives UTF-8 characters, it re-encode the header in RFC 2047 MIME encoding after verifying the signature. S3 will not decode the metadata when the object is retrieved, meaning that clients will receive the value not as multibyte char ç but instead as =?UTF-8?Q?multibyte_char_=C3=A7?=, which they may or may not decode automatically.

This behavior seems not to occur when the body is sent in the same TCP packet as the headers, which is why you see a difference between sending strings (which node attempts to concatenate with the headers) and buffers (which node sends separately). In either case, it's probably not what you want to happen, so this should be handled by URL-encoding metadata values before passing them to the SDK.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.