Uploading to S3 is broken

[johannesnagl]

Hi Miles!

We're experiencing multiple problems with the latest CakeUploader (4.5.5) and the latest AWS SDK PHP (2.7.2)

composer.json:

"mjohnson/uploader": "4.5.5",
"aws/aws-sdk-php": "2.7.2"

User.php:

  var $actsAs = [
    'Uploader.Attachment' => [
      'new_avatar' => [
        'nameCallback' => 'uploaderFileNameCallback',
        'dbColumn' => 'column1',
        'stopSave' => false,
        'allowEmpty' => true,
        'cleanup' => false,
        'transforms' => [
          '50x50' => ['class' => 'crop', 'width' => 50, 'height' => 50, 'dbColumn' => 'column2'],
        ],
        'transport' => [
          'class' => AttachmentBehavior::S3,
          'accessKey' => AWS_ACCESS_KEY,
          'secretKey' => AWS_SECRET_KEY,
          'bucket' => AWS_AVATAR_BUCKET_NAME,
          'region' => 'eu-central-1',
          'scheme' => 'https'
        ]
      ],
    ]
  ];

The problems in detail:

1. Original file is uploaded to the aws bucket (in the root), but stored in column1 with '/files/uploads/XXX.jpg'
2. The transformed file isn't stored at all but saved in the db column2 with "/files/uploads/XXX-cropped.jpg"

In previous versions of the Uploader (4.2.0) the same setup had moved the files correctly to AWS and saved the complete bucket-url (like https://bucket.s3.amazon.com/filename.jpg (without /files/uploads)) to the database.

The only thing we managed to fix was the following scenario:

adding the following code

public function beforeUpload($options) {
  $options['finalPath'] = '';
  $options['uploadDir'] = WWW_ROOT . $options['finalPath'];

  return $options;
}

to the user model would lead to the following situation:

1. Original file is uploaded to the aws bucket (in the root) AND saved in column1 without a path (like XXX.jpg). So a manual prepending of https://bucket.s3.amazon.com/ would help to show the image.
2. Nonetheless, the transformed file isn't stored and the column2 path would be /files/uploads/XXX-cropped.jpg

Do you have any help?

[milesj]

Would have to look into it, but nothing comes to me off the top of my head. Most likely something changed in the AWS SDK.

[johannesnagl]

do you know of any other stable implementations of the latest uploader with AWS support? would love to see other configuration(-hacks) :)

[milesj]

I haven't used the uploader in over a year, so I can't really say which version works best :/.

Perhaps try lowering the AWS version until it works? If it does work at some point, could you let me know which version it broke at?

[johannesnagl]

Hi miles!

It looks like it's related to the bucket(-region). When referencing an EU_WEST_1 bucket, everything's fine. When referencing an EU_CENTRAL_1 bucket, the upload process breaks.

So it's somehow unrelated to the AWS SDK version (still on 2.7.2) and the uploader. I think it's related to the fact, that EU_CENTRAL_1 only supports AWS Authenticating with V4 Standard (instead of the old V2 standard).

I will dig into this and keep you posted!

[johannesnagl]

ok,

the problem is the following call within milesj/transit: https://github.com/milesj/transit/blob/master/src/Transit/Transporter/Aws/S3Transporter.php#L159

The method S3Client::getEndpoint (https://github.com/aws/aws-sdk-php/blob/master/src/Aws/Common/Client/AbstractClient.php#L115) is deprecated and does not work with newer regions.

The solution for us, at the moment, is to define the region for our selves. (https://github.com/aws/aws-sdk-php/blob/master/src/Aws/S3/Resources/s3-2006-03-01.php)

Sorry for the misleading issue!

[milesj]

Awesome, thanks for looking into it. I'll look around for an alternative to getEndpoint(). Hopefully it's not too complicated. It's like the third time I've had to change that code because AWS changes their minds <_<.

[johannesnagl]

Hi miles, have a look at this comment: https://github.com/aws/aws-sdk-php/pull/385

[milesj]

That looks perfect, cheers.

[milesj]

This should be fixed in 4.6.0.

[johannesnagl]

thx, we updated to the latest version and it works like expected.

[milesj]

Sweet, thanks for the help also!