S3 Signature Version 4 eu-west-2 cant update thumbnail after upload

[jaredcassidy]

Type of issue

• [ ] Bug report
• [ ] Feature request
• [x] Support request

Uploader type

• [ ] Traditional
• [x] S3
• [ ] Azure

Support Request

#### Fine Uploader version 5.13.0 #### Question When uploading to S3 Signature Version 4 eu-west-2 cant update thumbnail. Have tested on standard S3 region with v2 Signature and the upload & thumbnail update works. When uploading to S3 Signature version 4 eu-west-2 the upload completes perfectly but fine uploader produces a CORS error when trying to update the thumbnail. I have checked the headers of the request for the thumbnail and there is no Access-Control-Allow-Origin, perhaps because it is a PermanentRedirect page to the other bucker url scheme. on v4 the thumbnail url ```https://s3.amazonaws.com/bucktname/xxx.jpeg?AWSAccessKeyId...``` gets a redirect page to ```https://bucketname.s3.amazonaws.com/xxx.jpeg?AWSAccessKeyId...``` with the message ```The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.``` on v2/standard region it does not seem to get redirected and therefore the CORS headers are passed Does the endpoint.php get the thumbnail S3 url from ```Aws\S3\S3Client``` ? #### Related code Using the example code from the docs and php server endpoint Console log from V4 upload ``` util.js:236 [Fine Uploader 5.13.0] Parsing template util.js:236 [Fine Uploader 5.13.0] Template parsing complete util.js:236 [Fine Uploader 5.13.0] Rendering template in DOM. util.js:236 [Fine Uploader 5.13.0] Template rendering complete util.js:236 [Fine Uploader 5.13.0] Received 1 files. util.js:236 [Fine Uploader 5.13.0] Attempting to validate image. util.js:236 [Fine Uploader 5.13.0] Generating new thumbnail for 0 util.js:236 [Fine Uploader 5.13.0] Attempting to draw client-side image preview. util.js:236 [Fine Uploader 5.13.0] Attempting to determine if WhatsApp Image 2017-02-14 at 12.43.09.jpeg can be rendered in this browser util.js:236 [Fine Uploader 5.13.0] First pass: check type attribute of blob object. util.js:236 [Fine Uploader 5.13.0] Second pass: check for magic bytes in file header. util.js:236 [Fine Uploader 5.13.0] Sending simple upload request for 0 util.js:236 [Fine Uploader 5.13.0] Submitting S3 signature request for 0 util.js:236 [Fine Uploader 5.13.0] Sending POST request for 0 util.js:236 [Fine Uploader 5.13.0] 'WhatsApp Image 2017-02-14 at 12.43.09.jpeg' is able to be rendered in this browser util.js:236 [Fine Uploader 5.13.0] EXIF header parse failed: 'No EXIF header to be found!' util.js:236 [Fine Uploader 5.13.0] EXIF data could not be parsed (No EXIF header to be found!). Assuming orientation = 1. util.js:236 [Fine Uploader 5.13.0] Sending upload request for 0 util.js:236 [Fine Uploader 5.13.0] Received response status 200 with body: util.js:236 [Fine Uploader 5.13.0] Simple upload request succeeded for 0 util.js:236 [Fine Uploader 5.13.0] Submitting upload success request/notification for 0 util.js:236 [Fine Uploader 5.13.0] Sending POST request for 0 util.js:236 [Fine Uploader 5.13.0] Received the following response body to an upload success request for id 0: {"tempLink":"https:\/\/s3.amazonaws.com\/xxx\/18e378c7-3a31-4e1d-80a6-448d10563dc8.jpeg?AWSAccessKeyId=xxxx&Expires=1487322695&Signature=r5LIkqpTF0LSgHYqv0NuIRIRVe4%3D","thumbnailUrl":"https:\/\/s3.amazonaws.com\/xxxx\/18e378c7-3a31-4e1d-80a6-448d10563dc8.jpeg?AWSAccessKeyId=xxxx&Expires=1487322695&Signature=r5LIkqpTF0LSgHYqv0NuIRIRVe4%3D"} util.js:236 [Fine Uploader 5.13.0] Upload success was acknowledged by the server. util.js:236 [Fine Uploader 5.13.0] Attempting to update thumbnail based on server response. (index):1 Access to Image at 'https://s3.amazonaws.com/xxxx/18e378c7-3a31-4e1d-80a6-448…YMQYDU23xxMX6A&Expires=1487322695&Signature=r5LIkxx0LSgHYqv0NuIRIRVe4%3D' from origin 'http://awstest.xxx.co.za' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://awstest.xxxx.co.za' is therefore not allowed access. util.js:241 [Fine Uploader 5.13.0] Problem drawing thumbnail! qq.log @ util.js:241 log @ uploader.basic.api.js:282 (anonymous) @ util.js:691 t.onerror @ image.js:96 ``` The S3 redirect page from the thumbnail url ``` PermanentRedirect The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. photospacepluploadtest photospacepluploadtest.s3.amazonaws.com A8FE2AF2E03B244F Av4hlKxAQLaT7PkWqxTqNFnnYaeyG1+kZVmuiP2/H+YIIT7WU2ItUTuiJiqGwmm02FhBYWG9ekc= ``` Fine Uploader client page ```

``` endpoint.php (not using env to test) ``` $serverPublicKey, 'secret' => $serverPrivateKey )); } // Only needed if the delete file feature is enabled function deleteObject() { getS3Client()->deleteObject(array( 'Bucket' => $_REQUEST['bucket'], 'Key' => $_REQUEST['key'] )); } function signRequest() { header('Content-Type: application/json'); $responseBody = file_get_contents('php://input'); $contentAsObject = json_decode($responseBody, true); $jsonContent = json_encode($contentAsObject); $headersStr = $contentAsObject["headers"]; if ($headersStr) { signRestRequest($headersStr); } else { signPolicy($jsonContent); } } function signRestRequest($headersStr) { $version = isset($_REQUEST["v4"]) ? 4 : 2; if (isValidRestRequest($headersStr, $version)) { if ($version == 4) { $response = array('signature' => signV4RestRequest($headersStr)); } else { $response = array('signature' => sign($headersStr)); } echo json_encode($response); } else { echo json_encode(array("invalid" => true)); } } function isValidRestRequest($headersStr, $version) { if ($version == 2) { global $expectedBucketName; $pattern = "/\/$expectedBucketName\/.+$/"; } else { global $expectedHostName; $pattern = "/host:$expectedHostName/"; } preg_match($pattern, $headersStr, $matches); return count($matches) > 0; } function signPolicy($policyStr) { $policyObj = json_decode($policyStr, true); if (isPolicyValid($policyObj)) { $encodedPolicy = base64_encode($policyStr); if (isset($_REQUEST["v4"])) { $response = array('policy' => $encodedPolicy, 'signature' => signV4Policy($encodedPolicy, $policyObj)); } else { $response = array('policy' => $encodedPolicy, 'signature' => sign($encodedPolicy)); } echo json_encode($response); } else { echo json_encode(array("invalid" => true)); } } function isPolicyValid($policy) { global $expectedMaxSize, $expectedBucketName; $conditions = $policy["conditions"]; $bucket = null; $parsedMaxSize = null; for ($i = 0; $i < count($conditions); ++$i) { $condition = $conditions[$i]; if (isset($condition["bucket"])) { $bucket = $condition["bucket"]; } else if (isset($condition[0]) && $condition[0] == "content-length-range") { $parsedMaxSize = $condition[2]; } } return $bucket == $expectedBucketName && $parsedMaxSize == (string)$expectedMaxSize; } function sign($stringToSign) { global $clientPrivateKey; return base64_encode(hash_hmac( 'sha1', $stringToSign, $clientPrivateKey, true )); } function signV4Policy($stringToSign, $policyObj) { global $clientPrivateKey; foreach ($policyObj["conditions"] as $condition) { if (isset($condition["x-amz-credential"])) { $credentialCondition = $condition["x-amz-credential"]; } } $pattern = "/.+\/(.+)\\/(.+)\/s3\/aws4_request/"; preg_match($pattern, $credentialCondition, $matches); $dateKey = hash_hmac('sha256', $matches[1], 'AWS4' . $clientPrivateKey, true); $dateRegionKey = hash_hmac('sha256', $matches[2], $dateKey, true); $dateRegionServiceKey = hash_hmac('sha256', 's3', $dateRegionKey, true); $signingKey = hash_hmac('sha256', 'aws4_request', $dateRegionServiceKey, true); return hash_hmac('sha256', $stringToSign, $signingKey); } function signV4RestRequest($rawStringToSign) { global $clientPrivateKey; $pattern = "/.+\\n.+\\n(\\d+)\/(.+)\/s3\/aws4_request\\n(.+)/s"; preg_match($pattern, $rawStringToSign, $matches); $hashedCanonicalRequest = hash('sha256', $matches[3]); $stringToSign = preg_replace("/^(.+)\/s3\/aws4_request\\n.+$/s", '$1/s3/aws4_request'."\n".$hashedCanonicalRequest, $rawStringToSign); $dateKey = hash_hmac('sha256', $matches[1], 'AWS4' . $clientPrivateKey, true); $dateRegionKey = hash_hmac('sha256', $matches[2], $dateKey, true); $dateRegionServiceKey = hash_hmac('sha256', 's3', $dateRegionKey, true); $signingKey = hash_hmac('sha256', 'aws4_request', $dateRegionServiceKey, true); return hash_hmac('sha256', $stringToSign, $signingKey); } // This is not needed if you don't require a callback on upload success. function verifyFileInS3($includeThumbnail) { global $expectedMaxSize; $bucket = $_REQUEST["bucket"]; $key = $_REQUEST["key"]; // If utilizing CORS, we return a 200 response with the error message in the body // to ensure Fine Uploader can parse the error message in IE9 and IE8, // since XDomainRequest is used on those browsers for CORS requests. XDomainRequest // does not allow access to the response body for non-success responses. if (isset($expectedMaxSize) && getObjectSize($bucket, $key) > $expectedMaxSize) { // You can safely uncomment this next line if you are not depending on CORS header("HTTP/1.0 500 Internal Server Error"); deleteObject(); echo json_encode(array("error" => "File is too big!", "preventRetry" => true)); } else { $link = getTempLink($bucket, $key); $response = array("tempLink" => $link); if ($includeThumbnail) { $response["thumbnailUrl"] = $link; } echo json_encode($response); } } // Provide a time-bombed public link to the file. function getTempLink($bucket, $key) { $client = getS3Client(); $url = "{$bucket}/{$key}"; $request = $client->get($url); return $client->createPresignedUrl($request, '+15 minutes'); } function getObjectSize($bucket, $key) { $objInfo = getS3Client()->headObject(array( 'Bucket' => $bucket, 'Key' => $key )); return $objInfo['ContentLength']; } // Return true if it's likely that the associate file is natively // viewable in a browser. For simplicity, just uses the file extension // to make this determination, along with an array of extensions that one // would expect all supported browsers are able to render natively. function isFileViewableImage($filename) { $ext = strtolower(pathinfo($filename, PATHINFO_EXTENSION)); $viewableExtensions = array("jpeg", "jpg", "gif", "png"); return in_array($ext, $viewableExtensions); } // Returns true if we should attempt to include a link // to a thumbnail in the uploadSuccess response. In it's simplest form // (which is our goal here - keep it simple) we only include a link to // a viewable image and only if the browser is not capable of generating a client-side preview. function shouldIncludeThumbnail() { $filename = $_REQUEST["name"]; $isPreviewCapable = $_REQUEST["isBrowserPreviewCapable"] == "true"; $isFileViewableImage = isFileViewableImage($filename); return !$isPreviewCapable && $isFileViewableImage; } ?> ``` The bucket CORS policy ``` http://* GET POST PUT DELETE HEAD 180 ETag * ```

[rnicholus]

I believe the problem is with the thumbnailUrl returned from your server. You're returning what AWS calls a "path-style" URL to the object in your bucket: https://s3.amazonaws.com/bucktname/xxx.jpeg?AWSAccessKeyId.... Since your bucket is located outside of the US East region, you must use the region-specific domain name. In your case, that would be https://s3-eu-west-2.amazonaws.com/bucktname/xxx.jpeg?AWSAccessKeyId.... This can be avoided by instead using a "virtual hosted–style" path to your bucket, such as https://bucketname.s3.amazonaws.com/xxx.jpeg?AWSAccessKeyId....

[jaredcassidy]

I think in endpoint.php the S3 Client library provides the url format, would you recommend I try modify that?

endpoint.php line 270-277

// Provide a time-bombed public link to the file.
function getTempLink($bucket, $key) {
    $client = getS3Client();
    $url = "{$bucket}/{$key}";
    $request = $client->get($url);

    return $client->createPresignedUrl($request, '+15 minutes');
}

[rnicholus]

That's a temporary solution. But I think the best solution is to change the logic in the Fine Uploader PHP S3 repository to always return a thumbnailUrl using the virtual hosted-style path format. I can't think of a good reason to use path-style urls, unless the bucket name is not DNS-compatible (which isn't supported outside of US East anyway).

[jaredcassidy]

Thanks for the help

For testing I have decided to upload my files with public-read permissions so that I don't need to have a time-bombed/signed url for the file (changing the signing code to return the correct url scheme is out of my ability at the moment)

in endpoint.php function verifyFileInS3 I have changed:

$link = getTempLink($bucket, $key);

to

$link = "http://" . $bucket . ".s3.amazonaws.com" . "/" . $key;