ReferenceError when trying to `PutObject` to S3 from React Native

g-popovic commented 1 week ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

When trying to execute a PutObject command to S3 with ContentEncoding: "gzip", we get the error ReferenceError: Property 'e' doesn't exist and the upload fails. This only happens on Android, iOS doesn't have this problem. This so far only happens when ContentEncoding is set to gzip - it works when it's undefined or set to base64 for example.

EDIT: after some further debugging into the lib code itself, we discovered that the actual error seems to be the following (tldr: SignatureDoesNotMatch):

{
  "parsedOutput": {
    "statusCode": 403,
    "reason": "",
    "headers": {
      "connection": "close",
      "content-type": "application/xml",
      "transfer-encoding": "chunked",
      ...
    },
    "body": {
      "Code": "SignatureDoesNotMatch",
      "Message": "The request signature we calculated does not match the signature you provided. Check your key and signing method.",
    ...

It looks like the library doesn't handle errors correctly, resulting in a very undescriptive and difficult to debug error. The SignatureDoesNotMatch error does seem to be potentially related to this already existing issue.

SDK version number

@aws-sdk/client-s3@3.598.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

react-native@0.72.12

Reproduction Steps

On an android device/emulator:

import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";

const s3Client = new S3Client({
    region: "us-east-1",
    credentials: {
        accessKeyId: "my_key_id",
        secretAccessKey: "my_secret",
    },
});

const command = new PutObjectCommand({
    Bucket: "my-bucket",
    Body: "H4sIAAAAAAAAA/NIzcnJVwjPL8pJUQQAoxwpHAwAAAA=", // gzipped "Hello World" string
    ContentType: "application/gzip", // also tried "text/csv" since that's the unzipped file extension
    ContentEncoding: "gzip" // removing this makes the error go away (however the resulting uploaded file is gibberish since it's still gzip-encoded)
});

await s3Client.send(command);

Observed Behavior

The file doesn't get uploaded to S3 and ReferenceError: Property 'e' doesn't exist gets thrown, with no further information or visible stack trace.

Expected Behavior

The gzipped file should get properly uploaded to S3, and/or the error should be more descriptive and provide more useful information.

Possible Solution

No response

Additional Information/Context

Note that the suggested polyfills have been imported:

import "react-native-get-random-values";
import "react-native-url-polyfill/auto";
import { ReadableStream } from "web-streams-polyfill";

globalThis.ReadableStream = ReadableStream;

I have also attempted to upload this gzipped file using Upload from @aws-sdk/lib-storage, however the same error was still being thrown.

NOTE: I have seen a similar-looking issue. However that is about a different service, so it seems it could be a separate issue.

aBurmeseDev commented 1 week ago

Hi @g-popovic - thanks for reaching out.

I recalled looking into the other issue mentioned, and interesting part is that this's only reproducible with Android? Do you mind adding this middleware stack to your code and share the requests from the output? That will give us some insight on the issue.

s3Client.middlewareStack.add(next => async (args) => {
 console.log(args.request)
 const response = await next(args);
 console.log(response);
 return next(args);
}, {step: 'finalizeRequest'})

For your reference, I found this old issue that might be worth taking a peek at. I would also suggest reaching out to React Native related repo that might get you unblocked if there are similar issues reported like the one linked above.

Best, John

g-popovic commented 1 week ago

and interesting part is that this's only reproducible with Android?

Thanks for the prompt response. Yes indeed, and on Android emulator. On iOS and iOS Simulator it runs without problem for me.

After adding the middleware you provided and trying to upload a CSV file, this is the log I saw in the console:

args.request:  {"method":"PUT","hostname":"relive-user-assets-tmp-uploads.s3.us-east-1.amazonaws.com","body":{"type":"Buffer","data":[31,139,8,0,0,0,0,0,0,0,157,208,75,10,195,32,16,128,225,125,207,50,145,241,85,77,207,209,85,55,65,140,37,130,137,37,154,66,122,250,166,33,165,43,17,186,117,102,62,126,12,113,130,96,50,184,224,32,251,209,193,211,133,104,125,94,187,52,198,152,7,24,226,236,95,113,202,38,116,198,218,101,54,118,133,222,207,206,102,191,157,250,212,61,204,146,92,127,106,40,163,164,149,66,81,13,92,17,46,148,86,10,144,32,2,67,38,26,60,55,140,94,81,95,24,94,184,184,1,37,140,130,252,140,247,157,187,9,201,253,16,33,15,164,101,69,68,215,16,197,191,72,177,68,240,10,162,183,151,29,209,88,44,17,181,18,189,157,30,72,177,68,86,75,228,241,39,154,22,75,100,181,228,63,36,207,139,123,3,244,66,34,252,42,2,0,0]},"protocol":"https:","path":"\/relive-tracker\/1718958033999-sdk_gphone64_arm64-unfiltered.csv","headers":{"content-type":"text\/csv","content-encoding":"gzip","content-length":"197","host":"relive-user-assets-tmp-uploads.s3.us-east-1.amazonaws.com","x-amz-user-agent":"aws-sdk-js\/3.550.0","user-agent":"aws-sdk-js\/3.550.0 ua\/2.0 os\/other lang\/js md\/rn api\/s3#3.550.0","amz-sdk-invocation-id":"569a821f-d9d2-46f6-92cd-1a2437adeb4a","amz-sdk-request":"attempt=1; max=3","x-amz-date":"20240621T082114Z","x-amz-content-sha256":"896e98a0cdd2490c35764b2641769d71fd05f08af7550da6f8597f655fd51e55","authorization":"AWS4-HMAC-SHA256 Credential=AKIAJOTARZ4XYG4ILIYQ\/20240621\/us-east-1\/s3\/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-user-agent, Signature=d966b8c7bfc3c4b0a770c2b613223ed7d9607d37101c34b43b2b677042e20618"},"query":{"x-id":"PutObject"}}

I notice that the response was not logged however, only the request, guess it must've errored out. Let me know if you need any other info from me that would help you with the debugging of this.

aws / aws-sdk-js-v3