dependency failures using punycode, getRandomBase64, and smithy in react-native

warrenronsiek commented 2 months ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

When calling cognito operations (and a variety of others), the aws-sdk has a series of dependencies that are undocumented and not included in the base package.

This is closely related to https://github.com/aws/aws-sdk-js-v3/issues/4877, https://github.com/aws/aws-sdk-js-v3/issues/5736, and https://github.com/aws/aws-sdk-js-v3/issues/6013. However the workarounds suggested https://github.com/aws/aws-sdk-js-v3/issues/4877#issuecomment-1656007484, https://github.com/aws/aws-sdk-js-v3/issues/4877#issuecomment-1803353706, and https://github.com/aws/aws-sdk-js-v3/issues/6013#issuecomment-2067866695 are only part of the problem. I would expect the AWS SDK to work "out of the box" and not have to do a variety of tricks with my JS bundler, dependency hacks, and installing unlisted dependencies.

SDK version number

3.567.0

Which JavaScript Runtime is this issue in?

React Native

Details of the browser/Node.js/ReactNative version

0.74.0

Reproduction Steps

// required imports from https://github.com/aws/aws-sdk-js-v3#getting-started
import 'react-native-get-random-values';
import 'react-native-url-polyfill/auto';
// import punycode here to make sure its installed and available to the program
import 'punycode';

import { userPoolId, userPoolClientID } from "./vars";
import {
  CognitoIdentityProviderClient,
  SignUpCommand,
  AdminConfirmSignUpCommand,
  InitiateAuthCommand,
} from "@aws-sdk/client-cognito-identity-provider";
import { CognitoIdentityClient } from "@aws-sdk/client-cognito-identity";
import { fromEnv } from "@aws-sdk/credential-providers";

const region = "us-west-2";
const standardClient = new CognitoIdentityProviderClient(
  {
    region,
  });

const signUp = (email, password) => {
  const params = {
    ClientId: userPoolClientID,
    Password: password,
    Username: email,
    UserAttributes: [
      {
        Name: "email",
        Value: email,
      }],
  };
  const command = new SignUpCommand(params);
  return standardClient.send(command);
};

// Exists for testing purposes.
const adminConfirmSignup = (email) => {
  const params = {
    UserPoolId: userPoolId,
    Username: email,
  };
  const adminClient = new CognitoIdentityProviderClient(
    {
      region,
      credentials: fromEnv(),
    },
  );
  const command = new AdminConfirmSignUpCommand(params);
  return adminClient.send(command);
};

const signIn = (email, password) => {
  const params = {
    AuthFlow: "USER_PASSWORD_AUTH",
    ClientId: userPoolClientID,
    AuthParameters: {
      USERNAME: email,
      PASSWORD: password,
    },
  };
  const command = new InitiateAuthCommand(params);
  return standardClient.send(command);
};

With these functions defined, the following should fail inside of a RN application:

const email = success+slksi92bob@simulator.amazonses.com;
const password = "P@33wordz!";
const signedUp = await signUp(email, password);
const confirmed = await adminConfirmSignup(email);
const signedIn = await signIn(email, password);

Note that this should work fine in e.g. a Jest testing environment. It just doesn't run in RN. This suggests a bundling/dependency issue.

Observed Behavior

Initially you cycle through a series of failures discussed in https://github.com/aws/aws-sdk-js-v3/issues/4877#issuecomment-1656007484. At the end, you get to Failed to construct URL with https://cognito-idp.us-west-2.amazonaws.com [TypeError: undefined is not an object (evaluating '_$$_REQUIRE(_dependencyMap[0], "punycode").ucs2.decode')] The first element in the stack refers to URLStateMachine which is in whatwg-url-without-unicode/lib/url-state-machine.js package which has a require("punycode") at the top, which seems like our culprit.

To bypass this, I hacked my node_modules/whatwg-url-without-unicode/node_modules/punycode/package.json such that the line "module": "punycode.es6.js" instead points to punycode.js. (In principle, this should be fixed by https://github.com/aws/aws-sdk-js-v3/issues/4877#issuecomment-1803353706, but for some black magic reason is not.)

Once I do this, I now get "Native module not found", "name": "Error", "stack": "getRandomBase64..." which I solved by npx pod-install (the documentation https://github.com/aws/aws-sdk-js-v3 says that they have to be imported, but doesn't mention needing to install pods or linking). Also, react-native-get-random-values does not have any bindings for windows, only android/ios. Turns out ios bindings work for macos, but android isn't sufficent. Hence, this is breaking if someone is using react-native-windows.

Now I get "Can't find variable: ReadableStream", "name": "ReferenceError", "stack": "isStreamingPayload" which seems to be the issue mentioned in https://github.com/aws/aws-sdk-js-v3/issues/5736. I solve this via stackoverflow and importing as follows:

import { ReadableStream } from 'web-streams-polyfill';
globalThis.ReadableStream = ReadableStream;

Finally, you get Can't find variable: TextDecoder as mentioned in https://github.com/aws/aws-sdk-js-v3/issues/4877#issuecomment-1803353706. Which is solved by adding the text-encoder-polyfill and import 'text-encoding-polyfill';.

Expected Behavior

AWS SDK should "just work" or be very close to it. This would imply the following:

aws-sdk dependencies (a.k.a. punycode, smithy) do not have bundle resolution issues
aws-sdk documentation indicates that simply importing react-native-get-random-values, should be sufficient. It doesn't mention installing pods, which is a requirement.
aws-sdk should include unlisted dependencies like web-streams-polyfill and text-encoder-polyfill or at least mention these things as independent react-native requirements in the same spot it mentions react-native-get-random-values in getting started.
aws-sdk should work for react-native-windows. This would involve either updating or not relying on react-native-get-random-values.

Possible Solution

No response

Additional Information/Context

No response

aBurmeseDev commented 1 month ago

Hi @warrenronsiek - thanks for reaching out, sharing feedback and workarounds.

I've tried to reproduce it and can confirm that some of the steps that you mentioned around dependencies are missing from our documentation. Feedback like this are crucial as we're always looking to improve our docs to be more clear, concise and accurate. I'll work on adding the missing pieces to our docs.

Thanks again, John

warrenronsiek commented 1 month ago

@aBurmeseDev thank you for your response! The documentation updates would help greatly. I also understand that react-native-windows compatibility is a niche issue and AWS might not be inclined to fix it. That said, this still leaves the bundle resolution issue, which is breaking and is not just a documentation update. Do you have an assessment on whether or not that will be fixed?

aws / aws-sdk-js-v3