TypeError: readableStream.tee is not a function (it is undefined)

umairm1alik commented 1 month ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

I am trying to implement chunk approach to upload files in react native , if the file size is less then 5 mb its working fine other wise i am getting this error

SDK version number

"@aws-sdk/client-s3": "^3.592.0", "@aws-sdk/lib-storage": "^3.620.0",

Which JavaScript Runtime is this issue in?

React Native

Details of the browser/Node.js/ReactNative version

"react-native": "0.74.1",

Reproduction Steps

index.js

// import 'react-native-url-polyfill/auto';
// import 'react-native-get-random-values';
// import {ReadableStream} from 'web-streams-polyfill'
// globalThis.ReadableStream = ReadableStream;

import 'react-native-polyfill-globals/auto'
import {AppRegistry} from 'react-native';
import App from './App';
import {name as appName} from './app.json';

AppRegistry.registerComponent(appName, () => App);

import {PutObjectCommand, S3Client, S3ClientConfig} from '@aws-sdk/client-s3';
import Config from 'react-native-config';
import RNFS from 'react-native-fs';
import {Buffer} from 'buffer';
import {handleS3Upload} from '../store/slices/homeSlice/homeSlice';
import {Audio, getFileSize, Image, Video} from 'react-native-compressor';
import {Upload} from '@aws-sdk/lib-storage';

const compressFileBasedOnType = async (
  filePath: string,
  contentType: string,
): Promise<string> => {
  let compressedFilePath = filePath;

  if (contentType.startsWith('image/')) {
    compressedFilePath = await Image.compress(filePath, {
      compressionMethod: 'auto',
    });
  } else if (contentType.startsWith('video/')) {
    compressedFilePath = await Video.compress(filePath, {
      compressionMethod: 'auto',
    });
  } else if (contentType.startsWith('audio/')) {
    compressedFilePath = await Audio.compress(filePath, {
      quality: 'medium',
    });
  } else {
    compressedFilePath = filePath;
  }

  return compressedFilePath;
};

interface UploadParams {
  filePath: string;
  fileName: string;
  contentType: string;
  dispatch?: (action: any) => void;
}

const createS3Client = (): S3Client => {
  const accessKeyId = Config.ACCESS_KEY_S3;
  const secretAccessKey = Config.SECRET_KEY_S3;

  if (!accessKeyId || !secretAccessKey) {
    throw new Error('AWS S3 credentials are not set');
  }

  const s3Config: S3ClientConfig = {
    region: Config.S3_BUCKET_REGION,
    credentials: {
      accessKeyId,
      secretAccessKey,
    },
  };

  return new S3Client(s3Config);
};

const uploadFileToS3 = async ({
  filePath,
  fileName,
  contentType,
  dispatch,
}: UploadParams): Promise<string> => {
  try {
    if (dispatch) {
      dispatch(handleS3Upload(true));
    }

    const compressedFilePath = await compressFileBasedOnType(
      filePath,
      contentType,
    );

    const fileContent = await RNFS.readFile(compressedFilePath, 'base64');
    const fileSize = await getFileSize(compressedFilePath);
    console.log('FileSize ====>', fileSize);
    const fileBuffer = Buffer.from(fileContent, 'base64');

    const s3Client = createS3Client();

    // const command = new PutObjectCommand({

    const upload = new Upload({
      client: s3Client,
      params: {
        Bucket: Config.BUCKET_NAME_S3,
        Key: fileName,
        Body: fileBuffer,
        ContentType: contentType,
      },

      partSize: 5 * 1024 * 1024, // 5 MB
      queueSize: 4, // 4 concurrent uploads
      leavePartsOnError: false, // Cleanup parts on error
    });

    upload.on('httpUploadProgress', progress => {
      console.log('Upload Progress:', progress);
      // You can also update your app's UI with the progress information here
    });

    await upload.done();
    if (dispatch) {
      dispatch(handleS3Upload(false));
    }

    const fileUrl = `https://${Config.BUCKET_NAME_S3}.s3.${Config.S3_BUCKET_REGION}.amazonaws.com/${fileName}`;
    console.log('fileUrl ======>', fileUrl);
    return fileUrl;
  } catch (error) {
    console.log('Error', error?.message, error);
    if (dispatch) {
      dispatch(handleS3Upload(false));
    }
    throw error;
  }
};

export default uploadFileToS3;

Observed Behavior

TypeError: readableStream.tee is not a function (it is undefined)

Expected Behavior

It should have to upload images , videos in chunk if its more then 5mb in size

Possible Solution

No response

Additional Information/Context

No response

kuhe commented 1 month ago

Are you able to use a version of the ReadableStream polyfill that implements ReadableStream.prototype.tee()?

umairm1alik commented 1 month ago

Before implementing Chunk with '@aws-sdk/lib-storage' I was uploading media files with aws-sdk/client-s3 and it was working. Even now if the file size is smaller than 5 mb everything working properly. But giving error for above 5 mb

trivikr commented 1 month ago

As @kuhe asked, are you using a version of the ReadableStream polyfill that implements ReadableStream.prototype.tee()?

The JS SDK v3 required ReadableStream polyfill to work in react-native environment. We added a recommendation to add web-streams-polyfill in the documentation in https://github.com/aws/aws-sdk-js-v3/pull/6302

LeandroTamola commented 1 month ago

I'm having the same issue with version "@aws-sdk/client-s3": "^3.620.1" and I'm already importing these in the index.js (root)

import "react-native-get-random-values";
import "react-native-url-polyfill/auto";
import "web-streams-polyfill/dist/polyfill";

aBurmeseDev commented 1 month ago

Hey all - thanks everyone for reporting. I was able to reproduce the error reported. For further analysis and potential resolution, I've prepared a couple test cases along with the versions of the relevant dependencies used during our testing.

Test Case: ListObjects and PutObject with S3
- install S3 client along with third-party libraries that provide polyfills for Web Streams API in React Native
```
"@aws-sdk/client-s3": "^3.620.1",
"web-streams-polyfill": "^4.0.0",
"react-native-get-random-values": "^1.11.0",
"react-native-url-polyfill": "^2.0.0"
```
- import libraries
```
import 'react-native-get-random-values';
import 'react-native-url-polyfill/auto';
import { ReadableStream } from "web-streams-polyfill";
import "web-streams-polyfill/dist/polyfill";
globalThis.ReadableStream = ReadableStream;
```
- Error thrown on both ListObjects and PutObject: TypeError: readableStream.tee is not a function (it is undefined)
- Tested different major versions of web-streams-polyfill and error remains.
- Downgraded to "@aws-sdk/client-s3": "^3.574.0" and the call with successful.

npm i @aws-sdk/client-s3@3.574.0

Test Case: ListTables and PutItem with DynamoDB

"@aws-sdk/client-dynamodb": "^3.620.1",
"web-streams-polyfill": "^4.0.0",
"react-native-get-random-values": "^1.11.0",
"react-native-url-polyfill": "^2.0.0",

No error thrown for both operations.

If you're encountering the reported error with S3, please downgrade to @aws-sdk/client-s3@3.574.0 as a workaround, while we further look into it.

umairm1alik commented 1 month ago

@aBurmeseDev thanks for the fast reply and solution. Its working with @aws-sdk/client-s3@3.574.0

trivikr commented 1 month ago

Error thrown on both ListObjects and PutObject: TypeError: readableStream.tee is not a function (it is undefined)

If you're encountering the reported error with S3, please downgrade to @aws-sdk/client-s3@3.574.0 as a workaround

I'm not able to reproduce the issue with standard code samples in https://github.com/aws-samples/aws-sdk-js-tests for S3 ListObjects with v3.575.0

The ListObjects command is successful on iOS as shown in screenshot below

Screenshot

I also downloaded local bundle, and verified that SDK version is v3.575.0. Bundle for reference: index.bundle.js.txt

Patch file for repro: test-client-s3-list-objects.patch To repro, just follow the README of https://github.com/aws-samples/aws-sdk-js-tests, and apply the patch using git apply.

On repro, the local bundle will be available on this URL.

umairm1alik commented 1 month ago

If you're encountering the reported error with S3, please downgrade to @aws-sdk/client-s3@3.574.0 as a workaround, while we further look into it.

@aBurmeseDev I am still facing issues with this workaround sometimes it works fine and sometimes it still gives this error [TypeError: readableStream.tee is not a function (it is undefined)]

AhmedChebbiDRW commented 1 month ago

Thank you , Its working with @aws-sdk/client-s3@3.574.0

aBurmeseDev commented 1 month ago

@umairm1alik - quick update here.

Initially, I was able to reproduce the error when using React Native with Expo. However, after starting a new React Native project without Expo, following the steps and code examples from the aws-sdk-js-tests, the issue was resolved.

This suggests that the root cause of the problem might be related to Expo itself or the way it integrates with the AWS SDK for JavaScript v3. To further investigate and identify the culprit, it would be helpful if you could share detailed steps to reproduce your workflow, including the dependencies you've installed.

Providing a step-by-step guide on how you set up your React Native project with Expo, the specific version of Expo you're using, and any other relevant dependencies or configurations, would assist in narrowing down the potential cause of the issue and determining if it's indeed related to Expo or if there are other factors at play.

therbta commented 1 month ago

Hey all - thanks everyone for reporting. I was able to reproduce the error reported. For further analysis and potential resolution, I've prepared a couple test cases along with the versions of the relevant dependencies used during our testing.

Test Case: ListObjects and PutObject with S3

install S3 client along with third-party libraries that provide polyfills for Web Streams API in React Native
"@aws-sdk/client-s3": "^3.620.1",
"web-streams-polyfill": "^4.0.0",
"react-native-get-random-values": "^1.11.0",
"react-native-url-polyfill": "^2.0.0"
import libraries
import 'react-native-get-random-values';
import 'react-native-url-polyfill/auto';
import { ReadableStream } from "web-streams-polyfill";
import "web-streams-polyfill/dist/polyfill";
globalThis.ReadableStream = ReadableStream;
Error thrown on both ListObjects and PutObject: TypeError: readableStream.tee is not a function (it is undefined)

Tested different major versions of web-streams-polyfill and error remains.

Downgraded to "@aws-sdk/client-s3": "^3.574.0" and the call with successful.

npm i @aws-sdk/client-s3@3.574.0

Test Case: ListTables and PutItem with DynamoDB
"@aws-sdk/client-dynamodb": "^3.620.1",
"web-streams-polyfill": "^4.0.0",
"react-native-get-random-values": "^1.11.0",
"react-native-url-polyfill": "^2.0.0",
No error thrown for both operations.

If you're encountering the reported error with S3, please downgrade to @aws-sdk/client-s3@3.574.0 as a workaround, while we further look into it.

Finally found a solution! Thank you so much!

kornicameister commented 1 month ago

I am seeing this in Lambda NodeJS 20 runtime. I would like to avoid bundling AWS SDK manually though, any suggestions here?

What's funny is that file seems to be uploaded (yes, PutObject here) without a problem.

umairm1alik commented 1 month ago

@aBurmeseDev I am using react native CLI not expo and my version is "react-native": "0.74.1" at first, my issue was resolved with @aws-sdk/client-s3@3.574.0 but still, sometimes I was getting this issue then I updated to @aws-sdk/client-s3@3.575.0 and my issue was resolved.

aBurmeseDev commented 3 weeks ago

Thanks for the update @umairm1alik!

It sounds like the error is intermittent for you, occurring with both @aws-sdk/client-s3@3.575.0 and @aws-sdk/client-s3@3.574.0. This suggests that the issue may be caused by something outside of the AWS SDK itself.

As I mentioned earlier in my comment, I was able to reproduce the error when using Expo, but when I started a new plain React Native project without Expo, it worked as expected.

To further validate my theory, I attempted to reproduce the issue using the react-native reproducer provided by the React Native community. However, I was unable to reproduce the error following the instructions in the README for setting up and using the provided steps to start the Metro Server and the application.

Can you try reproducing the issue using the same reproducer and steps? Here's patch file for repro ref: test-react-native-s3-list-objects.patch

aws / aws-sdk-js-v3