aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.42k stars 2.12k forks source link

React Native (IOS) not able to upload large files #10686

Open rcp522 opened 1 year ago

rcp522 commented 1 year ago

Before opening, please confirm:

JavaScript Framework

React Native

Amplify APIs

Authentication, Storage

Amplify Categories

auth, storage

Environment information

``` # Put output below this line # React Native project built with Expo System: OS: macOS 12.6 CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz Memory: 187.13 MB / 16.00 GB Shell: 5.8.1 - /bin/zsh Binaries: Node: 16.7.0 - ~/.nvm/versions/node/v16.7.0/bin/node Yarn: 1.22.19 - ~/.nvm/versions/node/v16.7.0/bin/yarn npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm Browsers: Chrome: 107.0.5304.110 Firefox: 102.5.0 Safari: 16.0 npmPackages: @babel/core: ^7.12.9 => 7.19.3 @react-native-async-storage/async-storage: ^1.17.10 => 1.17.10 @react-native-community/netinfo: ^9.3.5 => 9.3.5 @react-native-picker/picker: ^2.4.8 => 2.4.8 HelloWorld: 0.0.1 amazon-cognito-identity-js: ^5.2.11 => 5.2.11 aws-amplify: ^4.3.39 => 4.3.39 aws-amplify-react-native: ^6.0.5 => 6.0.5 expo: ~46.0.16 => 46.0.16 expo-document-picker: ~10.3.0 => 10.3.0 expo-image-picker: ^13.3.1 => 13.3.1 expo-status-bar: ~1.4.0 => 1.4.0 hermes-inspector-msggen: 1.0.0 react: 18.0.0 => 18.0.0 react-native: 0.69.6 => 0.69.6 npmGlobalPackages: @aws-amplify/cli: 9.2.1 @expo/ngrok: 4.1.0 @vue/cli: 5.0.4 aws-cdk: 2.33.0 expo-cli: 6.0.6 gatsby-cli: 4.19.0 npm: 7.20.3 sails: 1.5.2 typescript: 4.5.2 wscat: 5.0.0 yarn: 1.22.19 # React Native project built with React Native CLI System: OS: macOS 12.6 CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz Memory: 215.54 MB / 16.00 GB Shell: 5.8.1 - /bin/zsh Binaries: Node: 16.7.0 - ~/.nvm/versions/node/v16.7.0/bin/node Yarn: 1.22.19 - ~/.nvm/versions/node/v16.7.0/bin/yarn npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm Browsers: Chrome: 107.0.5304.110 Firefox: 102.5.0 Safari: 16.0 npmPackages: @babel/core: ^7.12.9 => 7.20.2 @babel/runtime: ^7.12.5 => 7.20.1 @react-native-async-storage/async-storage: ^1.17.11 => 1.17.11 @react-native-community/eslint-config: ^2.0.0 => 2.0.0 @react-native-community/netinfo: ^9.3.6 => 9.3.6 @react-native-picker/picker: ^2.4.8 => 2.4.8 HelloWorld: 0.0.1 amazon-cognito-identity-js: ^5.2.12 => 5.2.12 aws-amplify: ^4.3.43 => 4.3.43 aws-amplify-react-native: ^6.0.8 => 6.0.8 babel-jest: ^26.6.3 => 26.6.3 eslint: ^7.32.0 => 7.32.0 hermes-inspector-msggen: 1.0.0 jest: ^26.6.3 => 26.6.3 metro-react-native-babel-preset: 0.72.3 => 0.72.3 react: 18.1.0 => 18.1.0 react-native: 0.70.5 => 0.70.5 react-native-document-picker: ^8.1.2 => 8.1.2 react-test-renderer: 18.1.0 => 18.1.0 npmGlobalPackages: @aws-amplify/cli: 9.2.1 @expo/ngrok: 4.1.0 @vue/cli: 5.0.4 aws-cdk: 2.33.0 expo-cli: 6.0.6 gatsby-cli: 4.19.0 npm: 7.20.3 sails: 1.5.2 typescript: 4.5.2 wscat: 5.0.0 yarn: 1.22.19 ```

Describe the bug

Attempting to upload large files (1/5/10 gig) either has a network timeout (with Expo) or seems to run out of memory when created using React Native CLI. The app constantly crashes at 600mb ish for me. The files also take longer to upload (if you would compare it to a React application).

The cause for this looks like the way the file is uploaded. In order to pass the file from React Native to Amplify, the file needs to be read as a blob const blobFiless = await response.blob(); which loads the whole file in memory, so, the whole file is kept in memory while it is being uploaded. This increases memory usage especially for larger files and this sometimes causes the app to crash. In pure React, the file is streamed which is faster and does not produce issues.

This issue seems related to this one for android https://github.com/aws-amplify/amplify-js/issues/9736

Expected behavior

Ability to upload large files and as fast as the React implementation.

Reproduction steps

For Expo CLI:

  1. Create an Create React Native application using Expo CLI
  2. Init Amplify, add default auth, add storage
  3. Use the code snippet below for expo for a quick application that allows you to pick and upload files. It should output progress in terminal and on page.
  4. Download a sample 1gig or large file to the device

For React Native CLI:

  1. Create an React Native application using React native cli
  2. Init Amplify, add default auth, add storage
  3. Use the code snippet below for react native for a quick application that allows you to pick and upload files. It should output progress in terminal and on page.
  4. Download a sample 1gig or large file to the device

Code Snippet

Expo CLI

// App.js
import { StatusBar } from "expo-status-bar";
import { Button, StyleSheet, Text, View } from "react-native";
import { Amplify, Storage, Auth } from "aws-amplify";
import awsconfig from "./src/aws-exports";
import { withAuthenticator } from "aws-amplify-react-native";
import * as DocumentPicker from "expo-document-picker";
import { useState } from "react";
Amplify.configure({
  ...awsconfig,
  Analytics: {
    disabled: true,
  },
});

const App = ({ signOut, user }) => {
  const [image, setImage] = useState(null);
  const [progress, setProgress] = useState("0");

  const pickImage = async () => {
    console.log("PICKKING");

    const response = await DocumentPicker.getDocumentAsync({});

    if (!response.cancelled) {
      setImage(response.uri);
    }
  };

  const uploadImage = async () => {
    try {
      const response = await fetch(image);
      const blobFiless = await response.blob();
      // console.time("Upload");
      const res = await Storage.put("testfile", blobFiless, {
        progressCallback(progress) {
          console.log(`Uploaded: ${progress.loaded}/${progress.total}`);
          setProgress(`Uploaded: ${progress.loaded}/${progress.total}`);
        },
        contentType: "video/mp4",
        level: "private",
      });

      console.log("UPLOADED");
      // console.timeEnd("Upload");
    } catch (err) {
      console.log(err);
    }
  };

  return (
    <View style={styles.container}>
      <Text>Upload Image</Text>
      <Button
        title="Sign out"
        onPress={() => {
          Auth.signOut();
        }}
      ></Button>
      <Button title="Pick an image" onPress={pickImage}></Button>
      {image && (
        <>
          <Button title="Upload image" onPress={uploadImage} />
        </>
      )}
      <Text>{progress}</Text>
      <StatusBar style="auto" />
    </View>
  );
};

export default withAuthenticator(App);

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: "#fff",
    alignItems: "center",
    justifyContent: "center",
  },
});

Package.json

{
  "name": "test_upload",
  "version": "1.0.0",
  "main": "node_modules/expo/AppEntry.js",
  "scripts": {
    "start": "expo start",
    "android": "expo start --android",
    "ios": "expo start --ios",
    "web": "expo start --web"
  },
  "dependencies": {
    "@react-native-async-storage/async-storage": "^1.17.10",
    "@react-native-community/netinfo": "^9.3.5",
    "@react-native-picker/picker": "^2.4.8",
    "amazon-cognito-identity-js": "^5.2.11",
    "aws-amplify": "^4.3.39",
    "aws-amplify-react-native": "^6.0.5",
    "expo": "~46.0.16",
    "expo-image-picker": "^13.3.1",
    "expo-status-bar": "~1.4.0",
    "react": "18.0.0",
    "react-native": "0.69.6",
    "expo-document-picker": "~10.3.0"
  },
  "devDependencies": {
    "@babel/core": "^7.12.9"
  },
  "private": true
}

React Native CLI

/**
 * Sample React Native App
 * https://github.com/facebook/react-native
 *
 * @format
 * @flow strict-local
 */

// App.js
import React from 'react';
import type {Node} from 'react';
import {
  SafeAreaView,
  ScrollView,
  StatusBar,
  StyleSheet,
  Text,
  useColorScheme,
  View,
} from 'react-native';

import {Button} from 'react-native';
import {Amplify, Storage, Auth} from 'aws-amplify';
import {useState} from 'react';
import {withAuthenticator} from 'aws-amplify-react-native';

import {
  Colors,
  DebugInstructions,
  Header,
  LearnMoreLinks,
  ReloadInstructions,
} from 'react-native/Libraries/NewAppScreen';
import DocumentPicker, {types} from 'react-native-document-picker';

/* $FlowFixMe[missing-local-annot] The type annotation(s) required by Flow's
 * LTI update could not be added via codemod */
const Section = ({children, title}): Node => {
  const isDarkMode = useColorScheme() === 'dark';
  return (
    <View style={styles.sectionContainer}>
      <Text
        style={[
          styles.sectionTitle,
          {
            color: isDarkMode ? Colors.white : Colors.black,
          },
        ]}>
        {title}
      </Text>
      <Text
        style={[
          styles.sectionDescription,
          {
            color: isDarkMode ? Colors.light : Colors.dark,
          },
        ]}>
        {children}
      </Text>
    </View>
  );
};

const App: () => Node = () => {
  const isDarkMode = useColorScheme() === 'dark';

  const backgroundStyle = {
    backgroundColor: isDarkMode ? Colors.darker : Colors.lighter,
  };

  const [image, setImage] = useState(null);
  const [progress, setProgress] = useState('0');

  const pickImage = async () => {
    console.log('PICKKING');

    // const response = await DocumentPicker.getDocumentAsync({});
    // const response = await DocumentPicker.pickSingle({
    //   presentationStyle: 'fullScreen',
    //   copyTo: 'cachesDirectory',
    // });

    try {
      const res = await DocumentPicker.pick({type: [types.allFiles]});
      if (res.length > 0) {
        const selecFile = res[0];
        console.log(selecFile.uri);
        setImage(selecFile.uri);
      }
    } catch (err) {
      if (DocumentPicker.isCancel(err)) {
        // User cancelled the picker, exit any dialogs or menus and move on
      } else {
        throw err;
      }
    }

    // console.log(response);
    // setImage(res.uri);
  };

  const uploadImage = async () => {
    try {
      const response = await fetch(image);
      const blobFiless = await response.blob();
      // console.time("Upload");
      const res = await Storage.put('testfile', blobFiless, {
        progressCallback(progress) {
          console.log(`Uploaded: ${progress.loaded}/${progress.total}`);
          setProgress(`Uploaded: ${progress.loaded}/${progress.total}`);
        },
        contentType: 'video/mp4',
        level: 'private',
      });

      console.log('UPLOADED');
      // console.timeEnd("Upload");
    } catch (err) {
      console.log(err);
    }
  };

  return (
    <SafeAreaView style={backgroundStyle}>
      <StatusBar
        barStyle={isDarkMode ? 'light-content' : 'dark-content'}
        backgroundColor={backgroundStyle.backgroundColor}
      />
      <ScrollView
        contentInsetAdjustmentBehavior="automatic"
        style={backgroundStyle}>
        <Header />
        <View
          style={{
            backgroundColor: isDarkMode ? Colors.black : Colors.white,
          }}>
          <Text>Upload Image</Text>
          <Button
            title="Sign out"
            onPress={() => {
              Auth.signOut();
            }}></Button>
          <Button title="Pick an image" onPress={pickImage}></Button>
          {image && (
            <>
              <Button title="Upload image" onPress={uploadImage} />
            </>
          )}
          <Text>{progress}</Text>
        </View>
      </ScrollView>
    </SafeAreaView>
  );
};

const styles = StyleSheet.create({
  sectionContainer: {
    marginTop: 32,
    paddingHorizontal: 24,
  },
  sectionTitle: {
    fontSize: 24,
    fontWeight: '600',
  },
  sectionDescription: {
    marginTop: 8,
    fontSize: 18,
    fontWeight: '400',
  },
  highlight: {
    fontWeight: '700',
  },
});

export default withAuthenticator(App);

Package.json

{
  "name": "rn_file_upload",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "android": "react-native run-android",
    "ios": "react-native run-ios",
    "start": "react-native start",
    "test": "jest",
    "lint": "eslint ."
  },
  "dependencies": {
    "@react-native-async-storage/async-storage": "^1.17.11",
    "@react-native-community/netinfo": "^9.3.6",
    "@react-native-picker/picker": "^2.4.8",
    "amazon-cognito-identity-js": "^5.2.12",
    "aws-amplify": "^4.3.43",
    "aws-amplify-react-native": "^6.0.8",
    "react": "18.1.0",
    "react-native": "0.70.5",
    "react-native-document-picker": "^8.1.2"
  },
  "devDependencies": {
    "@babel/core": "^7.12.9",
    "@babel/runtime": "^7.12.5",
    "@react-native-community/eslint-config": "^2.0.0",
    "babel-jest": "^26.6.3",
    "eslint": "^7.32.0",
    "jest": "^26.6.3",
    "metro-react-native-babel-preset": "0.72.3",
    "react-test-renderer": "18.1.0"
  },
  "jest": {
    "preset": "react-native"
  }
}

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

Iphone14 Pro Max

Mobile Operating System

iOS 16.1

Mobile Browser

Stock (Safari)

Mobile Browser Version

Not sure, running in the Simulator

Additional information and screenshots

No response

ncarvajalc commented 1 year ago

Replicated the issue and both Expo and React Native CLI apps crashed when uploading 1GB file in iPhone 14 Pro. No crash in the iOS Simulator using Expo (iPhoneSE) but it did crash with React Native CLI using the same simulator when uploading 1GB file.