search

aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.43k stars 2.13k forks source link

read and write operations with Amplify datastore are too slow. #9368

Closed frankarnaud closed 2 years ago

frankarnaud commented 2 years ago

Before opening, please confirm:

JavaScript Framework

React Native

Amplify APIs

DataStore

Amplify Categories

api

Environment information

# Put output below this line System: OS: macOS 11.6 CPU: (4) x64 Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz Memory: 96.23 MB / 8.00 GB Shell: 5.8 - /bin/zsh Binaries: Node: 14.17.6 - /usr/local/bin/node Yarn: 1.22.11 - /usr/local/bin/yarn npm: 6.14.15 - /usr/local/bin/npm Browsers: Chrome: 96.0.4664.110 Safari: 15.0 npmPackages: @aws-amplify/datastore-storage-adapter: ^1.2.0 => 1.2.2 @azure/core-asynciterator-polyfill: ^1.0.0 => 1.0.0 @babel/core: ^7.12.9 => 7.15.8 (7.9.0) @babel/runtime: ^7.12.9 => 7.15.4 @expo/vector-icons: ^12.0.4 => 12.0.5 @react-native-async-storage/async-storage: ^1.15.9 => 1.15.9 @react-native-community/clipboard: ^1.5.1 => 1.5.1 @react-native-community/datetimepicker: ^3.4.3 => 3.5.2 @react-native-community/masked-view: ^0.1.10 => 0.1.11 @react-native-community/netinfo: ^6.0.2 => 6.0.2 @react-native-firebase/app: ^12.8.0 => 12.9.2 @react-native-firebase/crashlytics: ^12.8.0 => 12.9.2 @react-native-firebase/dynamic-links: ^12.8.0 => 12.9.3 @react-navigation/bottom-tabs: ^5.11.9 => 5.11.15 @react-navigation/material-bottom-tabs: ^6.0.7 => 6.0.9 @react-navigation/native: ^6.0.4 => 6.0.6 @react-navigation/native-stack: ^6.2.2 => 6.2.5 @react-navigation/stack: ^5.14.4 => 5.14.9 HelloWorld: 0.0.1 amazon-cognito-identity-js: ^5.2.0 => 5.2.0 (5.2.3) aws-amplify: ^4.3.8 => 4.3.8 aws-amplify-react-native: ^5.0.4 => 5.0.4 axios: ^0.23.0 => 0.23.0 (0.21.4) babel-jest: ~25.2.6 => 25.2.6 (26.6.3) babel-plugin-transform-remove-console: ^6.9.4 => 6.9.4 core-js: ^3.12.1 => 3.18.3 (1.2.7, 2.6.12) diff-match-patch: ^1.0.5 => 1.0.5 eslint: ^7.27.0 => 7.32.0 eslint-plugin-react: ^7.24.0 => 7.26.1 eslint-plugin-react-hooks: ^4.2.0 => 4.2.0 eslint-plugin-react-native: ^3.11.0 => 3.11.0 expo: ~41.0.0 => 41.0.1 expo-av: ^9.1.2 => 9.2.3 expo-document-picker: ~9.1.2 => 9.1.2 expo-file-system: ~11.0.2 => 11.0.2 expo-image-picker: ~10.1.3 => 10.1.4 expo-localization: ~10.1.0 => 10.1.0 expo-media-library: ^12.0.2 => 12.1.2 expo-print: ~10.1.0 => 10.1.0 expo-sharing: ^9.1.2 => 9.2.1 expo-status-bar: ~1.0.4 => 1.0.4 expo-updates: ~0.5.4 => 0.5.5 expo-web-browser: ~9.1.0 => 9.1.0 hermes-inspector-msggen: 1.0.0 jest: ~26.6.3 => 26.6.3 lottie-ios: ~3.1.8 => 3.1.9 lottie-react-native: 4.0.2 => 4.0.2 metro-react-native-babel-preset: ^0.66.0 => 0.66.2 (0.59.0) millify: ^4.0.0 => 4.0.0 react: 17.0.2 => 17.0.2 react-dom: ^17.0.2 => 17.0.2 react-fast-compare: ^3.2.0 => 3.2.0 (2.0.4) react-native: ~0.65.1 => 0.65.1 react-native-app-link: ^1.0.1 => 1.0.1 react-native-background-fetch: ^4.0.4 => 4.0.4 react-native-bootsplash: ^3.2.2 => 3.2.6 react-native-cn-quill: ^0.7.0 => 0.7.14 react-native-code-push: ^7.0.4 => 7.0.4 react-native-codegen: ^0.0.7 => 0.0.7 react-native-color-picker: ^0.6.0 => 0.6.0 react-native-device-info: ^8.4.7 => 8.4.7 react-native-draggable-flatlist: ^2.6.1 => 2.6.2 react-native-elements: ^3.4.2 => 3.4.2 react-native-fast-image: ^8.3.4 => 8.5.11 react-native-flipper: ^0.111.0 => 0.111.0 react-native-gesture-handler: ~1.10.2 => 1.10.3 react-native-get-random-values: ^1.7.0 => 1.7.0 react-native-material-menu: ^1.2.0 => 1.2.0 react-native-modal: ^11.10.0 => 11.10.0 react-native-modal-datetime-picker: ^11.0.0 => 11.1.0 react-native-pager-view: ^5.4.4 => 5.4.7 react-native-paper: ^4.9.2 => 4.9.2 react-native-phone-number-input: ^2.1.0 => 2.1.0 react-native-popup-menu: ^0.15.10 => 0.15.11 react-native-reanimated: ^2.2.2 => 2.2.3 react-native-render-html: ^6.0.5 => 6.1.0 react-native-safe-area-context: ^3.3.2 => 3.3.2 (3.2.0) react-native-screens: ^3.9.0 => 3.9.0 react-native-select-dropdown: ^1.4.0 => 1.4.0 react-native-share: ^5.2.2 => 5.3.0 react-native-simple-toast: ^1.1.3 => 1.1.3 react-native-skeleton-placeholder: ^3.0.4 => 3.0.4 react-native-sqlite-storage: ^6.0.1 => 6.0.1 react-native-svg: ^12.1.1-0 => 12.1.1 react-native-svg-transformer: ^0.14.3 => 0.14.3 react-native-tab-view: ^3.1.1 => 3.1.1 react-native-table-component: ^1.2.1 => 1.2.1 react-native-tags: ^2.2.0 => 2.2.1 react-native-unimodules: ~0.13.3 => 0.13.3 react-native-url-polyfill: ^1.3.0 => 1.3.0 react-native-vector-icons: ^8.1.0 => 8.1.0 react-native-web: ~0.13.12 => 0.13.18 react-native-webview: ^11.6.5 => 11.14.1 react-redux: ^7.2.5 => 7.2.5 react-test-renderer: ~17.0.2 => 17.0.2 redux: ^4.1.1 => 4.1.1 redux-devtools-extension: ^2.13.9 => 2.13.9 redux-flipper: ^2.0.1 => 2.0.1 redux-persist: ^6.0.0 => 6.0.0 redux-persist/integration/react: undefined () redux-thunk: ^2.3.0 => 2.3.0 reselect: ^4.0.0 => 4.0.0 (3.0.1) shortid: ^2.2.16 => 2.2.16 uuid: ^8.3.2 => 8.3.2 (3.4.0, 3.3.2, 7.0.2, 7.0.3) victory-native: ^35.3.2 => 35.5.5 xmldom: ^0.5.0 => 0.5.0 npmGlobalPackages: @aws-amplify/cli: 7.6.3 appcenter-cli: 2.10.6 expo-cli: 4.12.0 npm: 6.14.15 yarn: 1.22.11

Describe the bug

The read and write operations with datastore are too slow on android. Indeed to update an element we first do a query on id then a copyOf. In dev on a Redmi note 10 with 6GB ram the query takes about 8min which is much too long. When you make several successive requests, the time varies. We can do 65s, 161, 3min in the tests I performed. In production our users complain that successive operations can do 30mins. On IOS the problem does not arise whether it is with the emulator or an iphone. We are using @ aws-amplify / datastore-storage-adapter as mentioned in the doc.

Expected behavior

We want operations to be almost instantaneous since datastore is offline-first.

Reproduction steps

npx react-native init AmplifyDataStoreRN cd AmplifyDataStoreRN npx amplify-app@latest npm install aws-amplify @aws-amplify/datastore-storage-adapter react-native-sqlite-storage aws-amplify-react-native amazon-cognito-identity-js @react-native-community/netinfo @react-native-async-storage/async-storage @react-native-picker/picker

import { DataStore } from 'aws-amplify';
import { SQLiteAdapter } from '@aws-amplify/datastore-storage-adapter';

DataStore.configure({
  storageAdapter: SQLiteAdapter,
  syncPageSize: 1000,
  maxRecordsToSync: 10000,
});
type Article
  @model
  @key(name: "byCollection", fields: ["collectionID"])
  @auth(
    rules: [
      {
        allow: owner
        ownerField: "belongTo"
        identityClaim: "custom:managerName"
      }
    ]
  ) {
  id: ID!
...
  name: String
  priceOut: Float
  priceIn: Float
  belongTo: String!
  collectionID:ID!
...
}
try {
const price = 10
const start = Date.now()

// items already exist in the store.
      const oldArticle = await DataStore.query(Article, articleID); // get old one
      console.log('end at here',Date.now() - start)
        const newArticle = await DataStore.save(
        Article.copyOf(oldArticle, (updated) => {
          updated.priceIn = price; // new price saved
        })
      );

     console.log('new update here', Date.now() - start)
} catch (error) {
      console.log("update item error", error);
 }
//  Also this operation take time.
const start = Date.now()
        const newArticle = await DataStore.save(
       new Article({
      name:'name',
      priceIn:100,
      priceOut:200
})
      );
        console.log('end at here',Date.now() - start)
} catch (error) {
      console.log("create item error", error);
 }

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

Xiaomi Redmi note 10s

Mobile Operating System

Android 11

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

chrisbonifacio commented 2 years ago

Hi @frankarnaud 👋 Thank you for raising this issue. I reached out to the DataStore team and they noticed that you are using the SQLite adapter in an Expo project. Unfortunately, that adapter is not currently compatible with Expo and meant to be used with React Native CLI projects.

You will have to uninstall datastore-storage-adapter and remove SQLite Adapter from the DataStore configuration so that DataStore uses the default adapter, AsyncStorage.

frankarnaud commented 2 years ago

Hi @frankarnaud 👋 Thank you for raising this issue. I reached out to the DataStore team and they noticed that you are using the SQLite adapter in an Expo project. Unfortunately, that adapter is not currently compatible with Expo and meant to be used with React Native CLI projects.

You will have to uninstall datastore-storage-adapter and remove SQLite Adapter from the DataStore configuration so that DataStore uses the default adapter, AsyncStorage.

Thank you for being so responsive. Indeed we use the expo in our application but in the bare workflow mode which would be similar to react-native cli or expo eject. As mentioned above we have noticed this problem on android. Most of our operations take a very long time.

frankarnaud commented 2 years ago

We were already using datastore with AsyncStorage. But we encountered the mentioned issue #8405 which led us to use the recommendation from the official documentation.

chintannp commented 1 year ago

Hi @frankarnaud , We release ExpoSQLiteAdapter, designed to be used with Expo apps. It results in much improved performance for expo apps. You can checkout our documentation here.