search

aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.44k stars 2.13k forks source link

DataStoreStateError: Tried to execute `DataStore.query()` while DataStore was "Stopping". #10612

Closed youssef-almardini closed 1 year ago

youssef-almardini commented 2 years ago

Before opening, please confirm:

JavaScript Framework

React Native

Amplify APIs

DataStore

Amplify Categories

api

Environment information

``` # Put output below this line System: OS: macOS 12.2.1 CPU: (8) x64 Apple M1 Memory: 23.84 MB / 16.00 GB Shell: 5.8 - /bin/zsh Binaries: Node: 14.17.6 - ~/.nvm/versions/node/v14.17.6/bin/node Yarn: 1.22.19 - /opt/homebrew/bin/yarn npm: 8.12.2 - ~/.nvm/versions/node/v14.17.6/bin/npm Watchman: 2022.06.06.00 - /opt/homebrew/bin/watchman Browsers: Chrome: 98.0.4758.109 Safari: 15.3 npmPackages: @algolia/client-search: ^4.14.2 => 4.14.2 @algolia/transporter: ^4.14.2 => 4.14.2 @aws-amplify/cli-extensibility-helper: ^2.3.33 => 2.3.33 @aws-amplify/core: ^4.7.2 => 4.7.2 @aws-amplify/datastore: ^3.12.8 => 3.12.8 @babel/core: ^7.17.9 => 7.18.13 @babel/preset-env: ^7.16.11 => 7.18.10 @babel/preset-typescript: ^7.16.7 => 7.18.6 @react-native-async-storage/async-storage: ^1.17.4 => 1.17.10 @react-native-community/netinfo: ^9.3.0 => 9.3.0 @rollup/plugin-alias: ^3.1.9 => 3.1.9 @rollup/plugin-babel: ^5.3.1 => 5.3.1 @rollup/plugin-commonjs: ^21.0.3 => 21.1.0 @rollup/plugin-json: ^4.1.0 => 4.1.0 @rollup/plugin-multi-entry: ^4.1.0 => 4.1.0 @rollup/plugin-node-resolve: ^13.1.3 => 13.3.0 @rollup/plugin-typescript: ^8.3.1 => 8.4.0 @types/amplify: ^1.1.25 => 1.1.25 @types/jest: ^27.4.1 => 27.5.2 @types/jest-when: ^3.5.2 => 3.5.2 @types/node: ^17.0.30 => 17.0.45 (16.11.56) @typescript-eslint/eslint-plugin: ^5.35.1 => 5.36.1 @typescript-eslint/parser: ^5.18.0 => 5.34.0 aws-amplify: ^4.3.33 => 4.3.33 aws-amplify-react-native: ^6.0.5 => 6.0.5 aws-sdk: ^2.1142.0 => 2.1201.0 aws-sdk-mock: ^5.7.0 => 5.7.0 babel-jest: ^28.0.3 => 28.1.3 (27.5.1) babel-plugin-module-resolver: ^4.1.0 => 4.1.0 base64-js: ^1.5.1 => 1.5.1 deep-equal: ^2.0.5 => 2.0.5 eslint: ^8.12.0 => 8.22.0 eslint-config-prettier: ^8.5.0 => 8.5.0 eslint-import-resolver-alias: ^1.1.2 => 1.1.2 eslint-plugin-jsdoc: ^39.2.9 => 39.3.6 eslint-plugin-react: ^7.29.4 => 7.30.1 eslint-plugin-sort-exports: ^0.6.0 => 0.6.0 esm: ^3.2.25 => 3.2.25 fetch-mock: ^9.11.0 => 9.11.0 isomorphic-unfetch: ^3.1.0 => 3.1.0 jest: ^27.5.1 => 27.5.1 jest-junit: ^13.2.0 => 13.2.0 jest-when: ^3.5.1 => 3.5.1 jsdoc: ^3.6.10 => 3.6.11 mustache: ^4.2.0 => 4.2.0 nodemon: ^2.0.16 => 2.0.19 prettier: ^2.7.1 => 2.7.1 rimraf: ^3.0.2 => 3.0.2 (2.6.3, 2.7.1, 2.2.8) rollup: ^2.70.1 => 2.79.1 rollup-plugin-copy: ^3.4.0 => 3.4.0 rollup-plugin-dts: ^4.2.2 => 4.2.2 rollup-plugin-flat-dts: ^1.7.0 => 1.7.0 rollup-plugin-sourcemaps: ^0.6.3 => 0.6.3 rollup-plugin-terser: ^7.0.2 => 7.0.2 rollup-plugin-ts: ^3.0.2 => 3.0.2 ts-node: ^10.7.0 => 10.9.1 tsconfig-paths: ^3.14.1 => 3.14.1 tslib: ^2.3.1 => 2.4.0 (1.14.1) typedoc: ^0.22.15 => 0.22.18 typedoc-theme-hierarchy: ^1.1.1 => 1.3.8 typescript: ^4.6.3 => 4.7.4 (4.4.4) uuid: ^8.3.2 => 8.3.2 (3.4.0, 3.3.2, 8.0.0) npmGlobalPackages: @aws-amplify/cli: 9.2.1 @financiallease/react-native-amplify: 7.1.0 aws-amplify: 4.3.28 expo-cli: 5.4.9 html-to-xlsx: 2.2.1 nodemon: 2.0.19 npm: 8.12.2 react-devtools: 4.24.7 serve: 14.0.1 test-types: 1.0.1-0 ```

Describe the bug

DataStoreStateError: Tried to execute DataStore.query() while DataStore was "Stopping". This can only be done while DataStore is "Started" or "Stopped". To remedy: Ensure all calls to stop() and clear() have completed first. If this is not poss...

Hi, I have an amplify project to host backend for react-native mobile app. Suddenly it stopped to work. Any call to the data store will receive the message i mentioned above.

Our work plan is to provide amplify backend as an NPM package. This package is used from app developer to make calls to AWS-Amplify. Since 6 months everything works fine until 4 days go. We didn't had breaking changes or something. Just added a few lambda function which i don't think it could cause this issue.

We are starting datastore when app is open. DataStore.start() and clear it on SignIng event and SignOut event. We didn't make any change in this flow since a long time (4 months).

We tried to reproduce the issue locally but it is not possible for somehow. Since we are using typescript to write our NPM package. We test functionality locally with node or tsx where everything works, then we publish it and downloaded in another project react-native where everything works good. Once it is deployed, no action from DataStore could be executed. The app crashed and Sentry show us the message above. And again that start to happen 4 days ago until now.

Expected behavior

To be able to execute DataStore manipulation functionality (query, save, delete... etc) / Get connection with data store.

Reproduction steps

Whenever we call datastore functions in the react-native app. It will show this message:

DataStoreStateError: Tried to execute DataStore.query() while DataStore was "Stopping". This can only be done while DataStore is "Started" or "Stopped". To remedy: Ensure all calls to stop() and clear() have completed first. If this is not poss...

Code Snippet

// Put your code below this line.
    // where i clear data store and start it, just using auth events 
    // Clear the local datastore when signing out.
    // As advised in: https://docs.amplify.aws/lib/datastore/sync/q/platform/js/#clear-local-data
    EventHandler.OnSignOut = async () => {
        if (Application.hasBooted()) {
            await Application.clear();
        }
    };

    // Clear the local datastore when signing in.
    // As advised in: https://docs.amplify.aws/lib/datastore/sync/q/platform/js/#clear-local-data
    EventHandler.OnSignIn = async () => {
        if (Application.hasBooted()) {
            await Application.clear();
        }
    };

    // Boot the application as soon as the DataSync is completed
    // Check if it has not been booted already first
    EventHandler.OnDataSynced = async () => {
        if (!Application.hasBooted()) {
            await Application.boot();
        }
    };

    // Where i do datastore call
    /**
 * @name SaveAccount
 * @description: Save or update an account.
 *              If the account already exists in the datastore based
 *              on its id, it is updates. Otherwise a
 * @type {Function}
 * @param {AccountDetails} accountDetails
 * @param {AccountAttributes} params
 * @returns {Promise<AccountDetails>}
 */
const SaveAccount = async (
    accountDetails: AccountDetails,
    params?: AccountAttributes,
): Promise<AccountDetails> => {
    const current = await GetAccountDetails(accountDetails.id);
    if (current instanceof AccountDetails) {
        return await DataStore.save(
            AccountDetails.copyOf(
                current,
                updated => {
                    if (params) {
                        for (const key in params) {
                            updated[key] = params[key];
                        }
                    }
                },
            ),
        );
    }

    accountDetails = accountDetails instanceof AccountDetails
        ? accountDetails
        : new AccountDetails(accountDetails);

    return await DataStore.save(accountDetails);
};

Log output

``` // Put your logs below this line **DataStoreStateError: Tried to execute `DataStore.query()` while DataStore was "Stopping". This can only be done while DataStore is "Started" or "Stopped". To remedy: Ensure all calls to `stop()` and `clear()` have completed first. If this is not poss...** ```

aws-exports.js

/ eslint-disable / // WARNING: DO NOT EDIT. This file is automatically generated by AWS Amplify. It will be overwritten.

const awsmobile = { "aws_project_region": "", "aws_cognito_identity_pool_id": "***", "aws_cognito_region": "", "aws_user_pools_id": "", "aws_user_pools_web_client_id": "***", "oauth": {}, "aws_cognito_username_attributes": [ "EMAIL" ], "aws_cognito_social_providers": [], "aws_cognito_signup_attributes": [], "aws_cognito_mfa_configuration": "OFF", "aws_cognito_mfa_types": [ "SMS" ], "aws_cognito_password_protection_settings": { "passwordPolicyMinLength": 8, "passwordPolicyCharacters": [ "REQUIRES_LOWERCASE", "REQUIRES_NUMBERS", "REQUIRES_UPPERCASE" ] }, "aws_cognito_verification_mechanisms": [ "EMAIL" ], "aws_appsync_graphqlEndpoint": "", "aws_appsync_region": "", "aws_appsync_authenticationType": "API_KEY", "aws_appsync_apiKey": "", "aws_user_files_s3_bucket": "", "aws_user_files_s3_bucket_region": "" };

export default awsmobile;

Manual configuration

No response

Additional configuration

Package.json { "name": "@financiallease/react-native-amplify", "version": "7.1.0", "description": "", "author": "itsupport@financiallease.nl", "main": "dist/financiallease.js", "module": "dist/financiallease.js", "browser": "dist/financiallease.js", "typings": "dist/financiallease.d.ts", "types": "dist/financiallease.d.ts", "license": "LGPL", "scripts": { "BUNDLING": null, "build:clean": "rimraf dist/", "build": "npm run build:clean && rollup -c", "bundle-local": "npm run build && npm pack && mv -v financiallease-react-native-amplify-*.tgz /usr/local/npm/@financiallease/react-native-amplify.tgz", "LINTING": null, "autoformat": "npm run lint-typescript -- --fix && npm run lint-nodejs -- --fix", "lint-nodejs": "eslint --config amplify/.eslintrc.js 'amplify/backend/function//index.js'", "lint-typescript": "eslint --config .eslintrc.js '{src,test}//*.ts'", "lint": "npm run lint-typescript && npm run lint-nodejs", "coverage": "jest -c jest.config.ts --collectCoverage --coverageDirectory=\"./coverage\" --ci --reporters=default --reporters=jest-junit --watchAll=false", "test": "jest -c jest.config.ts", "DOC GENERATION": null, "docs:generate": "npm run build:clean && sed '/[[TOC]]/d' README.md > README.sanitized.md && typedoc --readme README.sanitized.md --entryPoints src --entryPointStrategy expand --out docs --theme hierarchy --name \"React Native Amplify - docs\" --includeVersion", "docs:serve": "node -r esm --inspect docker/server.js", "AMPLIFY BACKEND": null, "amplify-modelgen": "node amplify/scripts/amplify-modelgen.js", "amplify-push": "node amplify/scripts/amplify-push.js", "scan": "npm run build && npm run lint && npm run test && npm run docs:generate", "upgrade-amplify-deps": "npx npm-check-updates -i '/(@?aws-amplify|@react-native-community/netinfo)/' && npm update" }, "publishConfig": { "@financiallease:registry": "https://gitlab.com/api/v4/projects/35071033/packages/npm/" }, "dependencies": { "@algolia/client-search": "^4.14.2", "@algolia/transporter": "^4.14.2", "@aws-amplify/core": "^4.7.2", "@aws-amplify/datastore": "^3.12.8", "@react-native-async-storage/async-storage": "^1.17.4", "@react-native-community/netinfo": "^9.3.0", "@types/amplify": "^1.1.25", "aws-amplify": "^4.3.33", "aws-amplify-react-native": "^6.0.5", "aws-sdk": "^2.1142.0", "deep-equal": "^2.0.5" }, "devDependencies": { "@aws-amplify/cli-extensibility-helper": "^2.3.33", "@babel/core": "^7.17.9", "@babel/preset-env": "^7.16.11", "@babel/preset-typescript": "^7.16.7", "@rollup/plugin-alias": "^3.1.9", "@rollup/plugin-babel": "^5.3.1", "@rollup/plugin-commonjs": "^21.0.3", "@rollup/plugin-json": "^4.1.0", "@rollup/plugin-multi-entry": "^4.1.0", "@rollup/plugin-node-resolve": "^13.1.3", "@rollup/plugin-typescript": "^8.3.1", "@types/jest": "^27.4.1", "@types/jest-when": "^3.5.2", "@types/node": "^17.0.30", "@typescript-eslint/eslint-plugin": "^5.35.1", "@typescript-eslint/parser": "^5.18.0", "aws-sdk-mock": "^5.7.0", "babel-jest": "^28.0.3", "babel-plugin-module-resolver": "^4.1.0", "base64-js": "^1.5.1", "eslint": "^8.12.0", "eslint-config-prettier": "^8.5.0", "eslint-import-resolver-alias": "^1.1.2", "eslint-plugin-jsdoc": "^39.2.9", "eslint-plugin-react": "^7.29.4", "eslint-plugin-sort-exports": "^0.6.0", "esm": "^3.2.25", "fetch-mock": "^9.11.0", "isomorphic-unfetch": "^3.1.0", "jest": "^27.5.1", "jest-junit": "^13.2.0", "jest-when": "^3.5.1", "jsdoc": "^3.6.10", "mustache": "^4.2.0", "nodemon": "^2.0.16", "prettier": "^2.7.1", "rimraf": "^3.0.2", "rollup": "^2.70.1", "rollup-plugin-copy": "^3.4.0", "rollup-plugin-dts": "^4.2.2", "rollup-plugin-flat-dts": "^1.7.0", "rollup-plugin-sourcemaps": "^0.6.3", "rollup-plugin-terser": "^7.0.2", "rollup-plugin-ts": "^3.0.2", "ts-node": "^10.7.0", "tsconfig-paths": "^3.14.1", "tslib": "^2.3.1", "typedoc": "^0.22.15", "typedoc-theme-hierarchy": "^1.1.1", "typescript": "^4.6.3", "uuid": "^8.3.2" } }

Mobile Device

iphone

Mobile Operating System

ios

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

logs in production using sentry:

image

Screenshot 2022-11-04 at 10 14 13

chrisbonifacio commented 2 years ago

Hi @youssef-almardini šŸ‘‹ thanks for raising this issue. Just curious, could you share the lambda functions and/or let us know if you might be using DataStore at all in one of them?

If this is only happening in production, could you try pinning your @aws-amplify/datastore and aws-amplify versions to the versions you are running locally? (aws-amplify@4.3.33 & @aws-amplify/datastore@3.12.8) This error may be occurring because of changes we made recently as you mentioned, introduced in version 4.3.40 which might be the one being used in the production build.

We will continue to investigate this internally and report back with updates as they come. Thank you!

svidgen commented 2 years ago

The error you're seeing is a result of #10450 and is intended to replace much more pernicious error states. Specifically, we had cases where query/mutation operations could become interleaved with stopping or clearing operations in unpredictable and incorrect ways. In some observed cases, this resulted in visible database/transaction errors. But, the boundary was ill-defined enough that the result of operating against a "clearing" DataStore was actually just not predictable.

10450 corrects the boundaries around clear() and stop(), so that they do not resolve until clearing/stopping operations actually complete. It also essentially "locks" DataStore in the interim. Whereas the operations against DataStore in a clearing state would previously result in unpredictable, erroneous states or unexpected query results, infringements on that boundary are now predictable.

(We are exploring the option of queueing operations up behind this lock. But, there's some nuance here, both in terms of how internal operations interact with this mechanism and how customers could interact with DataStore and know which side that boundary an operation will be executed on. šŸ˜¬ For now, the solution is to make the boundary clear and visible so you have predictable contracts to program against.)

My suggestion for apps that must clear DataStore mid-session is to put DataStore calls into two categories:

  1. Those that should execute as soon as possible, even if DataStore is mid-clear(). E.g., A customer hits "sign out" and you immediately want to observeQuery to re-populate a table.
  2. Those that don't make sense to execute anymore if a clear() is in progress. E.g., if you're populating data and your customer hits "sign out", you probably want to abort those operations.

For those that should always execute, even if DataStore is clearing, I'd do something like this:

// at app start, set a flag to indicate the app is ready.
let isReady = Promise.resolve();

function signOut() {
  isReady = DataStore.clear();
}

async function getData() {
  await isReady;
  return DataStore.query(Model);
}

For operations that don't make sense if a clear has been invoked, either add another flag that be checked in-line, or try-catch and translate the given DataStoreStateError into something meaningful in your UI.

I might also add a third category:

  1. Operations that won't ever occur mid-clear. If an operation is tied to a button on a page that's already downstream of all the necessary await-ing, just make the call. šŸ¤ 

With all that said, if the errors only occurs in your test runner, this may be as simple has having the test runner await on the DataStore.clear() result before proceeding.

And, if you're already doing these things and still seeing the error, we'll need more detailed repro steps. Specifically, I'd want to start with:

  1. The precise sequence of operations you invoke against DataStore, ideally including a guess on timings
  2. All errors you see in the console
  3. All errors you see in graphql requests

I hope that helps!

chrisbonifacio commented 1 year ago

Hi šŸ‘‹ Closing this as we have not heard back from you. If you are still experiencing this issue and in need of assistance, please feel free to comment and provide us with any information previously requested by our team members so we can re-open this issue and be better able to assist you.

Thank you!

guy-a commented 1 year ago

@svidgen, this is generally working, but sometimes DataStore get stuck on "stopping"

Having something similar to this:

let isReady = Promise.resolve();

function stop() {
  isReady = DataStore.stop();
}

async function getData() {
  await isReady;
  return DataStore.query(Model);
}

When stuck on "stopping" isReady will never resolve and getData() will never return.

I deal with it with something similar to this, I rather deal with the error than to wait for data indefinitely:

function stop() {
  isReady = Promise.any([DataStore.stop(), sleep(1_000)])
}

My app can have frequent DataStore.stop() and start() and this is happening naturally form time to time. It difficult to reproduce it on purpose though. The only reliable way I found is to start DataStore fresh and while it's doing its first sync frequently stop(), start() and mess with the network condition.