aws-amplify / amplify-js

A declarative JavaScript library for application development using cloud services.
https://docs.amplify.aws/lib/q/platform/js
Apache License 2.0
9.42k stars 2.12k forks source link

DataStore observeQuery doesn't respond to deletions when a nested predicate is used. #11820

Open duckbytes opened 1 year ago

duckbytes commented 1 year ago

Before opening, please confirm:

JavaScript Framework

React Native

Amplify APIs

Authentication, GraphQL API, DataStore

Amplify Categories

auth, api

Environment information

``` # Put output below this line System: OS: Linux 6.4 Arch Linux CPU: (16) x64 AMD Ryzen 7 3700X 8-Core Processor Memory: 8.96 GB / 31.26 GB Container: Yes Shell: 5.9 - /bin/zsh Binaries: Node: 16.13.2 - ~/.nvm/versions/node/v16.13.2/bin/node Yarn: 1.22.19 - /usr/bin/yarn npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm npmPackages: @aws-amplify/datastore-storage-adapter: ^2.0.42 => 2.0.42 @aws-amplify/ui-react-native: ^1.2.20 => 1.2.20 @azure/core-asynciterator-polyfill: ^1.0.2 => 1.0.2 @babel/core: ^7.20.0 => 7.22.9 @expo/webpack-config: ^18.0.1 => 18.1.1 @react-native-async-storage/async-storage: 1.17.11 => 1.17.11 @react-native-community/netinfo: 9.3.7 => 9.3.7 @react-navigation/bottom-tabs: ^6.5.8 => 6.5.8 @react-navigation/material-bottom-tabs: ^6.2.16 => 6.2.16 @react-navigation/native-stack: ^6.9.13 => 6.9.13 @reduxjs/toolkit: ^1.9.5 => 1.9.5 @reduxjs/toolkit-query: 1.0.0 @reduxjs/toolkit-query-react: 1.0.0 @testing-library/jest-native: ^5.4.2 => 5.4.2 @testing-library/react-native: ^12.1.3 => 12.1.3 @types/jest: ^29.5.3 => 29.5.3 @types/react: ~18.0.14 => 18.0.38 HelloWorld: 0.0.1 amazon-cognito-identity-js: ^6.3.1 => 6.3.3 amazon-cognito-identity-js/internals: undefined () aws-amplify: ^5.3.8 => 5.3.8 core-js: ^3.31.1 => 3.31.1 experiments-app: 1.0.0 expo: ~48.0.18 => 48.0.20 expo-file-system: ~15.2.2 => 15.2.2 (15.3.0) expo-sqlite: ~11.1.1 => 11.1.1 expo-status-bar: ~1.4.4 => 1.4.4 faker: 5.5.3 => 5.5.3 jest-expo: ^49.0.0 => 49.0.0 mock-async-storage: ^2.2.0 => 2.2.0 moment: ^2.29.4 => 2.29.4 moment-timezone: ^0.5.43 => 0.5.43 react: 18.2.0 => 18.2.0 react-content-loader: ^6.2.1 => 6.2.1 react-content-loader/native: undefined () react-native: 0.71.8 => 0.71.8 react-native-dotenv: ^3.4.9 => 3.4.9 react-native-get-random-values: ~1.9.0 => 1.9.0 react-native-paper: ^5.9.1 => 5.9.1 react-native-paper-dates: ^0.18.12 => 0.18.12 react-native-safe-area-context: 4.5.0 => 4.5.0 react-native-screens: ~3.20.0 => 3.20.0 react-native-svg: ^13.5.0 => 13.5.0 react-native-testing-library-website: 0.0.0 react-native-url-polyfill: ^2.0.0 => 2.0.0 (1.3.0) react-native-web: ~0.18.10 => 0.18.12 react-navigation-example: 0.0.1 react-redux: ^8.1.1 => 8.1.1 redux-example: 0.0.1 redux-saga: ^1.2.3 => 1.2.3 redux-saga/effects: undefined () typescript: ^4.9.4 => 4.9.5 npmGlobalPackages: @aws-amplify/cli: 12.2.3 @bubblewrap/cli: 1.18.1 @ionic/cli: 6.20.3 amplify-cli: 1.0.0 cordova: 11.1.0 corepack: 0.10.0 create-react-native-app: 3.9.0 deadfile: 2.0.1 docsify-cli: 4.4.4 eas-cli: 3.18.3 graphql-language-service-cli: 3.3.16 jsonminify: 0.4.2 mjson: 0.4.2 native-run: 1.7.1 npm: 8.1.2 prebuild-install: 7.1.1 react_app: 0.1.0 react-dom: 17.0.2 react-js-to-ts: 1.4.0 react: 18.2.0 serve: 14.2.0 sharp-cli: 4.1.1 ts-node: 10.9.1 typescript: 4.8.2 uglify-js: 3.17.4 uglifyjs: 2.4.11 ```

Describe the bug

When using observeQuery with a nested field predicate, it does not respond to deletions. It seems to work fine with no predicates, or predicates on basic fields.

Expected behavior

I expect observeQuery to always respond to deletions.

Reproduction steps

Initialise a DataStore amplify project. Use optimistic concurrency and cognito for auth.

I use ExpoSQLiteAdapter for storageAdapter, but it seems to happen on default too.

Code Snippet

For example I have this model:

type TaskAssignee
@auth(rules: [
  {allow: private, operations: [read]},
  {allow: groups, groups: ["COORDINATOR", "RIDER", "ADMIN"], operations: [create, read, delete]},
])
@model {
  id: ID!
  tenantId: ID! @index(name: "byTenantId")
  role: Role!
  task: Task! @belongsTo
  assignee: User! @belongsTo
  archived: Int @default(value: "0") @index(name: "byArchived")
    @auth(rules: [
      {allow: private, operations: [read]},
      {allow: groups, groups: ["COORDINATOR", "RIDER", "ADMIN"], operations: [create, read, delete]},
    ])
}

and I want an observer to return all assignments of a particular role for the currently signed in user:


            DataStore.observeQuery(
                models.TaskAssignee,
                (a) =>
                    a.and((a) => [
                        a.role.eq(role),
                        a.assignee.id.eq(whoami?.id),
                    ])
            ).subscribe(async ({ items }) => {
                    // do stuff
           })

The above code does not respond to deletion events. However if I remove a.assignee.id.eq(whoami?.id), then it starts to respond again.

Log output

``` // Put your logs below this line ```
### aws-exports.js

const awsmobile = {
    "aws_project_region": "eu-west-1",
    "aws_appsync_graphqlEndpoint": "https://4uqedpdcfff7rlxih7mjsujd6m.appsync-api.eu-west-1.amazonaws.com/graphql",
    "aws_appsync_region": "eu-west-1",
    "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS",
    "aws_cognito_identity_pool_id": "eu-west-1:92e90580-beb0-4c4c-afb3-d37e7751b589",
    "aws_cognito_region": "eu-west-1",
    "aws_user_pools_id": "eu-west-1_iBDLs10x6",
    "aws_user_pools_web_client_id": "454f85i51rgptbe3n1di2oiq51",
    "oauth": {},
    "aws_cognito_username_attributes": [],
    "aws_cognito_social_providers": [],
    "aws_cognito_signup_attributes": [
        "EMAIL"
    ],
    "aws_cognito_mfa_configuration": "OFF",
    "aws_cognito_mfa_types": [
        "SMS"
    ],
    "aws_cognito_password_protection_settings": {
        "passwordPolicyMinLength": 8,
        "passwordPolicyCharacters": []
    },
    "aws_cognito_verification_mechanisms": [
        "EMAIL"
    ],
    "aws_user_files_s3_bucket": "platelet26fb7449fb884a3eb4c5fd7539c78dd3200245-deeev",
    "aws_user_files_s3_bucket_region": "eu-west-1",
    "geo": {
        "amazon_location_service": {
            "region": "eu-west-1",
            "search_indices": {
                "items": [
                    "plateletPlace-deeev"
                ],
                "default": "plateletPlace-deeev"
            }
        }
    }
};

Manual configuration

No response

Additional configuration

No response

Mobile Device

Android Emulator: Samsung_Galaxy_S8_API_30

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

{
    "name": "mobile",
    "version": "1.0.0",
    "main": "node_modules/expo/AppEntry.js",
    "scripts": {
        "start": "expo start",
        "android": "expo start --android",
        "ios": "expo start --ios",
        "web": "expo start --web",
        "test": "jest --watch --testTimeout=60000",
        "test:debug": "node --inspect-brk ./node_modules/jest/bin/jest.js --runInBand --testTimeout=100000000"
    },
    "dependencies": {
        "@aws-amplify/datastore-storage-adapter": "^2.0.42",
        "@aws-amplify/ui-react-native": "^1.2.20",
        "@azure/core-asynciterator-polyfill": "^1.0.2",
        "@expo/webpack-config": "^18.0.1",
        "@react-native-async-storage/async-storage": "1.17.11",
        "@react-native-community/netinfo": "9.3.7",
        "@react-navigation/bottom-tabs": "^6.5.8",
        "@react-navigation/material-bottom-tabs": "^6.2.16",
        "@react-navigation/native-stack": "^6.9.13",
        "@reduxjs/toolkit": "^1.9.5",
        "amazon-cognito-identity-js": "^6.3.1",
        "aws-amplify": "^5.3.8",
        "core-js": "^3.31.1",
        "expo": "~48.0.18",
        "expo-file-system": "~15.2.2",
        "expo-sqlite": "~11.1.1",
        "expo-status-bar": "~1.4.4",
        "faker": "5.5.3",
        "moment": "^2.29.4",
        "moment-timezone": "^0.5.43",
        "react": "18.2.0",
        "react-content-loader": "^6.2.1",
        "react-native": "0.71.8",
        "react-native-get-random-values": "~1.9.0",
        "react-native-paper": "^5.9.1",
        "react-native-paper-dates": "^0.18.12",
        "react-native-safe-area-context": "4.5.0",
        "react-native-screens": "~3.20.0",
        "react-native-svg": "^13.5.0",
        "react-native-url-polyfill": "^2.0.0",
        "react-native-web": "~0.18.10",
        "react-redux": "^8.1.1",
        "redux-saga": "^1.2.3"
    },
    "devDependencies": {
        "@babel/core": "^7.20.0",
        "@testing-library/jest-native": "^5.4.2",
        "@testing-library/react-native": "^12.1.3",
        "@types/jest": "^29.5.3",
        "@types/react": "~18.0.14",
        "jest-expo": "^49.0.0",
        "mock-async-storage": "^2.2.0",
        "react-native-dotenv": "^3.4.9",
        "typescript": "^4.9.4"
    },
    "private": true
}
cwomack commented 1 year ago

@duckbytes, thank you for opening this issue. I'll review this with our team and get a reply back soon!

svidgen commented 1 year ago

I'm having a hard time tracing through the case where a deletion can result in this behavior. I assume your code is landing us into this return line that filters out "irrelevant" operations. But, I'm not seeing how at the moment.

Can you show us the code use to create, update, and delete the record?

Does the behavior differ based on whether the userId provided to the condition is a string, null, or undefined?

duckbytes commented 1 year ago

Hi @svidgen here is the code that I use to create and delete records:

    async function addAssignee(user, role) {
        setIsPosting(true);
        try {
            const assignee = await DataStore.query(models.User, user.id);
            const task = await DataStore.query(models.Task, props.taskId);
            if (!assignee || !task)
                throw new Error(
                    `Can't find assignee or task: ${props.taskId}, userId: ${user.id}`
                );
            const result = await DataStore.save(
                new models.TaskAssignee({
                    assignee,
                    task,
                    role,
                    tenantId,
                })
            );
            if (role === userRoles.rider) {
                const status = await determineTaskStatus(
                    {
                        ...task,
                    },
                    [result]
                );
                await DataStore.save(
                    models.Task.copyOf(task, (updated) => {
                        updated.status = status;
                        if (assignee.riderResponsibility)
                            updated.riderResponsibility =
                                assignee.riderResponsibility;
                    })
                );
                if (
                    task.status === tasksStatus.new &&
                    status === tasksStatus.active
                ) {
                    dispatch(displayInfoNotification("Task moved to ACTIVE"));
                }
            }
            setState({ ...state, [result.id]: result });
            setIsPosting(false);
        } catch (error) {
            console.log(error);
            setIsPosting(false);
            dispatch(displayErrorNotification(errorMessage));
        }
    }

    async function deleteAssignment(assignmentId) {
        setIsDeleting(true);
        try {
            if (!assignmentId) throw new Error("Assignment ID not provided");
            const existingTask = await DataStore.query(
                models.Task,
                props.taskId
            );
            if (!existingTask) throw new Error("Task doesn't exist");
            const existingAssignment = await DataStore.query(
                models.TaskAssignee,
                assignmentId
            );
            if (existingAssignment) await DataStore.delete(existingAssignment);
            const status = await determineTaskStatus(
                existingTask,
                Object.values(_.omit(state, assignmentId)).filter(
                    (a) => a.role === userRoles.rider
                )
            );
            let riderResponsibility = existingTask.riderResponsibility;
            let isRiderUsingOwnVehicle = existingTask.isRiderUsingOwnVehicle;
            if (
                existingAssignment &&
                existingAssignment.role === userRoles.rider
            ) {
                const riders = Object.values(state)
                    .filter(
                        (a) =>
                            a.role === userRoles.rider && a.id !== assignmentId
                    )
                    .map((a) => a.assignee);
                if (riders.length > 0) {
                    const rider = riders[riders.length - 1];
                    if (rider && rider.riderResponsibility) {
                        riderResponsibility = rider.riderResponsibility;
                    }
                } else {
                    riderResponsibility = null;
                    isRiderUsingOwnVehicle = 0;
                }
            }
            await DataStore.save(
                models.Task.copyOf(existingTask, (updated) => {
                    updated.status = status;
                    updated.riderResponsibility = riderResponsibility;
                    updated.isRiderUsingOwnVehicle = isRiderUsingOwnVehicle;
                })
            );
            setState((prevState) => _.omit(prevState, assignmentId));
            setIsDeleting(false);
        } catch (error) {
            console.log(error);
            setIsDeleting(false);
            dispatch(displayErrorNotification(errorMessage));
        }
    }

There is some extra stuff in there that probably isn't relevant, but the part that queries and deletes the item, which aren't being responded to in the observeQuery is pretty simple:

            const existingAssignment = await DataStore.query(
                models.TaskAssignee,
                assignmentId
            );
            if (existingAssignment) await DataStore.delete(existingAssignment);

and this is the main code for creating a record:

            const assignee = await DataStore.query(models.User, user.id);
            const task = await DataStore.query(models.Task, props.taskId);
            if (!assignee || !task)
                throw new Error(
                    `Can't find assignee or task: ${props.taskId}, userId: ${user.id}`
                );
            const result = await DataStore.save(
                new models.TaskAssignee({
                    assignee,
                    task,
                    role,
                    tenantId,
                })
            );

I don't do any updates to these records, only create and delete them.

assignee is a non-nullable field. I'm not sure if passing in null or undefined makes a difference, it would be difficult to test it properly. I did add an extra line though if (!whoami) return; so that it doesn't set up the observer until whoami.id is definitely defined. The behaviour is the same though.