Create operation not working consistently

[DeclanLacey]

Before opening, please confirm:

• [X] I have searched for duplicate or closed issues and discussions.
• [X] I have read the guide for submitting bug reports.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

React

Amplify APIs

Authentication, Storage

Amplify Version

v6

Amplify Categories

No response

Backend

Amplify Gen 2 (Preview)

Environment information

``` # Put output below this line System: OS: Windows 10 10.0.19045 CPU: (16) x64 AMD Ryzen 7 2700X Eight-Core Processor Memory: 5.96 GB / 15.95 GB Binaries: Node: 20.16.0 - C:\Program Files\nodejs\node.EXE npm: 10.8.1 - C:\Program Files\nodejs\npm.CMD Browsers: Edge: Chromium (128.0.2739.79) Internet Explorer: 11.0.19041.4355 npmPackages: %name%: 0.1.0 @aws-amplify/backend: ^1.2.1 => 1.2.1 @aws-amplify/backend-cli: ^1.2.6 => 1.2.6 @aws-amplify/ui-react: ^6.3.1 => 6.3.1 @aws-amplify/ui-react-internal: undefined () @aws-amplify/ui-react-server: undefined () @eslint/js: ^9.9.0 => 9.10.0 @types/aws-lambda: ^8.10.145 => 8.10.145 @types/react: ^18.3.3 => 18.3.5 @types/react-dom: ^18.3.0 => 18.3.0 @vitejs/plugin-react: ^4.3.1 => 4.3.1 aws-amplify: ^6.6.0 => 6.6.0 aws-amplify/adapter-core: undefined () aws-amplify/analytics: undefined () aws-amplify/analytics/kinesis: undefined () aws-amplify/analytics/kinesis-firehose: undefined () aws-amplify/analytics/personalize: undefined () aws-amplify/analytics/pinpoint: undefined () aws-amplify/api: undefined () aws-amplify/api/server: undefined () aws-amplify/auth: undefined () aws-amplify/auth/cognito: undefined () aws-amplify/auth/cognito/server: undefined () aws-amplify/auth/enable-oauth-listener: undefined () aws-amplify/auth/server: undefined () aws-amplify/data: undefined () aws-amplify/data/server: undefined () aws-amplify/datastore: undefined () aws-amplify/in-app-messaging: undefined () aws-amplify/in-app-messaging/pinpoint: undefined () aws-amplify/push-notifications: undefined () aws-amplify/push-notifications/pinpoint: undefined () aws-amplify/storage: undefined () aws-amplify/storage/s3: undefined () aws-amplify/storage/s3/server: undefined () aws-amplify/storage/server: undefined () aws-amplify/utils: undefined () aws-cdk: ^2.158.0 => 2.158.0 aws-cdk-lib: ^2.158.0 => 2.158.0 chartist: ^1.3.0 => 1.3.0 constructs: ^10.3.0 => 10.3.0 currency.js: ^2.0.4 => 2.0.4 esbuild: ^0.23.1 => 0.23.1 (0.21.5) eslint: ^9.9.0 => 9.10.0 eslint-plugin-react-hooks: ^5.1.0-rc.0 => 5.1.0-rc-fb9a90fa48-20240614 eslint-plugin-react-refresh: ^0.4.9 => 0.4.12 globals: ^15.9.0 => 15.9.0 (11.12.0, 14.0.0) react: ^18.3.1 => 18.3.1 react-dom: ^18.3.1 => 18.3.1 react-router-dom: ^6.26.2 => 6.26.2 tsx: ^4.19.1 => 4.19.1 typescript: ^5.6.2 => 5.6.2 (4.4.4, 4.9.5) typescript-eslint: ^8.0.1 => 8.5.0 vite: ^5.4.1 => 5.4.5 npmGlobalPackages: @angular/cli: 18.2.0 @babel/core: 7.24.9 npm-check: 6.0.1 npm: 10.8.2 ```

Describe the bug

On the click of a button, multiple functions are run. Each of these functions attempt to add data to the backend, for the current authenticated user to access. An example of one of those functions is below.

export const addTransactionData = async () => {
    for (let i = 0; i < initialData.transactions.length; i++) {
        try {
            client.models.Transaction.create(initialData.transactions[0])
        }catch (error) {
            console.log(error)
        }
    }
}

Currently the function is simply grabbing data from a json file and looping through it, and on each iteration calling client.models.Transaction.create(). The try block completes successfully, and in the network tab in google chrome the request receives a response of 200.

However, the issue is that despite there being no apparent errors, the data is only sporadically added. For example sometimes when the function is run (or any of the other functions adding other data from the same JSON file) no data is added at all, and other times only some of the data accessed during the loop is added. It was recommended to open an issue here as they believed it is possible that this is a bug with amplify.

I will also note that when running functions to get data from the backend, such as the one below. There are also no errors and the random data that was added is shown and displayed on the frontend.

export const getTransactions = async () => {
    const { data, errors } = await client.models.Transaction.list();
    if (errors) {
        console.log(errors);
    } else {
        return data;
    }
};

Expected behavior

The expected behavior would be that when client.models..create() is run, it would consistently add the data passed into the method, and if there is an error then an error would be thrown.

Reproduction steps

1. Run the function containing the code attempting to add a new record using client.models..create()

2. Look in the network tab of the browser and see that each of the calls receives a 200 code

3. Run the function containing the code that retrieves all of the data for the model that you attempted to create a new record for earlier.

4. See that the record may or may not have been added, and if multiple records are added, that 0 or more were added, in no particular order.

Code Snippet

// The code below is all of the code of the functions that are not running correctly
// Put your code below this line.
import { generateClient } from "aws-amplify/data";
import { type Schema } from "@/../../amplify/data/resource";
import initialData from "../../src/data/data.json"

const client = generateClient<Schema>({
    authMode: "userPool",
});

export const addBalanceData = async () => {
    client.models.Balance.create(initialData.balance)
}

export const addPotData = async () => {
    for (let i = 0; i < initialData.pots.length; i++) {
        client.models.Pot.create(initialData.pots[i])
    }
}

export const addTransactionData = async () => {
    for (let i = 0; i < initialData.transactions.length; i++) {
        try {
            client.models.Transaction.create(initialData.transactions[0])
        }catch (error) {
            console.log(error)
        }
    }
}

export const addBudgetData = async () => {
    for (let i = 0; i < initialData.budgets.length; i++) {
        client.models.Budget.create(initialData.budgets[i])
    }
}

export const getBalances = async () => {
    const { data, errors } = await client.models.Balance.list();
    if (errors) {
      console.log(errors);
    } else {
      return data;
    }
};

export const getBudgets = async () => {
    const { data, errors } = await client.models.Budget.list();
    if (errors) {
        console.log(errors);
    } else {
        return data;
    }
};

export const getTransactions = async () => {
    const { data, errors } = await client.models.Transaction.list();
    if (errors) {
        console.log(errors);
    } else {
        return data;
    }
};

export const getPots = async () => {
    const { data, errors } = await client.models.Pot.list();
    if (errors) {
        console.log(errors);
    } else {
        return data;
    }
};

//// The code below is the code from the resource file in the amplify/data directory

import { type ClientSchema, a, defineData } from '@aws-amplify/backend';
import { postConfirmation } from '../auth/post-confirmation/resource';

const schema = a.schema({
  UserProfile: a
      .model({
        email: a.string(),
        profileOwner: a.string(),
      })
      .authorization((allow) => [
        allow.ownerDefinedIn("profileOwner"),
      ]),
  Balance: a
    .model({
      current: a.float(),
      income: a.float(),
      expenses: a.float()
    })
    .authorization((allow) => [
      allow.ownerDefinedIn("profileOwner"),
    ]),

  Transaction: a
    .model({
      avatar: a.string(),
      name: a.string(),
      category: a.string(),
      date: a.string(),
      amount: a.float(),
      recurring: a.boolean()
    })
    .authorization((allow) => [
      allow.ownerDefinedIn("profileOwner"),
    ]),

  Budget: a 
    .model({
      category: a.string(),
      maximum: a.float(),
      theme: a.string()
    })
    .authorization((allow) => [
      allow.ownerDefinedIn("profileOwner"),
    ]),

  Pot: a
    .model({
      name: a.string(),
      target: a.float(),
      total: a.float(),
      theme: a.string()
    })
    .authorization((allow) => [
      allow.ownerDefinedIn("profileOwner"),
    ]),
})
.authorization(allow => [allow.resource(postConfirmation)]);

export type Schema = ClientSchema<typeof schema>;

export const data = defineData({
  schema,
  authorizationModes: {
    defaultAuthorizationMode: 'iam',
  },
});

Log output

``` // Put your logs below this line ```

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

[chrisbonifacio]

Hi @DeclanLacey 👋 thanks for raising this issue. Just out of curiosity, the for loop code snippet you shared defines an async function but does not await any of the requests. I'm not certain but maybe there is some rate limiting going on. Have you tried awaiting each request to allow them to process sequentially?

Do you notice any difference if you created an array of promises and used Promise.all()?

Otherwise, would you be able to provide a sample of the JSON data that we can use to reproduce the issue?

[DeclanLacey]

Hello @chrisbonifacio I had tried using a simple await before each of the requests in the for loop before, as well as creating an array of promises and then using Promise.all(). However I tried them again to ensure I did not miss anything, and the result was unfortunately the same. It does seem to be getting the data just fine from the JSON file, because when I run

Promise.all(promiseArray).then((values) => { console.log(values) })

It does log all of the data that it looped over from the JSON file, however again it will only add a random number of these to the database. I have attached the JSON file that I have been using to this comment. Thank you for your help, and please let me know if you have any other questions!

data.json

[chrisbonifacio]

@DeclanLacey I wonder if maybe you are getting throttled.

Do you think you might be hitting any of the service quotas mentioned in the AppSync docs? https://docs.aws.amazon.com/general/latest/gr/appsync.html

Otherwise, maybe consider implementing a custom mutation that performs a batch insert of records instead. https://docs.amplify.aws/react/build-a-backend/data/connect-to-existing-data-sources/connect-external-ddb-table/#batchputitem

I think this would allow you to process inserts in batches of 25 records at a time into the database. It will also return the keys of records that could not be processed successfully.

[DeclanLacey]

@chrisbonifacio

I shouldn't be hitting any of the service quotas, as I am not dealing with very large amounts of data. And the inconsistency persists even when I am attempting to insert a single record.

As for the custom mutation, I had tried something similar that I found in the docs last week but the link you sent seems slightly different so I am going to give that a try tomorrow. However, Im not sure if it will help, as even when attempting to insert a single record it does not work consistently (in fact it seems to almost never work when trying to insert just a single record)

[chrisbonifacio]

@DeclanLacey this might be a silly question but is this code snippet from the issue description still being used in your app or has it been changed?

export const addTransactionData = async () => {
    for (let i = 0; i < initialData.transactions.length; i++) {
        try {
            client.models.Transaction.create(initialData.transactions[0])
        }catch (error) {
            console.log(error)
        }
    }
}

The way this snippet is written, only the first transaction will be inserted into the database multiple times because it's not using the index as it increments.

[DeclanLacey]

@chrisbonifacio Sorry, that was my mistake! I forgot to change it back to i instead of 0 when I submitted the issue. I changed it to 0 to test if it possibly had an issue with some of the JSON data for some reason. However, the problem persisted even when trying to insert the same data 19 times in the loop. But I do have it back to i now to insert the current index which is also still having the same issue.

[chrisbonifacio]

Thanks for confirming! I will attempt to reproduce and report back with any findings.

[chrisbonifacio]

Hi @DeclanLacey I am unable to reproduce the behavior where some records are not persisted to the server. Would you be able to provide a small sample app that reproduces the behavior? There might be some framework specific code or missing business logic that might be causing the issue here.

[DeclanLacey]

@chrisbonifacio Currently the code for the project having this issue is in a public repository. It currently is not a terribly large application, would it be possible for you to clone the current project and see if the same issue is reproduced on your machine? Otherwise, I can try to create a very simple amplify project and see if I can reproduce the issue with a different project.

The current repo: https://github.com/DeclanLacey/Personal-Finance

[chrisbonifacio]

Hi @DeclanLacey apologies for the delay. Thank you for providing a repo that I was able to use locally to try and reproduce the issue.

Unfortunately, I was not able to reproduce the described behavior. I added a little bit a of logging to compare the number of transactions in initialData, ran the Add transaction data process and observed that the number of transactions created matched the number of transactions in the initalData set.

[DeclanLacey]

@chrisbonifacio Thank you for attempting to reproduce the issue! I did want to clarify however, the additional logging that you added, was this within the addTransactionData function? I had also created additional logging before and had noticed that when logging each of the created objects, I got the same result, this is in line with the network tab showing a response of 200 for each of the transactions were created.

However, when then calling getTransactions, the data is not all there (or sometimes none of it is). Were you able to successfully then fetch the data using client.models.Transactions.list() and received back all of the transactions that were just added? (This is what I have not been able to do successfully)

[DeclanLacey]

@chrisbonifacio I had the pleasure of meeting @mtliendo today at a local tech conference. We took a look at my code and were able to reproduce the issues that I have been having. It seemed as though the data was being added into the DynamoDB tables, and actually was being received by the front end when a call was made to do so, but the data was being lost when it was returned by the function and then attempted to be shown on the front end.

However, after deleting the data that was already in the DynamoDB tables, it now seems to be working! However this a bit confusing to me, as I was having this issue even when there was very little data in the tables. It is possible that the stored data was an issue and along the way, I changed something else related to the issue. Although, it is still curious to me as to why the issue Michael and I reproduced was happening, where it seemed as though only selective data was being shown from the data stored in the DynamoDB tables.

[chrisbonifacio]

@DeclanLacey Awesome! Glad you were able to get unblocked. That is a bit strange that existing data in the tables might've caused an issue. The only thing I can think of is some kind of ConditionalCheckFailedException from DynamoDB but that usually happens in scenarios like an item with a primary key value (id) being put into the table where an item already exists with the same id.

Looking at the schema from the repo you shared, it might also be possible that data was being created with an owner field value that was not accessible by the current user. Transaction has a owner field of profileOwner but the field doesn't exist on the schema. I'm not 100% sure if that caused the issue but if it happens again, it may be worth verifying that the data being queried has a profileOwner value that matches the user's attributes in the format <sub>::<username>.

Transaction: a
    .model({
      avatar: a.string().required(),
      name: a.string().required(),
      category: a.string().required(),
      date: a.string().required(),
      amount: a.float().required(),
      recurring: a.boolean().required()
    })
    .authorization((allow) => [
      allow.ownerDefinedIn("profileOwner"),
    ]),