How to use BatchPutItem to create several items across multiple tables ( I believe this is a task not documented very well which may help lots of people))

[totuslink]

Note: If your question is regarding the AWS Amplify Console service, please log it in the AWS Amplify Console repository

Which Category is your question related to? Appsync

Amplify CLI Version 4.52.0

What AWS Services are you utilizing? Appsync DynamoDB

Provide additional details e.g. code snippets

I want to do BatchPutItem request to multiple tables, at first I follow this guide Creating GraphQL Batch Operations for AWS Amplify with AppSync and Cognito Then try to get everything working from there. Here is my step.

Config

1. Create a Appsync Datasource point to but grant special IAM policy which have access to multiple tables, here is the policy screenshot

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchGetItem",
                "dynamodb:BatchWriteItem",
                "dynamodb:PutItem",
                "dynamodb:GetItem",
                "dynamodb:Scan",
                "dynamodb:Query",
                "dynamodb:UpdateItem"
            ],
            "Resource": [
                "arn:aws:dynamodb:<region>:<id>:table/Node-r3nu2ysuhza7lkwssv6vr75d6q-test",
                "arn:aws:dynamodb:<region>:<id>:table/Domain-r3nu2ysuhza7lkwssv6vr75d6q-test",
                "arn:aws:dynamodb:<region>:<id>:table/HologramNode-r3nu2ysuhza7lkwssv6vr75d6q-test",
                "arn:aws:dynamodb:<region>:<id>:table/UserNode-r3nu2ysuhza7lkwssv6vr75d6q-test",
                "arn:aws:dynamodb:<region>:<id>:table/Weight-r3nu2ysuhza7lkwssv6vr75d6q-test"
            ]
        }
    ]
}

2. Write custom mutation schema

# Custom Mutation

input CreateNodeInput {
  baseType: String!
  id: ID
  url: String
  title: String
  wordCounts: Int
  favicon: String
  domainID: ID
  weightID: ID
  owner: ID!
}

input CreateWeightInput {
  id: ID
  baseType: String!
  weightType: String!
  hologramNodeID: ID
  userNodeID: ID
  nodeID: ID
  hologramID: ID
  calculation: AWSJSON!
  value: Float!
  version: AWSJSON
  accessPolicy: accessPolicy!
  owner: ID!
}

input CreateHologramNodeInput {
  id: ID
  hologramID: ID
  nodeID: ID
  accessPolicy: accessPolicy!
  weightID: ID
  createdByID: ID
  owner: ID!
}

input CreateUserNodeInput {
  id: ID
  userID: ID
  nodeID: ID
  title: String
  note: String
  weightID: ID
  version: AWSJSON
  owner: ID!
}

input CreateDomainInput {
  baseType: String!
  hostName: String
  id: ID
  owner: ID!
}

type CreateNodeBuildingNecessaryItemsResult {
  domain: Domain,
  node: Node,
  weight: [Weight],
  hologramNode: HologramNode,
  userNode: UserNode 
}

type Mutation {
  CreateNodeBuildingNecessaryItems(
    domain: CreateDomainInput,
    node: CreateNodeInput,
    hologramNodeWeight: CreateWeightInput,
    userNodeWeight: CreateWeightInput,
    hologramNode: CreateHologramNodeInput
    userNode: CreateUserNodeInput
  ): CreateNodeBuildingNecessaryItemsResult
}

3. Here is my custom resolver: Mutation.CreateNodeBuildingNecessaryItems.req.vtl

## Mutation.CreateNodeBuildingNecessaryItems.req.vtl
## I remove auth part to better target problems

## [Start] Prepare DynamoDB BatchPutItem Request. **
#set( $locDomain = [] )
#set( $locNode = [] )
#set( $locHologramNode = [] )
#set( $locUserNode = [] )
#set( $locHologramNodeWeight = $ctx.args.hologramNodeWeight )
#set( $locUserNodeWeight = $ctx.args.userNodeWeight )

## [Start] Domain **
$util.qr($ctx.args.domain.put( "createdAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.domain.put( "updatedAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.domain.put( "__typename", "Domain" ))
$util.qr($ctx.args.domain.put( "id", $util.defaultIfNullOrBlank($locDomain.id, $util.autoId()) ))
$util.qr($locDomain.add($util.dynamodb.toMapValues($ctx.args.domain)))
## [End] Domain **

## [Start] Node **
$util.qr($ctx.args.node.put( "createdAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.node.put( "updatedAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.node.put( "__typename", "Node" ))
$util.qr($ctx.args.node.put( "id", $util.defaultIfNullOrBlank($locNode.id, $util.autoId()) ))
$util.qr($locNode.add($util.dynamodb.toMapValues($ctx.args.node)))
## [End] Node **

## [Start] HologramNode **
$util.qr($ctx.args.hologramNode.put( "createdAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.hologramNode.put( "updatedAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.hologramNode.put( "__typename", "HologramNode" ))
$util.qr($ctx.args.hologramNode.put( "id", $util.defaultIfNullOrBlank($locHologramNode.id, $util.autoId()) ))
$util.qr($locHologramNode.add($util.dynamodb.toMapValues($ctx.args.hologramNode)))
## [End] HologramNode **

## [Start] UserNode **
$util.qr($ctx.args.userNode.put( "createdAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.userNode.put( "updatedAt", $util.time.nowISO8601() ))
$util.qr($ctx.args.userNode.put( "__typename", "UserNode" ))
$util.qr($ctx.args.userNode.put( "id", $util.defaultIfNullOrBlank($locUserNode.id, $util.autoId()) ))
$util.qr($locUserNode.add($util.dynamodb.toMapValues($ctx.args.userNode)))
## [End] UserNode **

#set( $locWeight = [] )

## [Start] HologramNodeWeight **
$util.qr($locHologramNodeWeight.put( "createdAt", $util.time.nowISO8601() ))
$util.qr($locHologramNodeWeight.put( "updatedAt", $util.time.nowISO8601() ))
$util.qr($locHologramNodeWeight.put( "__typename", "Weight" ))
$util.qr($locHologramNodeWeight.put( "id", $util.defaultIfNullOrBlank($locHologramNodeWeight.id, $util.autoId()) ))
$util.qr($locWeight.add($util.dynamodb.toMapValues($locHologramNodeWeight)))
## [End] HologramNodeWeight **

## [Start] UserNodeWeight **
$util.qr($locUserNodeWeight.put( "createdAt", $util.time.nowISO8601() ))
$util.qr($locUserNodeWeight.put( "updatedAt", $util.time.nowISO8601() ))
$util.qr($locUserNodeWeight.put( "__typename", "Weight" ))
$util.qr($locUserNodeWeight.put( "id", $util.defaultIfNullOrBlank($locUserNodeWeight.id, $util.autoId()) ))
$util.qr($locWeight.add($util.dynamodb.toMapValues($locUserNodeWeight)))
## [End] UserNodeWeight **

## [Start] set table **
{
  "version": "2018-05-29",
  "operation": "BatchPutItem",
  "tables": {
    "Domain-r3nu2ysuhza7lkwssv6vr75d6q-test": $util.toJson($locDomain),
    "Node-r3nu2ysuhza7lkwssv6vr75d6q-test": $util.toJson($locNode),
    "HologramNode-r3nu2ysuhza7lkwssv6vr75d6q-test": $util.toJson($locHologramNode),
    "UserNode-r3nu2ysuhza7lkwssv6vr75d6q-test": $util.toJson([]),
    ##"Weight-r3nu2ysuhza7lkwssv6vr75d6q-test": $utils.toJson($locWeight)
  }
}
## [End] set table **

## [End] Prepare DynamoDB BatchPutItem Request. **

4. Here is Mutation.CreateNodeBuildingNecessaryItems.res.vtl

## If there was an error with the invocation
## there might have been partial results
#if ( $util.isNull($ctx.result) )
  $util.unauthorized()
#end

#if ($ctx.error)
  $util.appendError($ctx.error.message, $ctx.error.type)
#end
## Also returns data for the field in the GraphQL response

#set( $response = {
  "domain": $ctx.result.data.Domain-r3nu2ysuhza7lkwssv6vr75d6q-test,
  "node": $ctx.result.data.Node-r3nu2ysuhza7lkwssv6vr75d6q-test,
  "hologramNode": $ctx.result.data.HologramNode-r3nu2ysuhza7lkwssv6vr75d6q-test,
  "weight": $ctx.result.data.Weight-r3nu2ysuhza7lkwssv6vr75d6q-test,
  "userNode": $ctx.result.data.UserNode-r3nu2ysuhza7lkwssv6vr75d6q-test
} )

$utils.toJson($response)

5. Create customeResource.json to let cloudFormation identify resource

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "An auto-generated nested stack.",
  "Metadata": {},
  "Parameters": {
    "AppSyncApiId": {
      "Type": "String",
      "Description": "The id of the AppSync API associated with this project."
    },
    "AppSyncApiName": {
      "Type": "String",
      "Description": "The name of the AppSync API",
      "Default": "AppSyncSimpleTransform"
    },
    "env": {
      "Type": "String",
      "Description": "The environment name. e.g. Dev, Test, or Production",
      "Default": "NONE"
    },
    "S3DeploymentBucket": {
      "Type": "String",
      "Description": "The S3 bucket containing all deployment assets for the project."
    },
    "S3DeploymentRootKey": {
      "Type": "String",
      "Description": "An S3 key relative to the S3DeploymentBucket that points to the root\nof the deployment directory."
    }
  },
  "Resources": {
    "EmptyResource": {
      "Type": "Custom::EmptyResource",
      "Condition": "AlwaysFalse"
    },
    "CreateNodeBuildingNecessaryItemsResolver": {
      "Type": "AWS::AppSync::Resolver",
      "Properties": {
        "ApiId": {
          "Ref": "AppSyncApiId"
        },
        "DataSourceName":"NodeBuilding",
        "TypeName": "Mutation",
        "FieldName": "CreateNodeBuildingNecessaryItems",
        "RequestMappingTemplateS3Location": {
          "Fn::Sub": [
            "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Mutation.CreateNodeBuildingNecessaryItems.req.vtl",
            {
              "S3DeploymentBucket": {
                "Ref": "S3DeploymentBucket"
              },
              "S3DeploymentRootKey": {
                "Ref": "S3DeploymentRootKey"
              }
            }
          ]
        },
        "ResponseMappingTemplateS3Location": {
          "Fn::Sub": [
            "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Mutation.CreateNodeBuildingNecessaryItems.res.vtl",
            {
              "S3DeploymentBucket": {
                "Ref": "S3DeploymentBucket"
              },
              "S3DeploymentRootKey": {
                "Ref": "S3DeploymentRootKey"
              }
            }
          ]
        }
      }
    }
  },
  "Conditions": {
    "HasEnvironmentParameter": {
      "Fn::Not": [
        {
          "Fn::Equals": [
            {
              "Ref": "env"
            },
            "NONE"
          ]
        }
      ]
    },
    "AlwaysFalse": {
      "Fn::Equals": ["true", "false"]
    }
  },
  "Outputs": {
    "EmptyOutput": {
      "Description": "An empty output. You may delete this if you have at least one resource above.",
      "Value": ""
    }
  }
}

6. Query appsync with mutation graphql string

buildNodeTest: async ( context ) => { // eslint-disable-line no-unused-vars
      //const currentHologramNodes = context.state.currentHologramNodes;
      const currentHologram = context.rootState.hologram.currentHologram;
      //const userNodes = context.state.userNodes;
      const userID = context.rootState.auth.userData.id

      const calculation = {
        "weight": {
          "link": 1,
          "note": 0.01,
        }
      };

      const createEssentialItemVariables = {
        domain: {
          "baseType": "Domain",
          "owner": userID
        },
        node: {
          "baseType": "Node",
          "owner": userID
        },
        hologramNodeWeight: {
          "baseType": "Weight",
          "weightType": "hologramNode",
          "owner": userID,
          "value": 0,
          "calculation": JSON.stringify(calculation),
          "accessPolicy": currentHologram.accessPolicy
        },
        userNodeWeight: {
          "baseType": "Weight",
          "weightType": "userNode",
          "owner": userID,
          "value": 0,
          "calculation": JSON.stringify(calculation),
          "accessPolicy": currentHologram.accessPolicy
        },
        hologramNode: {
          "accessPolicy": currentHologram.accessPolicy,
          "owner": userID
        },
        userNode: {
          "owner": userID
        }
      }

      try {
        const result = await API.graphql(graphqlOperation(createNodeBuildingNecessaryItemsMutation, createEssentialItemVariables));
        return Promise.resolve(result)
      } catch(err){
        console.error(err)
        return Promise.reject(err)
      }

    },

Issue

1. Every time I try to invoke the query the request become pending, after 8~10 seconds it received no response and display CORS error. ( There is no log at cloudwatch )

2. If I intendedly make a error at req.vtl, like pass a empty list, there will be request error log at cloudwatch and the client will receive error code.

How can I solve this kind of problem.

REF

• Creating GraphQL Batch Operations for AWS Amplify with AppSync and Cognito
• Practical use cases for AWS AppSync Pipeline Resolvers – Part 3: Heterogeneous Data Sources
• Resolver to batch update multiple tables with a connection aws-amplify/amplify-cli#1097
• Add DynamoDB table name as an output for default model stacks aws-amplify/amplify-cli#1145
• Amplify API - Add support for DynamoDB batch operations. aws-amplify/amplify-category-api#443
• Access denied when BatchGet Item aws-amplify/amplify-cli#4816

[totuslink]

Is this issue solvable? Does amplify support this feature?

[akshbhu]

Hi @totuslink

Can you try the same query on Appsync console?

[totuslink]

Hi @akshbhu , Thank for your response!!

After login with my cognito admin account and enter every information at appsync console then press play, the loading spinner keep spinning. 2 min after the spinner disappear and console display Network error which is just like operating at client side.

[totuslink]

Hi, Amplify ~

I still lost in this problem which is very hard to solve, if there is any error log I believe I can solve it by myself. But there isn't any error log on my cloudwatch.

Can anyone help us light up a path.

[totuslink]

@akshbhu Sorry for bothering, but do you have other method to BatchPutItem into multiple tables?

Any help will be appreciated.

[akshbhu]

Hi @totuslink

Are you still stuck on this issue?

[SaurabBajgai]

Is this still pending? can we perform batchputItem?9727 is what I am currently facing

[EiffelFly]

@sarv-fuse I am the creator of Totuslink, Thanks for getting back, but I am afraid that I can't offer lots of details, the project had closed for a while.

Some clues I can offer are:

• The Batch put item is not working across multiple tables but it can work on a single table.
• The document needs to be updated, there are lots of misleading information.

[SaurabBajgai]

@sarv-fuse I am the creator of Totuslink, Thanks for getting back, but I am afraid that I can't offer lots of details, the project had closed for a while.

Some clues I can offer are:

• The Batch put item is not working across multiple tables but it can work on a single table.
• The document needs to be updated, there are lots of misleading information.

@EiffelFly Thank you for your reply, I am trying to do BatchputItem for single table and currently i am facing

• "message": "Unsupported operation 'BatchPutItem'. Datasource Versioning only supports the following operations (TransactGetItems,PutItem,BatchGetItem,Scan,Query,GetItem,DeleteItem,UpdateItem,Sync)"

Will there be issue enabling versioning for API to do BatchPutItem?

[EiffelFly]

@sarv-fuse

Maybe this SO article will help

https://stackoverflow.com/questions/61045181/aws-amplify-custom-resolver-unsupported-operation-batchputitem

[josefaidt]

Closing due to inactivity

[OperationalFallacy]

Is Amplify planning to have a more user-friendly way to do batch operations on a model?

A user-friendly way would be something like this

import { writeBatch, doc } from "firebase/firestore"; 

// Get a new write batch
const batch = writeBatch(db);

// Set the value of 'NYC'
const nycRef = doc(db, "cities", "NYC");
batch.set(nycRef, {name: "New York City"});

// Update the population of 'SF'
const sfRef = doc(db, "cities", "SF");
batch.update(sfRef, {"population": 1000000});

// Commit the batch
await batch.commit();