search

sensedeep / dynamodb-onetable

DynamoDB access and management for one table designs with NodeJS
https://doc.onetable.io/
MIT License
686 stars 109 forks source link

[BUG]: TypeError: Cannot read property 'ConditionExpression' of null #42

Closed ericmarcos closed 3 years ago

ericmarcos commented 3 years ago

Describe the bug I'm evaluating dynamodb-onetable for a project and I can't seem to make it work, even for just a basic "create" operation, please see code sample below.

To Reproduce

import Dynamo from 'dynamodb-onetable/Dynamo'
import { Table } from 'dynamodb-onetable'
import { DynamoDBClient } from '@aws-sdk/client-dynamodb'

/**
 *  in order to use this layout, you will need to start up DynamoDB local
 *  and provision the table
 *
 *  docker run -p 8000:8000 amazon/dynamodb-local
 *
aws dynamodb create-table \
   --endpoint-url http://localhost:8000 \
   --table-name TestLocal \
   --attribute-definitions AttributeName=pk,AttributeType=S AttributeName=sk,AttributeType=S \
     AttributeName=gs1pk,AttributeType=S AttributeName=gs1sk,AttributeType=S \
     AttributeName=gs2pk,AttributeType=S AttributeName=gs2sk,AttributeType=S \
     AttributeName=gs3pk,AttributeType=S AttributeName=gs3sk,AttributeType=S \
   --key-schema KeyType=HASH,AttributeName=pk KeyType=SORT,AttributeName=sk \
   --billing-mode PAY_PER_REQUEST \
   --global-secondary-indexes 'IndexName=gs1,KeySchema=[{KeyType="HASH",AttributeName="gs1pk"},{KeyType="SORT",AttributeName="gs1sk"}],Projection={ProjectionType="ALL"}' \
     'IndexName=gs2,KeySchema=[{KeyType="HASH",AttributeName="gs2pk"},{KeyType="SORT",AttributeName="gs2sk"}],Projection={ProjectionType="ALL"}' \
     'IndexName=gs3,KeySchema=[{KeyType="HASH",AttributeName="gs3pk"},{KeyType="SORT",AttributeName="gs3sk"}],Projection={ProjectionType="ALL"}'
 */

const client = new Dynamo({ client: new DynamoDBClient({ region: 'us-west-1', endpoint: 'http://localhost:8000' }) })

const table = new Table({
    client: client,
    name: 'TestLocal2',
    uuid: 'ulid',
    delimiter: '#',
    schema: {
        indexes: {
            primary: { hash: 'pk', sort: 'sk' },
            gs1:     { hash: 'gs1pk', sort: 'gs1sk', follow: true },
        },
        models: {
            User: {
                pk:          { type: String, value: 'USR#${id}' },
                sk:          { type: String, value: 'USR#${id}' },
                id:          { type: String, uuid: 'ulid' },
                name:        { type: String, required: true },
                status:      { type: String, default: 'active' },
                zip:         { type: String },
            },
            Event: {
                pk:          { type: String, value: 'USR#${userId}' },
                sk:          { type: String, value: 'EVT#${id}' },
                id:          { type: String, uuid: 'ulid' },
                name:        { type: String, required: true },
                gs1pk:       { type: String, value: 'EVT#${id}' },
                gs1sk:       { type: String, value: '' },
            }
        }
    }
})

const User = table.getModel('User')
const Event = table.getModel('Event')

const main = async () => {
    try {
        let account = await User.create({
            name: 'Eric',
        })

        console.log(account)
    } catch (e) {
        console.log(e)
    }
}

main()

I compile this script with Babel: babel index.js --out-dir dist

And run it with node dist/index.js

Which results in this error:

TypeError: Cannot read property 'ConditionExpression' of null
    at serializeAws_json1_0PutItemInput (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/client-dynamodb/dist/cjs/protocols/Aws_json1_0.js:5483:19)
    at Object.serializeAws_json1_0PutItemCommand (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/client-dynamodb/dist/cjs/protocols/Aws_json1_0.js:344:27)
    at serialize (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/client-dynamodb/dist/cjs/commands/PutItemCommand.js:121:30)
    at /some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/middleware-serde/dist/cjs/serializerMiddleware.js:5:27
    at /some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/middleware-logger/dist/cjs/loggerMiddleware.js:6:28
    at DynamoDBClient.send (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/smithy-client/dist/cjs/client.js:23:20)
    at Dynamo.<anonymous> (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/dynamodb-onetable/dist/cjs/Dynamo.js:95:38)
    at Generator.next (<anonymous>)
    at /some-path/2021-05-27-dynamodb-onetable-test/node_modules/dynamodb-onetable/dist/cjs/Dynamo.js:17:71
    at new Promise (<anonymous>)

I've also tried to set all the attributes of the model:

let account = await User.create({
    id: '01ARZ3NDEKTSV4RRFFQ69G5FAV',
    name: 'Eric',
    status: 'active',
    zip: 'XYZ'
})

and a get another error:

ResourceNotFoundException: ResourceNotFoundException
    at deserializeAws_json1_0PutItemCommandError (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/client-dynamodb/dist/cjs/protocols/Aws_json1_0.js:2904:41)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async /some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/middleware-serde/dist/cjs/deserializerMiddleware.js:6:20
    at async /some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/middleware-signing/dist/cjs/middleware.js:12:24
    at async StandardRetryStrategy.retry (/some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/middleware-retry/dist/cjs/defaultStrategy.js:56:46)
    at async /some-path/2021-05-27-dynamodb-onetable-test/node_modules/@aws-sdk/middleware-logger/dist/cjs/loggerMiddleware.js:6:22

Expected behavior A User item is created

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Additional context I'm using the local version of DynamoDB (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)

mobsense commented 3 years ago

Thanks for the excellent issue description. I can reproduce this and will get back to you with diagnosis soon.

mobsense commented 3 years ago

Okay, diagnosed this issue.

The core issue is the handling of value templates was not taking into account dependent fields. i.e. the UUID needed to be generated before preparing the PK field. The fix is to calculate the order of dependencies in the properties and to calculate the property values according to that order.

There was an associated issue with create when a PK/SK is not fully provided in the API properties.

We'll add a samples directory with a sample modeled on your bug code for verification.

See latest commit for the fix and for the samples/crud

Fixed in 1.4.2

mobsense commented 3 years ago

FYI: DynamoDB won't let you write empty strings.

So

 gs1sk:       { type: String, value: '' },

Should always have a non-empty value.

Also, if you prefer, once defining the UUID in the table, each field can use 

uuid: true

to use the Table level setting, which you set to 'ulid'.

Last tip, define a logger to see the actual generated DynamoDB code.

Add this to your Table params:

    logger: (type, message, context) => {
        if (type == 'trace' || type == 'data') return
        console.log(type, message, JSON.stringify(context, null, 4))
    }
ericmarcos commented 3 years ago

@mobsense Thanks for the quick fix! I just tested it and it works now =)

I've applied your tips as well, however now I'm facing a new problem: I can't create a Table from my app. I used to create the table via CLI before, but I'd like to do it all programatically. I've opened a new issue (#43) for this so I'm closing this one.