awslabs / dynamodb-data-mapper-js

A schema-based data mapper for Amazon DynamoDB.
https://awslabs.github.io/dynamodb-data-mapper-js/
Apache License 2.0
817 stars 106 forks source link

Multiple requests on a single query #210

Closed escar closed 2 years ago

escar commented 2 years ago

I have this code


constructor () {
    const client = new DynamoDB({
        region: 'eu-west-1',
        endpoint: dynamoDBUrl,
        httpOptions: { timeout: 10000 },
    });
    this.mapper = new DataMapper({ client });
}

async myFunc(memberId: string): Promise<Model> {
    const options: QueryOptions = {
        indexName: 'indexName',
        pageSize: 1,
        scanIndexForward: true,
    };

    const query = this.mapper.query(Model, { foo: 'bar', foo2: 'bar2' }, options);

    for await (const result of query) {
        return result;
    }

    return null;
}

This runs on a lambda function once (triggered by API Gateway).

The majority of times, everything looks fine and the lambda executes very quickly.

However, sometimes the lambda takes between 30-60 seconds to execute. When checking traces in Cloudwatch, I can see there's multiple calls to this Model table, sometimes 50+ requests.

DynamoDB's default timeout is 2 minutes, but I set it to 10 seconds, so in case of failure, it should be retried sooner. But the weirdest thing is that all these requests duration average is around 10-20ms, so it doesn't seem it's failing.

I can't see anything wrong in the code, just wondering if there is something I don't know about dynamodb-data-mapper...

escar commented 2 years ago

The table has a lot of records, dynamo can only load 1MB in each request... It has to do with pagination 🤦