Error in obtaining the data | obtains the information but apart gives TIMEOUT

[manuelperezg]

Hello,

First of all congratulations for creating the library for v3 version, this is very cool.

Second: When it comes to replicate the example adapting it to my code,

export async function dynamoGetDaxv3() {
  const tableName = "vb-pokemons2-cache-dev";
    // ======================
    /*  const daxClient = new AmazonDaxClient({
         client: new DynamoDBClient({
             endpoint: "#",
             region: 'us-east-1'
         })
     }); */
    const documentDaxClient = new AmazonDaxClient({
        client: DynamoDBDocumentClient.from(new DynamoDBClient({
            endpoint: "#",
            region: 'us-east-1'
        }))
    });
    // ======================
    const dataG = []
    for (let index = 1; index < 100; index++) {
        const params = {
            TableName: tableName,
            Key: {
                id: index,
            },
        };
        try {
            const data = await documentDaxClient.send(new GetCommand(params));
            dataG.push(data.Item)
            // console.log("Success:", data.Item);
            console.log('id: ' + data.Item.id + ' name: ' + data.Item.name);
        } catch (err) {
            console.error("Error:", err);
        }
    }
    return {
        statusCode: 200,
        body: JSON.stringify({ message: "SUCCESS", data: dataG }),
        headers: {
            'Content-Type': 'application/json'
        }
    };
}

It makes the request very well to my service and gets the data, but when it gets the information, in the middle of the process it gives me a timeout error. Could you help me with this,

Is it a bug in the library?

I send you a big hug, greetings

Manuel P

[kwojcicki]

Hey Manuel, glad its somewhat working :D

What size of instances are you using? Wondering if the underlying DAX cluster is under provisioned and not able to handle the load (granted the load here seems quite low only 100 loads).

[paul-uz]

I'm hitting the same issue.

Code below:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, ScanCommand, ScanCommandInput, ScanCommandOutput } from '@aws-sdk/lib-dynamodb';
import { NodeHttpHandler } from '@smithy/node-http-handler';
import { AbstractHandler, EnvVars, waitForLogger } from '@paul-uz/gc-sdk';
import AmazonDaxClient from 'amazon-dax-client-sdkv3';

const requestHandler = new NodeHttpHandler({
  connectionTimeout: 30000,
  socketTimeout: 30000,
});

class Handler extends AbstractHandler {
  daxClient = new AmazonDaxClient({
      client: this.getClient('DynamoDBDocumentClient'),
  });

  async main(): Promise<void> {
    try {
      const totalSegments = 10_000;  // Your total segments
      const getItemsArray = Array.from({ length: totalSegments }, (_, i) =>
        this.getAll('My-Table', [], i, totalSegments)
      );

      const result = await Promise.all(getItemsArray);

      console.log(result);
    } catch (error: unknown) {
      this.logger?.error(error);
    }

    if (this.logger) {
      await waitForLogger(this.logger);
    }

    return;
  }

  getAll = async (
    tableName: string,
    items: Record<string, any>[] = [],
    segment: number,
    totalSegments: number,
    evaluationKey?: Record<string, any>,
  ): Promise<Record<string, any>[]> => {
    const params: ScanCommandInput = {
      TableName: tableName,
      Segment: segment,
      TotalSegments: totalSegments,
    };

    if (evaluationKey) {
      params.ExclusiveStartKey = evaluationKey;
    }

    const result: ScanCommandOutput = await this.daxClient.send(new ScanCommand(params));

    if (result.Items) {
      items.push(...result.Items);
    }

    if (result.LastEvaluatedKey) {
      return this.getAll(tableName, items, segment, totalSegments, result.LastEvaluatedKey);
    } else {
      return items;
    }
  }
}

export const handler = new Handler({
  clients: [
    DynamoDBDocumentClient.from(new DynamoDBClient({
      endpoint: 'daxs://foobar.dax-clusters.eu-west-1.amazonaws.com',
      region: 'eu-west-1',
      maxAttempts: 2,
      requestHandler,
    })),
  ],
});
export const main = handler.main.bind(handler);

Despite creating the requestHandler and setting the timeout to 30s, it still times out after 10s.

[kwojcicki]

Despite creating the requestHandler and setting the timeout to 30s, it still times out after 10s.

DAX doesn't respect the handler passed to the underlying DDB client and instead makes its own.

I didn't do a great job of allowing that to be configurable and can work on that.

Regarding things timing out, over the weekend I'll try to spam my DAX cluster a bit and see what happens