aws-powertools / powertools-lambda-typescript

Powertools is a developer toolkit to implement Serverless best practices and increase developer velocity.
https://docs.powertools.aws.dev/lambda/typescript/latest/
MIT No Attribution
1.59k stars 143 forks source link

Docs: refresh examples #1103

Closed ijemmy closed 8 months ago

ijemmy commented 2 years ago

Description of the improvement

Summary of the proposal The repository contains - at present - examples in CDK and SAM that showcase how to use the Powertools utilities available. The new idempotency utility should be part of these example.

How, where did you look for information In the CDK & SAM examples.

Missing or unclear documentation N/A

Improvement Include examples of the new utility.

Related existing documentation

https://github.com/awslabs/aws-lambda-powertools-typescript/tree/main/examples

Related issues, RFCs

447

dreamorosi commented 1 year ago

As part of this issue I was thinking of addressing a couple of points: a. add the Idempotency utility to the examples b. refactor the part of the example that covers Parameters c. add Batch Processing d. improve the README

a. Add Idempotency

As part of this issue we should add something about the Idempotency utility. Based on the current sample I think the most logical place to add the utility would be in the put-item function as specifically modify the handler in such a way that the Idempotency utility is used to ensure that the same item is inserted only once.

Currently the code that inserts the item is part of the handler and looks like this:

const body = JSON.parse(event.body);
const { id, name } = body;

await docClient.send(
  new PutCommand({
    TableName: tableName,
    Item: {
      id,
      name,
    },
  })
);

I propose to extract the logic above to its own function and use the makeIdempotent function wrapper on it using all or some of the arguments.

b. Refactor Parameters

A couple of months ago we added the Parameters utility to the examples in conjunction with the GA of the utility. At the time I recall us struggling to find a way to use it and eventually we settled on creating a second REST API endpoint and using the utility to retrieve an arbitrary value which is then returned in the response.

After looking at the usage again, and also listening customer feedback, I think there's an opportunity to improve this area and use the utility for something more useful.

My first choice would have been to store the name of the Idempotency table (from the previous point) in a SSM parameter and then use the Parameters utility to retrieve it. This however is not possible at the moment given that we must instantiate the DynamoDBPersistenceLayer outside of the function handler and using await at the top-level requires ESM.

For context, and to maybe revisit this once we finally support ESM in the coming months, this is what I mean:

import { getParameter } from '@aws-lambda-powertools/parameters/ssm';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';

const persistenceLayer =  new DynamoDBPersistenceLayer({
  tableName: await getParameter('/dev/idempotency-table-name');
});

With that in mind, I think we could revisit this in a month or so and hopefully we'll be able to use the Parameters utility to retrieve the name of the Idempotency table using ES Modules.

c. Add Batch Processing

Given that we are inserting items in a DynamoDB Table, we could enable the Stream on the table and plug a fourth Lambda that consumes the stream and uses Batch Processing to do so.

The trigger for this function could be set to new items (aka inserts NOT updates / deletes). This function would provide an async record handler that would process each record in the batch and do something on it (TBD). Ideas:

d. Improve the README

At the moment the README is a bit bare and I think we could improve it by adding a couple of things:


@am29d what do you think? If we agree I'll move this out of the GA milestone.

AllyMurray commented 8 months ago

Is there a working example for using v2 with cdk?

We are using SST/CDK and after migrating to V2 the transpiled JS is not valid, the lambda fails with __classPrivateFieldSet is not defined. I see there are private fields used in the IdempotencyConfig.

dreamorosi commented 8 months ago

Hi, I've started working on a refreshed version earlier today and I should have a branch sometime later this week, however the private fields were present already in v1 (see blame view here) and the examples are already using v2.

I'd suggest to check your typescript version and the target version you're transpiling to check that they're recent and targeting Node.js 16 or newer. If none of them fixes the issue I'd recommend opening a dedicated issue.

AllyMurray commented 8 months ago

Thanks @dreamorosi, I did see it was present in v1. I assumed it was something to do with transpiling the esm code now, our repo was already esm.

Target is es2022 and our version of typescript wasn't that far behind but I had already tried using the latest of that.

When you have a branch you can share could you please post the link here? If I still can't get it running after having a look I will create a small repo to reproduce and open an issue.

dreamorosi commented 8 months ago

I just published my working branch, which you can find here. Note that it's still very unpolished, but hopefully it'll help you.

dreamorosi commented 8 months ago

Hi @AllyMurray, just wanted to let you know that I have encountered the same issue and I'm sure it's not an issue with your setup but a bug on our side.

I have opened an issue #2230 to track this and I think I have already a fix available. If I'm right, we should be able to make a release tomorrow with the fix.

AllyMurray commented 8 months ago

That's great @dreamorosi, thanks for sorting that out so quickly. I did have a look at the branch and couldn't see anything obvious so I planned to create a repo to reproduce the issue but you beat me to it 😄

dreamorosi commented 8 months ago

No worries, I initially understood the issue was using v2 in general, but then when I got to adding the idempotency module I got the same error and understood.

It was actually so timely that you commented in the morning, otherwise I'd have thought I was doing something wrong and spent hours trying to fix the wrong thing.

Anyway the fix is merged and we're running tests now, if everything works we should be able to release later today.

github-actions[bot] commented 8 months ago

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

github-actions[bot] commented 7 months ago

This is now released under v2.0.4 version!