unmarshallOutput issue with @aws-sdk/lib-dynamodb

[sawilde]

Describe the bug As you know you've had to pin electrodb to an old version of the @aws-sdk/lib-dynamodb due to an issue with a missing export.

I should let you know that this export has been missing since version 3.495.0 but because of the line

https://github.com/tywalch/electrodb/blob/22af839342db46ba82ee062e95258af81f5d040a/src/client.js#L10

this was not noticed until the @aws-sdk/lib-dynamodb removed the utils.js file in the last few releases.

I have raised the issue here https://github.com/aws/aws-sdk-js-v3/issues/6276 and letting you know so you can keep track

[bestickley]

@sawilde, thanks for creating this issue. I think I just ran into it. In my lambda logs I see:

2024-09-06T22:01:37.315Z    undefined   ERROR   Uncaught Exception  
{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module '@aws-sdk/lib-dynamodb/dist-cjs/commands/utils'\nRequire stack:\n- /var/task/index.mjs",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module '@aws-sdk/lib-dynamodb/dist-cjs/commands/utils'",
        "Require stack:",
        "- /var/task/index.mjs",
        "    at _loadUserApp (file:///var/runtime/index.mjs:1087:17)",
        "    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1119:21)",
        "    at async start (file:///var/runtime/index.mjs:1282:23)",
        "    at async file:///var/runtime/index.mjs:1288:1"
    ]
}

I'm bundling my app with the following Dockerfile. Notice --external:"@aws-sdk/*".

ARG NODE_VERSION=20
FROM public.ecr.aws/docker/library/node:$NODE_VERSION-alpine as builder
WORKDIR /app
# see https://turbo.build/repo/docs/guides/tools/docker#the-solution
COPY ./json/ .
COPY ./pnpm-*.yaml ./
RUN corepack enable pnpm && pnpm install --frozen-lockfile
COPY ./full/ .
ARG RELATIVE_PATH_TO_HANDLER
RUN pnpm esbuild $RELATIVE_PATH_TO_HANDLER --banner:js="const require = (await import('node:module')).createRequire(import.meta.url);" --bundle --external:"@aws-sdk/*" --format=esm --minify --platform=node --target=node$NODE_VERSION --outfile=index.mjs

FROM public.ecr.aws/lambda/nodejs:$NODE_VERSION as runner
COPY --from=builder /app/index.mjs ./index.mjs
CMD ["index.handler"]

Do you have any recommendations on how to resolve? See: https://github.com/tywalch/electrodb/issues/398

[Xenoha]

What about the provided SDK version in the lambda runtime? I am running into this issue in the lambda environment using the CDK bundling where the aws V3 sdks are removed for runtime versions in the build.

[Xenoha]

Team, AWS just publish 3.695 5hours ago, and ElectroDB just returns empty for all my entity.get methods. It first started by getting the above error, I downgraded the lambda runtime and removed the error, but dynamo access using electro wouldn't throw an error and would come back empty. I installed 3.395 only, but since I am using typescript the aws-sdk modules are not included in the builds.

Right now, this package has broken my entire stack. Is there a plan to fix/upgrade?

[tywalch]

I'm not familiar with your flow but it sounds like your build process removes Electro's internal aws dependency? You're using the latest version of Electro? The current status of this issue was that I had assumed it was at least mitigated (not a permanent fix) by version pinning.

[Xenoha]

@tywalch thanks for the quick reply.

I'm just trying to follow the above comment by @bestickley . He is using esbuild a the end of the day that has the --external flag '@aws-sdk/*'. This flag for esbuild will remove all lambda dependencies from the build (aws-sdk is provided in the lambda runtime). So, I'm not sure how me installing and using 3.395 would fix any of these issues with this flag still set.

Right now, I installed the 3.395 version of aws-sdk/client-dynamodb and all its dependencies. I then removed the 'aws-sdk/*` from my esbuild external modules command so that it would add 3.395 to the bundle. This removes the 'Cannot find modules error' BUT now the lambda just returns undefined with no error for any call I make.

I'm using electro version 2.14.3

I think the best way to reproduce is to remove a lambda stack that you currently have working and redeploy with a fresh install and upgraded packages of the aws-sdk and electro. This will make sure that you are using the latest lambda runtime version along for your runtime (I'm using Node V20).

for build context. This is my lambda config in the typescript cdk:

super(scope, id, { ...rest, handler: 'handler', runtime: Runtime.NODEJS_20_X, tracing: Tracing.ACTIVE, architecture: Architecture.X86_64, timeout: Duration.seconds(30), layers: rest.layers ? [...rest.layers, powerToolsLayer] : [powerToolsLayer], environment: { NODE_OPTIONS: '--enable-source-maps', ...rest.environment, }, logGroup: new LogGroup(scope,${id}-log-group, { logGroupName:/aws/lambda/${id}, retention: getRetention(stage), removalPolicy: stage === 'Prod' ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY, }), bundling: { minify: false, sourceMap: true, keepNames: true, sourcesContent: true, format: OutputFormat.CJS, esbuildArgs: { '--tree-shaking': 'true' }, externalModules: [/* '@aws-sdk/*', */ '@aws-lambda-powertools/*'], }, })