Node.js v18 image lacks ESM bundles in @aws-sdk

[elieux]

I was hoping that the Node.js v18 image would allow me to skip packaging @aws-sdk, but it seems the packages provided in the image are stripped and only contain the dist-cjs bundles but not the dist-es bundles. I went all out on ES modules and this is kind of a bummer.

> docker run --rm -it --entrypoint /bin/ls amazon/aws-lambda-nodejs:18 /var/runtime/node_modules/@aws-sdk/client-s3 -l
total 132
-rw-r--r--   1 root root 100375 Feb  6 09:55 CHANGELOG.md
drwxr-xr-x   7 root root   4096 Feb 28 14:21 dist-cjs
-rw-r--r--   1 root root  11390 Feb  6 09:55 LICENSE
-rw-r--r--   1 root root   4295 Feb  6 09:55 package.json
-rw-r--r--   1 root root   5780 Feb  6 09:55 README.md

> npm pack @aws-sdk/client-s3@3.188.0 --dry-run 2>&1 | sed -n "/Tarball Contents/,/Tarball Details/{p}" | grep -v Tarball | cut -c20- | cut -d/ -f1 | sort | uniq
dist-cjs
dist-es
dist-types
CHANGELOG.md
LICENSE
package.json
README.md

[paambaati]

@elieux Slightly relevant question, but were you able to get the entrypoint to work correctly? Say your handler is index.mjs, how do you set the Dockerfile CMD?

If I use CMD ["index.handler"], it is still looking up index.js and not index.mjs, even if the package type is set to module.

[krk]

AWS SDK v3 is designed to be used from ES Modules, independent of its internal implementation. In other words, dist-cjs folder is sufficient for it to be imported as an ESM:

Dockerfile:

FROM public.ecr.aws/lambda/nodejs:18

COPY index.mjs /var/task/

CMD ["index.handler"]

index.mjs:

import { S3Client } from "@aws-sdk/client-s3";

const s3 = new S3Client({ region: process.env.AWS_REGION });

export const handler = async (event) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({
      "typeof S3Client": typeof S3Client,
      "config.serviceId": s3.config.serviceId,
    }),
  };
  return response;
};

Result:

{
  "statusCode": 200,
  "body": "{\"typeof S3Client\":\"function\",\"config.serviceId\":\"S3\"}"
}

[elieux]

@paambaati, I haven't tried in Docker, but in Lambda it just works. The runtime code looks for different files based on package.json and a predefined order. I hope it's okay to paste the code here.

    async function _tryRequire(appRoot, moduleRoot, module2) {
      verbose("Try loading as commonjs: ", module2, " with paths: ,", appRoot, moduleRoot);
      const lambdaStylePath = path.resolve(appRoot, moduleRoot, module2);
      const extensionless = _tryRequireFile(lambdaStylePath);
      if (extensionless) {
        return extensionless;
      }
      const pjHasModule = _hasPackageJsonTypeModule(lambdaStylePath);
      if (!pjHasModule) {
        const loaded2 = _tryRequireFile(lambdaStylePath, ".js");
        if (loaded2) {
          return loaded2;
        }
      }
      const loaded = pjHasModule && await _tryAwaitImport(lambdaStylePath, ".js") || await _tryAwaitImport(lambdaStylePath, ".mjs") || _tryRequireFile(lambdaStylePath, ".cjs");
      if (loaded) {
        return loaded;
      }
      verbose("Try loading as commonjs: ", module2, " with path(s): ", appRoot, moduleRoot);
      const nodeStylePath = __require.resolve(module2, {
        paths: [appRoot, moduleRoot]
      });
      return __require(nodeStylePath);
    }

[elieux]

@krk, thanks. I'd tried it and it didn't work. I'll check again and report back.

[iheffernan]

This is an issue depending on what you are importing, regardless of whether you're using CommonJS or ESModule. The issue is that, for some reason, not all the exported types are resolving. Here's a quick example to illustrate...

import { S3Client } from "@aws-sdk/client-s3";

import pkg from '@aws-sdk/client-s3'

const s3 = new S3Client({});

export const handler = async(event) => {
    console.log(`MetadataDirective = ${pkg.MetadataDirective}`);
    if (pkg) console.log(`MetadataDirective.REPLACE = ${pkg.MetadataDirective.REPLACE}`);
};

In this example, S3Client has no problem resolving as a named export. If I try to include MetadataDirective as a named export, I get an error stating Named export 'MetadataDirective' not found and recommending that I use the default export. The above code then uses the default export, but MetadataDirective is undefined. This even though this is exported in models_0.* in the client-s3 package.

Fallback is simply to include the SDKs as dependencies in my Lambda which inflates the size, but works. Would be helpful to understand which exports we can count on being present so we can make informed decisions on devDependencies.

[elieux]

Hmm. It seems to work now, which is nice.

For future reference, this is the error I was getting previously:

{
    "errorType": "Error",
    "errorMessage": "Cannot find package '@aws-sdk/client-lambda' imported from /var/task/XXX.mjs",
    "code": "ERR_MODULE_NOT_FOUND",
    "stack": [
        "Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@aws-sdk/client-lambda' imported from /var/task/XXX.mjs",
        "    at new NodeError (node:internal/errors:400:5)",
        "    at packageResolve (node:internal/modules/esm/resolve:894:9)",
        "    at moduleResolve (node:internal/modules/esm/resolve:987:20)",
        "    at moduleResolveWithNodePath (node:internal/modules/esm/resolve:938:12)",
        "    at defaultResolve (node:internal/modules/esm/resolve:1202:79)",
        "    at nextResolve (node:internal/modules/esm/loader:163:28)",
        "    at ESMLoader.resolve (node:internal/modules/esm/loader:842:30)",
        "    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)",
        "    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:77:40)",
        "    at link (node:internal/modules/esm/module_job:76:36)"
    ]
}

[slootjes]

The bundles SDK is an older version which doesn't export (all) types, I ran into the same issue. The "solution" is to pack the relevant AWS SDKs yourself.