SignatureV4 module loads unneeded dependencies

As per title, the @smithy/signature-v4 package loads some dependencies (@aws-crypto/*) even though they are not needed / not used in the context of signing a request. This happens because of the package importing a utility function from the @smithy/eventstream-codec, and in doing so, it also loads the full content of the package and its dependencies.

This results in a bundle size that is 47% larger than it could be. Below an in depth exploration with reproduction steps.

Use Case

In a Node.js environment (AWS Lambda) I am using the @smithy/signature-v4 package to sign requests. The use case is a Lambda@Edge attached to a CloudFront distribution with a Lambda function URL as origin. Given that Lambda function URLs support only IAM as authentication method, I'm using the Lambda@Edge to manipulate the request & sign it (among other validations), so that by the time it reaches the origin it's authenticated.

Below a high level diagram of what I just described above:

Reproduction steps

I have prepared a bare bones repository with the function that uses the @smithy/signature-v4 package that you can use to reproduce the steps below, you can find it here and follow along.

1. Clone the repository & install dependencies

Run git clone git@github.com:dreamorosi/smithy-sigv4.git and then npm ci to setup the repo.

2. Bundle code

Run npm run bundle in the project's root.

This step uses esbuild (which is default in both AWS CDK and AWS SAM) to create a bundle of the function (index.ts) and all its dependencies. As you can see from the command I use to bundle, I am using ESM and enabling tree shaking:

esbuild --bundle index.ts --format=esm --platform=node --main-fields=module,main --tree-shaking=true --outfile=out/index.mjs --metafile=out/meta.json

The command generates two output files, one is the function code including dependencies (out/index.mjs) and the other is a meta file (out/meta.json) which we'll use in the next step. The main output file is not minified so you can optionally inspect the content to verify the claims of this issue.

3. Analyze the bundle

In your browser, open https://esbuild.github.io/analyze/ and drag (or select) the meta file (out/meta.json) generated at the previous step. The resulting chart should look like this:

As you can see from the images above, the final bundle size (unminified) is 57.2 kb, half of which is comprised by dependencies under @aws-crypto, namely: @aws-crypto/util and @aws-crypto/crc32.

These two dependencies are also the only two that are not dual bundled (aka CJS + ESM like all @smithy/* and @aws-sdk/* packages are), which result in them not being excluded from tree shaking entirely.

By analyzing at the dependency tree, we can see that the @aws-crypto/* packages are brought in via the @smithy/eventstream-codec package:

A quick search in the node_modules/@smithy/signature-v4 shows that the @smithy/eventstream-coded package is imported only once:

find node_modules/@smithy/signature-v4 -type f -exec grep -l "@smithy/eventstream-codec" {} + 

# result
node_modules/@smithy/signature-v4/dist-es/SignatureV4.js
node_modules/@smithy/signature-v4/dist-cjs/index.js
node_modules/@smithy/signature-v4/package.json

And specifically only to use a single function (see import here):

import { HeaderMarshaller } from "@smithy/eventstream-codec";

This function, interestingly enough, doesn't rely nor import either of the @aws-crypto/* packages as evidenced in its implementation, but since it's brought in via barrel file it still get them included in the bundle.

4. Create alternate bundle to verify

To verify that this is the case, and only for demonstration purposes (read next section for more alternatives), we can manually modify the import found at L1 of node_modules/@smithy/signature-v4/dist-es/SignatureV4.js (which corresponds to this line in the source) like this:

-- import { HeaderMarshaller } from "@smithy/eventstream-codec";
++ import { HeaderMarshaller } from "@smithy/eventstream-codec/dist-es/HeaderMarshaller";

Using this method we are bypassing the default export and instead importing only that specific file (and other any file it imports).

If we run npm run bundle again, and analyze the output we can see that the @smithy/eventstream-codec module is now tree shaken correctly and the @aws-crypto/* are no longer part of the final bundle:

Not only that, but also the total size (unminified) drops to 30kb (-47.5%)

Potential Solutions

The workaround mentioned above is definitely not viable nor sustainable, however I believe the team should consider isolating that HeaderMarshaller utility function and avoid bringing in the rest of the @smithy/eventstream-codec package and all its dependencies (@aws-crypto/*).

Extract the utility in its own package or move to other existing package

This is pretty self explanatory, but essentially the suggestion is to move the HeaderMarshaller utility either in its own published package or in some other package (i.e. @smithy/signature-v4 or @smithy/http-protocol).

The team is better positioned to make a more informed choice on where it should land since I don't know where else that function is used.

Allow stable sub path exports

If you want to keep the HeaderMarshaller where it's now, another option would be to allow consumer modules to import it in isolation. This could be done for example via the exports field in the package.json.

To achieve this, you'd need to add following sections to the package.json file of the @smithy/eventstream-codec:

{
  "exports": {
    ".": {
      "import": "./dist-es/index.js",
      "require": "./dist-cjs/index.js"
    },
    "./headermarshaller": {
      "import": "./dist-es/HeaderMarshaller.js",
      "require": "./dist-cjs/HeaderMarshaller.js"
    }
  },
  "typesVersions": {
    "<4.0": {
      "dist-types/*": [
        "dist-types/ts3.4/*"
      ]
    },
    "*": {
      "headermarshaller": [
        "dist-types/ts3.4/*"
      ]
    }
  },
}

This way, consumer module (i.e. @smithy/signature-v4) would be able to import it like this:

-- import { HeaderMarshaller } from "@smithy/eventstream-codec";
++ import { HeaderMarshaller } from "@smithy/eventstream-codec/headermarshaller";

using this notation lets the module resolution skip the default import, which results in the same 30kb bundle I showed above.

smithy-lang / smithy-typescript