Generated client protocol code should be bundled with each command, rather than shipped as a single file

[willyg302]

Describe the feature

Note: This might be more appropriate for https://github.com/awslabs/smithy-typescript depending on the solution, but filing here as the SDK is what I'm directly consuming.

Observe the following:

# Just to show we're in an installed client
λ pwd | cut -d / -f 10-20
node_modules/@aws-sdk/client-lambda/dist-cjs

# The protocol file shipped with this client is a huge single file
λ find protocols/ -type f | xargs du -hs
260K    protocols/Aws_restJson1.js

# In contrast the commands are small separate files
λ find commands/ -type f | xargs du -hs | cut -d$'\t' -f 1 | sort | uniq -c
     66 4.0K
      1 8.0K

Looking at the protocol file, it appears to be generated/concatenated boilerplate for each command. This request is for the command-specific protocol boilerplate to be separated out in a manner similar to commands/.

Use Case

I own a website using the JS SDK v3. We're already following SDK best practices such as using bare-bones clients with individual command imports.

We use webpack to bundle our code, and recently we've started caring a lot about bundle optimization. We have highly optimized "initial chunks" that are needed for page load, and that make very few AWS API calls so we only need to import a handful of commands for them. We separately have very large "dynamic chunks" that are fetched on-demand, as users load less-visited parts of our application. These make many AWS API calls and import many more commands.

The problem that a single protocol file presents is that webpack operates on the level of a "module" which can be seen as roughly equivalent to a file. If a given module is required by multiple chunks, it will be added to the chunk that needs it "first" (which is almost always an initial chunk). Therefore, if we use the same SDK client in both an initial and dynamic chunk, the protocol module (needed by both chunks) is placed in the initial chunk. This negatively impacts the size and performance of the initial chunk since the protocol file is so large, even though we use comparatively few commands from the client in the initial chunk.

Proposed Solution

If the protocol file could be split like the commands are, each would be seen as a separate module by bundlers like webpack. In our case, an initial chunk would then truly include the minimum code necessary for the AWS API calls that it makes.

Other Information

No response

Acknowledgements

• [ ] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

3.398.0 at the moment

Environment details (OS name and version, etc.)

N/A

[kuhe]

In my own usage of webpack, I've seen that it tree-shakes the unused portions of the protocol file. Is there anything preventing you from using that feature?

[willyg302]

We do, and we've observed the same. That's not the problem.

Continuing the above Lambda example, my app makes use of 42 out of the 67 commands in the Lambda client. So yes, when bundled by webpack the resulting protocol module will only include the code needed for these 42 commands.

However, these 42 commands are used in the "main" part of the app (dynamic chunk) whereas upon initial load we only make use of 1 command (initial chunk). Despite this, the initial chunk (which we want to be small and fast) ends up including the entire unsplittable protocol module, due to the way protocols is currently structured.

Hopefully that makes more sense.

[kuhe]

Increasing the number of files has a negative impact on the Node.js runtime, which is our primary usage.

If you only need one request, you can try to templatize it and make it outside of the SDK. Use https://www.npmjs.com/package/@aws-sdk/util-create-request to check the parameters of your request and then make a chunk with only the templatized request function and the @smithy/signature-v4 signer to sign the request.

You could also delegate this work to a server.

[github-actions[bot]]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.