S3 commands hang after uploading an empty stream using `@aws-sdk/lib-storage`

martinslota commented 2 weeks ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

Immediately ending the stream feeding the Upload utility from @aws-sdk/lib-storage sometimes leads to the S3 client getting "locked up": a subsequent command sent through it never finishes.

SDK version number

@aws-sdk/client-s3@3.637.0, @aws-sdk/lib-storage@3.637.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.15.0

Reproduction Steps

Detailed steps to reproduce are listed in https://github.com/martinslota/aws-sdk-lib-storage-race. Those steps run against Localstack, but the same code also works against actual AWS S3.

Observed Behavior

After uploading an empty stream, commands sent through the S3 client never finish.

Expected Behavior

After uploading an empty stream, commands sent through the S3 client should continue to work as usual.

Possible Solution

No response

Additional Information/Context

No response

aBurmeseDev commented 1 week ago

Hi @martinslota - thanks for reaching out.

Before proceeding, I'd like to request your assistance in confirming that the issue persists when running the code against the actual AWS service as the example you shared appears to be using a LocalStack endpoint (https://s3.localhost.localstack.cloud:4566). While LocalStack is a great tool for local development and testing, it's always a good practice to validate the behavior against the real AWS service before raising a formal issue.

Additionally, to better understand the underlying interactions between your code and the AWS SDK, please add middlewareStack to S3 client in your code to get request logs. This middleware will log the context, request input and response output for each AWS SDK operation, providing valuable insights into the data being sent and received.

client.middlewareStack.add(
  (next, context) => async (args) => {
    console.log("AWS SDK context", context.clientName, context.commandName);
    console.log("AWS SDK request input", args.input);
    const result = await next(args);
    console.log("AWS SDK request output:", result.output);
    return result;
  },
  {
    name: "MyMiddleware",
    step: "build",
    override: true,
  }
);

Looking forward to your response!

martinslota commented 1 week ago

Hi @martinslota - thanks for reaching out.

👋

Before proceeding, I'd like to request your assistance in confirming that the issue persists when running the code against the actual AWS service as the example you shared appears to be using a LocalStack endpoint (https://s3.localhost.localstack.cloud:4566). While LocalStack is a great tool for local development and testing, it's always a good practice to validate the behavior against the real AWS service before raising a formal issue.

Yes, the same happens against the actual AWS S3 service. I also adapted https://github.com/martinslota/aws-sdk-lib-storage-race so it runs against S3 by default.

Additionally, to better understand the underlying interactions between your code and the AWS SDK, please add middlewareStack to S3 client in your code to get request logs. This middleware will log the context, request input and response output for each AWS SDK operation, providing valuable insights into the data being sent and received.
client.middlewareStack.add(
  (next, context) => async (args) => {
    console.log("AWS SDK context", context.clientName, context.commandName);
    console.log("AWS SDK request input", args.input);
    const result = await next(args);
    console.log("AWS SDK request output:", result.output);
    return result;
  },
  {
    name: "MyMiddleware",
    step: "build",
    override: true,
  }
);
Looking forward to your response!

I added the code above but additionally redacted the Body field from response output. The output looks like this.

Previously I also tried with an hour-long timeout instead of 5 seconds and I still saw the script fail.

aws / aws-sdk-js-v3