aws / aws-sdk-js-v3

Modularized AWS SDK for JavaScript.
Apache License 2.0
2.96k stars 557 forks source link

await Upload.done() never returns if stream is (still) empty #6118

Closed kdeberk closed 1 month ago

kdeberk commented 1 month ago

Checkboxes for prior research

Describe the bug

I have a tiny program that generates a stream with PassThrough and calls await upload.done() with that stream before anything is written to that stream. The await never resolves and the program terminates without logging anything else.

No exception is thrown. It seems as if the program gets stuck and then simply exits.

SDK version number

@aws-sdk/lib-storage@3.578.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v18.19.0

Reproduction Steps

#!/usr/bin/env npx node

const { PassThrough } = require("stream");
const { Upload } = require("@aws-sdk/lib-storage");
const { S3Client } = require("@aws-sdk/client-s3");

function getS3Stream() {
    const client = new S3Client({ region: "eu-west-1" });
    const pt = new PassThrough();
    const upload = new Upload({
    client,
    params: { Bucket: "some-bucket", Key: "some-key", Body: pt },
    });

    const promise = new Promise((resolve, reject) => {
    upload.done()
        .then(() => {
        console.log("upload.done() resolved"); // not logged
        resolve()
        })
        .catch((err) => {
        console.log("upload.done() threw error"); // not logged
        reject(err)
        })
    });

    return [pt, promise]
}

async function main() {
    const [stream, promise] = getS3Stream();

    try {
    console.log("Before"); // only thing that is logged
    await Promise.all([
        (resolve, reject) => {
        try {
            stream.write("Hello, World!");
            stream.end();
            resolve();
        } catch(err) {
                    console.error(err); // not logged
            reject(err);
        }
        },
        promise,
    ]);
    console.log("After");
    } catch(err) {
    console.error(err); // not logged
    throw err;
    } finally {
    console.log("finally"); // not logged
    }
}

main()
    .then(() => {
    console.log("Done") // not logged
    })
    .catch((err) => {
    console.error("Error:", err) // not logged
    process.exit(1);
    })

async function main() {
    const [stream, promise] = getS3Stream();

    await Promise.all([
    (resolve, reject) => {
        stream.write("Hello, World!");
        stream.end();
        resolve();
    },
    promise,
    ]);
    console.log("Finished uploading");
}

main()
    .then(() => {
    console.log("Done") // not logged
    })
    .catch((err) => {
    console.error("Error:", err) // not logged
    process.exit(1);
    })

Observed Behavior

The program terminates, only logged Before. It terminates with exit code 0.

Expected Behavior

Other things being logged other than Before. If the upload fails, then I expect to see some exception with information about why it is invalid.

Possible Solution

No response

Additional Information/Context

No response

kdeberk commented 1 month ago

I realized my mistake shortly after posting this.

In the example I posted, the first object passed to Promise.all is invalid.

In the real code on which I based the above one, I forgot to close the stream after writing it. Once the stream was closed, node no longer detected a stuck execution runtime and the program logged all the remaining strings.

To the maintainers, you can delete this issue if you want.

github-actions[bot] commented 3 weeks ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.