Large file download

[mmajcica]

Would you be so kind to provide us an example on how to efficiently download a large file with the http client?

It may sound trivial as a question, however for someone starting with Node and TS, aiming to build an efficient build task, this could be very useful.

Thanks

[bryanmacfarlane]

https://github.com/Microsoft/typed-rest-client/blob/master/test/httptests.ts#L85

The key is it pipes a stream to a file (or another stream) so matter how large the file is, it will stream it chunk by chunk.

[bryanmacfarlane]

Closing since question answered. Let us know if you have issues.

[mmajcica]

Thanks @bryanmacfarlane I'll try this right away!

[bryanmacfarlane]

The other key is the server - hopefully it's streaming the contents but most should. Most servers don't timeout out as long as the stream is moving data.

[bryanmacfarlane]

NP :)

[mmajcica]

The example you pointed will not work. We are not awaiting for the stream write to finish.

I'll post the example and the solution, as it may help others. Check if you agree.

//
// Http get piping to another stream
// response message is an IncomingMessage which is a stream
//
cm.heading('get request and pipe stream');
let file: NodeJS.WritableStream = fs.createWriteStream(sampleFilePath);
(await httpc.get('https://httpbin.org/get')).message.pipe(file);
body = fs.readFileSync(sampleFilePath).toString();
cm.outputHttpBinResponse(body);

In this case we are not awaiting the file stream to finish. I found out that the following approach is necessary:

const file: NodeJS.WritableStream = fs.createWriteStream(filePath);
const response = await this.client.get(url);

await new Promise((resolve, reject) => {
    file.on("error", reject);
    file.on("finish", resolve);

    response.message.pipe(file);
});

If you have a better suggestion, I'll be happy to see it.

What if we implement another method on IHttpClientResponse like saveBodyToFile?