search

kevva / download

Download and extract files
MIT License
1.28k stars 200 forks source link

RangeError: Invalid string length with stream #219

Open FrederickEngelhardt opened 2 years ago

FrederickEngelhardt commented 2 years ago

When using a stream the response attempts to check the size. For files larger than 4GB this creates an error.

RangeError: Invalid string length
    at Array.join (<anonymous>)
    at PassThrough.stream.getBufferedValue (/root/servers/file-queue-server/node_modules/download/node_modules/get-stream/buffer-stream.)
    at /root/servers/file-queue-server/node_modules/download/node_modules/get-stream/index.js:44:23
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)

Is this a limitation of the download library? The download file still downloads but the error is always thrown at > 99% of the download. Is there a config option to disable this buffer size check?

Sample of how I'm downloading a file.

const file = createWriteStream(filePath, {})
const dl = download(url)

The file can download. Tested with files ranging from 50mb >= 100GB the error will always occur for files >4GB due to it attempting to compute a buffer larger than the maximum safe memory.