search

richardgirges / express-fileupload

Simple express file upload middleware that wraps around busboy
MIT License
1.52k stars 259 forks source link

Can't upload a 34-36 Gb file #370

Closed LiveGobe closed 4 months ago

LiveGobe commented 4 months ago

Hi, when I'm trying to upload a 34-36 Gb files using express-fileupload, the Node process starts using up all memory and finally gives this error:

SystemError: A system error occurred: undefined returned undefined (undefined)
    at new SystemError (node:internal/errors:257:5)
    at new NodeError (node:internal/errors:368:7)
    at node:internal/fs/streams:446:10
    at FSReqCallback.wrapper [as oncomplete] (node:fs:936:5)

Files of sizes 2-4 Gb upload just fine. Using debug option I got these messages:

Express-file-upload: Uploading file->fg-01.bin, bytes:9089187689...
Express-file-upload: Error write temp file: SystemError [ERR_SYSTEM_ERROR]: A system error occurred:
Express-file-upload: Cleaning up temporary file D:\livegobe-website\tmp\tmp-1-1708509538028...
SystemError: A system error occurred: undefined returned undefined (undefined)
    at new SystemError (node:internal/errors:257:5)
    at new NodeError (node:internal/errors:368:7)
    at node:internal/fs/streams:446:10
    at FSReqCallback.wrapper [as oncomplete] (node:fs:936:5)
Express-file-upload: Cleaning up temporary file D:\livegobe-website\tmp\tmp-1-1708509538028 done.

How I'm using express-fileupload with Express router:

router.route("/filestorage/file").post(fileUpload({ useTempFiles: true, tempFileDir: path.join(process.cwd(), "tmp"), defParamCharset: "utf-8", uploadTimeout: 0, debug: true }), (req, res) => {

What could be the cause for that?

LiveGobe commented 4 months ago

I apologize for my negligence, it turned out that the problem was in nginx with request buffering being turned on by default.