anacronw / multer-s3

multer storage engine for amazon s3
MIT License
660 stars 190 forks source link

Is this streaming in chunks? #112

Closed benspring closed 3 years ago

benspring commented 5 years ago

Is this using streams/chunks to upload my file?

I need multerS3 to upload my file in chunks under 100mb. I added partSize but it doesn't seem to work as I get an error by Cloudflare for requests over 100mb.

const upload = multer({
    storage: multerS3({
        s3: s3,
        options: { partSize: 5 * 1024 * 1024, queueSize: 10 },
        bucket: config.aws.s3Bucket,
        contentType: multerS3.AUTO_CONTENT_TYPE, //auto detecting mime type for s3 upload
        key: async function (req, file, cb) {
            console.log("file mimetype is " + file.mimetype);
            console.log("contentType is " + multerS3.AUTO_CONTENT_TYPE)
            file.originalname = func.makeFilenameUnique(file.originalname)
            file_name = file.originalname;
            console.log("first file name is " + file_name)
            cb(null, file_name);
        }
    }),
....
LinusU commented 5 years ago

I think that the S3 client chooses the part size by itself 🤔

There is no config option named options in this library, but you could configure that on the S3 instance that you are passing in.

const s3 = new S3({ partSize: ... })
const upload = multer({ storage: multerS3({ s3, ... }) })