[Bug] Getting Error while uploading file "Unexpected end of form at Multipart._final

[sohebcakewalk]

I am using angular as a front end and Firebase cloud function (Express.js) REST API as backend.

Busboy: 1.6.0 (also tried 1.5.0) Node: 16

export async function uploadFile(req: Request) {
    return new Promise<any>((resolve, reject) => {
        const bb = busBoy({ headers: req.headers });
        let uploadData = { file: "", type: "" };
        const uploadId = db.collection("_").doc().id;
        let fileExt: string = "";
        let newFileName: string = "";
        const uuidToken = uuidv4();
        const bucketName = admin.storage().bucket().name;
        let fbCollTicketData: any;

        // busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
        bb.on('file', (name, file, info) => {
            console.log("Execution started, file Block");
            const { filename, encoding, mimeType } = info;
            fileExt = filename.substring(filename.lastIndexOf('.'));
            newFileName = `${uploadId}${fileExt}`;
            console.log(`File [${name}]: filename: ${filename}, encoding: ${encoding}, mimetype: ${mimeType}`);
            const filepath = path.join(os.tmpdir(), newFileName);
            uploadData = { file: filepath, type: mimeType };
            const fstream = fs.createWriteStream(filepath);
            file.pipe(fstream);
            console.log("Execution Ended, file Block");
            // file.on('data', function (data) {
            //     console.log('File [' + fieldname + '] got ' + data.length + ' bytes');
            // });
        });
        //busboy.on('field', function (fieldname, val, fieldnameTruncated, valTruncated, encoding, mimetype) {
        bb.on('field', (name, val, info) => {
            console.log("Execution started, field Block");
            //console.log('Field [' + fieldname + ']: value: ' + val);
            console.log(`Field [${name}]: value: %j`, val);
            if (name == 'data' && val) {
                fbCollTicketData = <any>JSON.parse(val);
            }
            console.log("Execution Ended, field Block");
        });
        bb.on('error', (e) => {
            console.log("Execution started, error Block");
            console.log(e);
            console.log("Execution Ended, error Block");
        });
        bb.on('close', () => {
            console.log('Done parsing form!');
            admin.storage().bucket().upload(uploadData.file, {
                destination: `${STORAGE.CLEANINGPICS}/${newFileName}`,
                resumable: false,
                contentType: uploadData.type,
                metadata: {
                    metadata: {
                        //cacheControl: 'public, max-age=31536000',
                        firebaseStorageDownloadTokens: uuidToken,
                    }
                }
            }).then(async (resp) => {
                resolve(fbCollTicketData);
                fs.unlinkSync(uploadData.file);
            }).catch(err => {
                reject(err);
            })
        });
        req.pipe(bb);
    });
};

In cloud function when i check Functions log it shows me Error "Unexpected end of form at Multipart._final"

@mscdex - Please help.

[mscdex]

A lot of those "cloud function" services will buffer the contents of the request somewhere, so normal node.js http request stream data isn't available and is most likely what is causing the error. If that's the case you'll need to find out where the data is being stored and then bb.end(data) instead of req.pipe(bb).

[TheBunyip]

I hit this. Thanks for the insight @mscdex.

[SourceBoy]

This works with GCP Cloud Functions:

  await new Promise((resolve, reject) => {
    bb.once('close', resolve).once('error', reject).on('file', (name, file_stream, info) => {
      file_stream.resume();
      console.dir({ name, file_stream, info }, { depth: null });
    }).end(request.rawBody);
  });

  return response.sendStatus(200);

[astoiccoder]

Hit the same error with a Nextjs application deployed with Vercel. Found the solution here https://github.com/vercel/next.js/discussions/11634#discussioncomment-1865018 Needed to add the following to my api file in my nexjs application

export const config = {
  api: {
    bodyParser: false,
  },
};

[hmahmoud3]

This works with GCP Cloud Functions:

  await new Promise((resolve, reject) => {
    bb.once('close', resolve).once('error', reject).on('file', (name, file_stream, info) => {
      file_stream.resume();
      console.dir({ name, file_stream, info }, { depth: null });
    }).end(request.rawBody);
  });

  return response.sendStatus(200);

This solved it for me, but I would love to know why it works.

It seems that doing .once and chaining .on plays well with GCP. Do you have any idea why that's necessary?

[SourceBoy]

This works with GCP Cloud Functions:

  await new Promise((resolve, reject) => {
    bb.once('close', resolve).once('error', reject).on('file', (name, file_stream, info) => {
      file_stream.resume();
      console.dir({ name, file_stream, info }, { depth: null });
    }).end(request.rawBody);
  });

  return response.sendStatus(200);

This solved it for me, but I would love to know why it works.

It seems that doing .once and chaining .on plays well with GCP. Do you have any idea why that's necessary?

It's the .end(request.rawBody) part that makes it work:

A lot of those "cloud function" services will buffer the contents of the request somewhere, so normal node.js http request stream data isn't available and is most likely what is causing the error. If that's the case you'll need to find out where the data is being stored and then bb.end(data) instead of req.pipe(bb).

[GautheyValentin]

I wrote a nest js service that implement the solution of @SourceBoy You can find on stackoverflow

[PeterPlevko]

I wrote a nest js service that implement the solution of @SourceBoy You can find on stackoverflow

Yes this fixed it thank you very much <3

[BossySmaxx]

This works with GCP Cloud Functions:

  await new Promise((resolve, reject) => {
    bb.once('close', resolve).once('error', reject).on('file', (name, file_stream, info) => {
      file_stream.resume();
      console.dir({ name, file_stream, info }, { depth: null });
    }).end(request.rawBody);
  });

  return response.sendStatus(200);

This solved it for me, but I would love to know why it works.

It seems that doing .once and chaining .on plays well with GCP. Do you have any idea why that's necessary?

instead of resolving the function in busboy.on('close', resolve), resolve it like below if you wanna process the file further immediately:

const fileStream = fs.createWriteStream(`<yout file name>`);
file.pipe(fileStream);
fileStream.on('close', () => {
        console.log("file write completed: closing")
        resolve(filedetails);
});

because on busboy.on('close', resolve) file is still not created in it's entirety/fully, I had this this issue where I was reading the file using xlsx and it was reading the file but the file content was not in it's entire form so I was getting wrong data so I had to use the fileStream.on('close', resolve) inside busboy.on('file', () => {...}) and then I could use the file for further processing because the file is whole now in the memory.