expressjs / multer

Node.js middleware for handling `multipart/form-data`.
MIT License
11.47k stars 1.04k forks source link

Unexpected end of form at Multipart._final #1144

Open amulagrawal02 opened 1 year ago

amulagrawal02 commented 1 year ago

Getting this error while using multer with react.js and node.js

Unexpected end of form
at Multipart._final (E:\nearme\backend\node_modules\busboy\lib\types\multipart.js:588:17)
at callFinal (internal/streams/writable.js:610:10)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
storageErrors: []
}

react.js file

const fd = new FormData();
fd.append("profile", data.profile);
fd.append("name", data.name);
fd.append("email", data.email);
fd.append("password", data.password);

const res = await axios.post("http://localhost:8000/signup/data", fd, {
  headers: { "Content-Type": "multipart/form-data" },
});

node js file:

const multer = require("multer");
const storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, "../public");
},
filename: function (req, file, cb) {
const uniqueSuffix = Date.now() + "-" + Math.round(Math.random() * 1e9);
cb(null, file.fieldname + "-" + uniqueSuffix);
},
});

const upd = multer({ storage: storage }).single("profile");
router.post("/data", (req, res) => {
upd(req, res, function (err) {
if (err instanceof multer.MulterError) {
console.log("A Multer error occurred when uploading");
} else if (err) {
console.log("An unknown error occurred when uploading", err);
}
});
});

For more details about project refer this https://github.com/amulagrawal02/nearme

AhmedSaidi99 commented 1 year ago

I have the same issue :(

PedroEliasCS commented 1 year ago

I have the same issue

LinusU commented 1 year ago

Could someone try and reproduce this directly with Busboy, and submitting an issue to them?

bassamanator commented 1 year ago

I'm also having the same issue.

krivonosov11 commented 1 year ago

I also have the same issue :(

LinusU commented 1 year ago

@bassamanator @krivonosov11 could you try and reproduce this directly with Busboy, and submit an issue to them?

exeldtarkus commented 1 year ago

I think, the problem is in the express and body-parser module, I just eliminated it

app.use(bodyParser. text({type: '/'}));

and it works !

agborkowski commented 1 year ago

same for me ;/ maybe downgrade ?

PedroEliasCS commented 1 year ago

I did several things to try to solve the problem, but the only one that solved it was to migrate to Busboy. Example: https://github.com/StarsEncodingTechnology/Sped-rejustes/blob/main/functions/src/middleware/upload/spedtxt.ts.

Nono-Song commented 1 year ago

same here

catosaurusrex2003 commented 1 year ago

i also have the same issue and nothing here seems to work

svarup commented 1 year ago

same here

tgroshon commented 1 year ago

We had a similar issue on 1.4.5-lts.1 and 1.4.4-lts.1. Downgrading to 1.4.3 fixed the issue for us. NOTE: 1.4.3 has an outstanding security issue due to dicer.

malekhammou commented 1 year ago

Same issue...

s36ast14n commented 1 year ago

We had a similar issue on 1.4.5-lts.1 and 1.4.4-lts.1. Downgrading to 1.4.3 fixed the issue for us. NOTE: 1.4.3 has an outstanding security issue due to dicer.

Can confirm, that it works with a downgrade to 1.4.3 ... Btw: with the current version of busboy 1.6.0 the same error occurs.

malekhammou commented 1 year ago

My problem was this line:

   app.use(
    bodyParser.urlencoded({
      extended: false,
    })
  );

replaced it by

   app.use(
    bodyParser.urlencoded({
      extended: true,
    })
  );

and it worked.

archisvaze commented 1 year ago

Can confirm, that it works with a downgrade to 1.4.3 ... Btw: with the current version of busboy 1.6.0 the same error occurs.

Yes! downgrading to multer 1.4.3 worked! thanks

officialkevinbrian commented 1 year ago

What caused this too me was because I was using multiple middleware on the same route, I was using global middleware and then applied another middleware in sub route. so this caused conflits. #1118

glespinosa commented 1 year ago

I think, the problem is in the express and body-parser module, I just eliminated it

app.use(bodyParser. text({type: '/'}));

and it works !

@exeldtarkus what version of your express,multer are you using? Cause I did this but same error persists!

aygunoz commented 1 year ago

Yes! downgrading to multer 1.4.3 worked! thanks great

Hamza-Sajid commented 1 year ago

Kindly solve this issue the only solution i found is downgrading to version @1.4.2, but this version contains a severe levels of security issues.

benedekdaniel commented 1 year ago

I'm having the same issue, there should be a more convenient way than downgrading (I'm on 1.4.5-lts.1), in my case I do have body-parser as it's needed on other routes so that might collide with multer? I tried setting bodyparser.urlencoded({extended:true}) before the multer middleware as mentioned here but that didn't solve it for me, it's strange because I can't reproduce this locally, this only happens on our live environments. Kindly asking for help on this one, here is the full issue: https://stackoverflow.com/questions/76064309/unexpected-end-of-form-at-callfinal-error

LbhFront-end commented 11 months ago

nestjs,@nestjs/platform-express^9.0.0 use multer 1.4.4-lts.1,downgrading 1.4.3 or add bodyParser.urlencoded bodyParser.type don't work.How to fixed It

Jihed-khelifi commented 11 months ago

I was able to fix the issue by removing bodyparser and replacing with its express counterpart

app.use(express.json());

and then you can route handler/controller you find all the form-data fields in the body including file(s) here's an example:

suppose we have this route

routes.js router.post("/dummy-route", upload.any(), RouteController)

upload returned from multer()

RouteController.js

const RouteController = async (req, res) => {
    const files = req.files
    const { name, email, password } = req.body
}
md-fahad-ali commented 11 months ago

Downgrading may not be the good idea npm WARN deprecated multer@1.4.3: Multer 1.x is affected by CVE-2022-24434. This is fixed in v1.4.4-lts.1 which drops support for versions of Node.js before 6. Please upgrade to at least Node.js 6 and version 1.4.4-lts.1 of Multer. If you need support for older versions of Node.js, we are open to accepting patches that would fix the CVE on the main 1.x release line, whilst maintaining compatibility with Node.js 0.10.

PeterNguyen-BK commented 11 months ago

I have the same issue, I cannot do the downgrade because the security issue, also try another solution in this but cannot work. Did anyone solve the problem without downgrade? Im using nestjs.

md-fahad-ali commented 11 months ago

I just make another method to upload files that are not related to Multer instead you can use existing packages to do that

//handler.js using next-connect

import multiparty from "multiparty";
import bodyParser from "body-parser";
import handler from "./handler";
import { promises as fs } from "fs";

handler.use(async (req, res, next) => {
  const filePath = path.join(process.cwd(), 'public', 'uploads');
  console.log(filePath);
  const form = new multiparty.Form({ uploadDir: filePath, });

  await form.parse(req, function (err, fields, files) {
    req.body = fields;
    req.files = files;
    next();
  });
});

// Define the route handler
handler.post(async (req, res) => {
  // console.log(req.body, req.files);
  res.json(req.files);
  // res.json(fields, files);
});
//frontend request demo 
async function submitForm(e) {
    e.preventDefault();
    const company_slug = slugify(coName.toLowerCase());
    // console.log(img, coName, company_slug, description, option);

    const formData = new FormData();
    formData.append("name", coName);
    formData.append("description", description);
    formData.append("image", ufile);
    formData.append("company_slug", company_slug);
    formData.append("option", option);
    formData.append("_csrf",  props?.props?.data,);

    try {
      const result = await axios.post(
        `/api/create`,
        formData,
        {
          withCredentials: true,
          headers: {
            Accept: "application/json",
            "Content-Type":"multipart/form-data",
            "xsrf-token": props?.props?.csrf,
          },
        }
      );

      console.log(result.data);
    } catch (error) {
      console.log(error);
    }
  }

This is a demo that you can run and get the result also after upload you can check the file if it was ok and then rename otherwise delete this or something else here in their full doc of https://www.npmjs.com/package/multiparty

Note: Ignore another thing that I use in my frontend request demo just see how I send the request :)

CelineAma commented 11 months ago

Please, I need help because I have an issue with the same error. I am testing with postman > form-data.

//the controller file; exports.createPost = async (req, res, next) => { try {

console.log("Received request:", req.body);

const { id } = req.params;
// const { userId, body, files } = req;
const { body } = req.body;
const pictures = req.files;
const userId = req.userId;

// console.log("User ID:", userId);
// console.log("Community ID:", id);

const community = await Community.findById(id);

if (!community) {
  const error = new Error("Community not found");
  error.statusCode = 404;
  throw error;
}

if (!community.members.includes(userId)) {

  // console.log('Community members:', community.members);
  // console.log('User ID:', userId);

  const error = new Error("User not joined");
  error.statusCode = 400;
  throw error;
}

let compressedPictures = [];

if (pictures && pictures.length > 0) {
  const compressedImages = await Promise.all(
    files.map(async (file) => {
      const compressedBuffer = await sharp(file.buffer)
        .resize(800)
        .toBuffer();
      return {
        ...file,
        buffer: compressedBuffer,
      };
    })
  );

  // console.log("Compressed Images:", compressedImages);

  compressedPictures = compressedImages.map((file) => file.location);
}

// console.log("Pictures before assignment:", pictures);

const post = new Post({
  ...body,
  userId,
  communityId: id,
  pictures: compressedPictures,
});

// console.log("Post before save:", post);

await post.save();

return res.status(201).json({ message: "Post created successfully", post });

} catch (err) { if (!err.statusCode) { err.statusCode = 500; } return next(err); } };

tnemelclement commented 10 months ago

For NextJS, you just have to add this code at the top of the file.

export const config = { api: { bodyParser: false } };

TaherJerbi commented 10 months ago

For people still experiencing this issue, I found a similar bug when using multer or busboy in a Google Cloud Function with an express router.

I found an answer that fixed the issue for me here: https://github.com/mscdex/busboy/issues/296#issuecomment-1107458297

A lot of those "cloud function" services will buffer the contents of the request somewhere, so normal node.js http request stream data isn't available and is most likely what is causing the error. If that's the case you'll need to find out where the data is being stored and then bb.end(data) instead of req.pipe(bb).

I believe multer is using req.pipe in the middleware, which may be causing the issue. I am still trying to make multer work with our Cloud Function with no luck. So I am using busboy with bb.end(req.rawBody) instead.

Hope this helps!

public uploadController = async (req: Request, res: Response, next: NextFunction) => {
  const bb = busboy({ headers: req.headers });

  bb.on('file', (name, file, info) => {
    const { filename, encoding, mimeType } = info;
    console.log(
      `File [${name}]: filename: %j, encoding: %j, mimeType: %j`,
      filename,
      encoding,
      mimeType
    );
    file.on('data', (data) => {
      console.log(`File [${name}] got ${data?.length} bytes`);
    }).on('close', () => {
      console.log(`File [${name}] done`);
    });
  });
  bb.on('field', (name, val, info) => {
    console.log(`Field [${name}]: value: %j`, val);
  });
  bb.on('close', () => {
    console.log('Done parsing form!');
    res.status(200).send({ message: 'Done parsing form!' });
  });

  bb.end(req.body);
};
didiercolens commented 6 months ago

if you're reading this because you get an uncaughtException "Unexpected end of form" the root cause is that the multer code removes the busboy error listener too early, the fix is in https://github.com/expressjs/multer/pull/1177. It would be great the Multer team merges it as it is trivial to repro the issue.

NikhilDevAS commented 6 months ago

i also have same issue in nextjs application

nerder commented 3 months ago

For people using NestJS:

  @Post('upload-controller')
  async uploadController(
    @Req() req: Request,
    @Res() res: Response,
  ): Promise<void> {
    try {
      const bb = busboy({ headers: req.headers });
      let fileContent = '';

      bb.on('file', (name, file, info) => {
        const { filename, encoding, mimeType } = info;
        console.log(`File [${name}]: filename: %j, encoding: %j, mimeType: %j`, filename, encoding, mimeType);
        file
          .on('data', data => {
            console.log(`File [${name}] got ${data?.length} bytes`);
            fileContent += data.toString();
          })
          .on('close', () => {
            console.log(`File [${name}] done`);
          });
      });

      bb.on('close', () => {
        res.status(201).send({ message: 'File processed successfully' });
      });

      bb.end(req.body);
    } catch (e) {
      console.log(e);
      throw internalServerError(`Error`, e as Error);
    }
  }
raphaelcomph-dev commented 3 months ago

NestJS users, please check this answer. https://stackoverflow.com/questions/76220138/handling-multipart-form-data-post-with-nest-in-cloud-functions-firebase

I'm using "busboy": "^1.6.0", and "@nestjs/platform-express": "^10.0.0", It worked for me local and in GCP Firebase Functions.

nerder commented 2 months ago

Is there any plan to fix this? It's just quite ugly having to use busboy for this compare to a nice decorator

agrawalrohant commented 1 month ago

Any possible fix for express users?

mahdisafigholi commented 1 month ago

when i use @socket.io/sticky to load distribution , its happend,because File chunks were distributed between several instances and were not uploaded properly.

borhanedinee commented 2 weeks ago

my problem was that i was using multer as global middleware with this line in app.js app.use(upload.any())

and in the routes that expect to recieve form-data / files also i was calling the upload as follows adminRouter('route' , upload.single('file') , controller)

and after removing the upload.any from the app.js and only calling it or upload.single() on specific routes that expect to accepet formdata and files the problem is solved

AbdulrahmanTareqSaad commented 1 week ago

In my case the problem was that express-openapi-validator middleware conflicts with multer and the solution is here