Add multipart/form-data support

[dougwilson]

This is just such a basic type of the web, it's hard to keep ignoring it. Because of the pattern of this module, though, we really cannot support files. But I don't see why we can support it, but just drop files. Thoughts?

[reecefenwick]

As someone who doesn't use multipart form data requests, I can still see the value in it.

It probably makes sense for this module to be able to parse multipart forms.

But is there a need? Are people happy to bring in an extra library to handle it (e.g busboy)?

[dougwilson]

People don't ever seem to be happy :)

[Fishrock123]

Ahh form-data

It probably makes sense for this module to be able to parse multipart forms.

Yes, this would be nice. :)

[julien51]

:+1:

[macmichael01]

Yes this is a highly needed feature. :)

Basically the body parsing options are: 1) field inputs - Works good. 2) file inputs - need something else 3) file and field inputs - need something else as body parser does not know how to process fields when enctype is set to multipart.

[macmichael01]

What can we do to gain some traction on this issue? Would someone like to collaborate on implementing this?

[dougwilson]

Hi @macmichael01 so you are interested in a parser that would drop the files, then?

[macmichael01]

"a parser that would drop the files" - That would be a nice work around for now. Full file upload support would be even better. I'm guessing that are some security obstacles.

[dougwilson]

@macmichael01 , this module will never have full file upload, as it does not fit into the fundamental design. You'll always have to reach for the modules listed in the readme for parsing with file support.

[macmichael01]

@dougwilson That's fine. Should I open a new ticket for suggestion that you made?

[dougwilson]

Should I open a new ticket for suggestion that you made?

I'm not sure. Which suggestions would that be? Are they different from the post at the top of this issue?

[macmichael01]

Is there anything that I can help out with on this? This is a blocker for a project that I am working on. I briefly scanned through the source but I have no idea how you would determine which fields to strip and which to keep. Let me know what I can do to help :)

[dougwilson]

Hi @macmichael01 , I'm' not sure how this is a blocker. Are you not able to use any of the multipart parsers that are listed at the top of the readme? As for what to do, you'll need to integrate one of those parsers into this module as lib/types/multipart.js. Those modules have their own documentation explaining how to determine which fields are text or files and you would just have to drop the file fields and accumulate the text fields into an object.

[macmichael01]

I am using this starter kit: https://github.com/sahat/hackathon-starter. This project uses the CSRF library https://github.com/krakenjs/lusca (which is also using body-parser). Since body-parser doesn't understand multipart, I receive a CSRF error when a multipart form is submitted. I was hoping perhaps there is a way to grab just the _csrf field from the multipart to satisfy CSRF. I'm not entirely sure how I would go about doing this since CSRF relies directly on body-parser. I've tried everything that I can think of but no luck.

[dougwilson]

Ah, gotcha. I'm not really sure what the answer is besides you can definitely do CSRF tokens with multipart if you put all the pieces together yourself. Perhaps that starter kit just needs to be improved? This module has never supported multipart, ever. You can go all the way back to the first 1.0.0 (https://github.com/expressjs/body-parser/tree/1.0.0) on Jan 6, 2014 and it still doesn't support multipart.

Adding multipart to this module is going to be a large task, and not something that is going to happen overnight without either someone implementing, providing a PR, and iterating on it or funding to get it added.

This project uses the CSRF library https://github.com/krakenjs/lusca (which is also using body-parser).

I'm not sure what you mean, as lusca does not have a dependency on this module at all (in fact, it actually has zero dependencies).

[macmichael01]

I figured something like this would be a major undertaking.

Let me re-investigate my issue again tonight, perhaps lusca was not the issue after all.

[macmichael01]

It looks like body-parser is used with lusca. it's a required configuration in your app.js. lusca refers to the req.body for the _csrf token key. Since body-parser doesn't support multipart body data, req.body is {}. I really don't see an easy work around other than ditching lusca for something else and using something like node formidable. Just thought I'd share. Thanks for your help @dougwilson!

[franciscolourenco]

+1 for this, specially since XMLHttpRequest 2 now supports FormData()

Currently using multer just for this purpose. This way it doesn't accept any files, but doesn't look very pretty:

formdataParser = require('multer')().fields([])
app.use(formdataParser)

[0x70b1a5]

This is a blocker for my current project, too. It's been two years on this issue, so it should be closed if it isn't going to get fixed. It's cruel to keep us in suspense.

[dougwilson]

Hi @0x70b1a5 it was my impression from the conversation above that a PR would have been coming, but none ever materialized. You're absolutely welcome to tackle this based on the implementation discussed above. The issue is open so people are aware that it is something they could work on and contribute towards if they are so inclined to do so. I personally simply use the multipart parsers listed in the README directly and don't have any issues accepting multipart data; the issue was opened because I kept closing the requests for this, so this would be the tracking request. If I closed this, I'm sure new issues will start popping up just like they used to. If you have a better solution, please let me know.

[hjs-robin]

oh ~ i'm not the only one~

[chrisdothtml]

For anyone else stumbling across this issue, express-form-data does the job pretty well

[chrisdothtml]

@reecefenwick

Are people happy to bring in an extra library to handle it (e.g busboy)?

I think the main annoyance is that we already are bringing in an extra library (in addition to express) to parse our body (body-parser). Having to bring yet another library in because the body parser doesn't support all types of bodies is a pain

[abinhho]

If we uses serverless and a node server like NestJs. The request body will be converted to Buffer and forward to nodejs server, if we don't have multipart parser, we can not read the file upload.

[Deshanm123]

use Formidable to handle "multipart/form-data".

[jimmywarting]

simple dependency free workaround:

app.post('/upload', async (req, res) => {
  const formData = await new Response(req, { headers: req.headers }).formData()
  const files = formData.getAll('files')
})

(require NodeJS v18+)