Handling file uploads with multipart/form-data POST requests

[gramliu]

What are you trying to do? I'm sending form data through a POST request. This contains some text fields and a field containing binary file data but this sends an error when body parsing.

Have you tried this first? Since server is maintained mainly only by Francisco Presencia, it is recommended that you try to fix the problem on your own before asking a question. But if you have tried and couldn't fix it, feel free to ask! Make sure to do this first though:

• [x] Try to find an answer by reading the documentation and tutorials.
• [x] Try to find an answer by searching previous Github issues.
• [x] Try to find an answer by searching the Web.
• [x] Try to find an answer by inspection or experimentation.
• [x] Try to find an answer by asking a skilled friend.
• [x] Try to find an answer by reading the source code.

Ask your question As I stated above, I'm trying to send form data encoded in a POST request with the multipart/form-data header. I believe this gets parsed by default using the body parser. At first I was getting an error that the payload was too large so I increased the limit. After this, I am now stuck with another error saying that there are "too many parameters". I don't think this should be the case since I'm only sending two parameters: text that describes the data and the binary data of type application/octet-stream.

I looked into the documentation for this and this might be due to the fact that express.js's body-parser library "does not handle multipart bodies" and instead recommend alternatives, with formidable as an alternative. I read from the docs that Formidable is being used to parse data. Is there any way to force the server to use Formidable to parse body requests as well?

Additional context (optional) Here's the upper part of my request. The octet-stream extends pretty far since I'm testing with an audio file

[franciscop]

This should work, so there is definitely a bug somewhere, thanks for filling this issue! To try to debug it, let me ask a couple of questions:

• What is the size of the file? I've only tried it with up to few MB
• What is the exact error that you are seeing? Could you copy/paste it please?
• Do you use a form on the front-end to send it? If so, could you share the form code please?
• What is your Node.js version?

I believe this gets parsed by default using the body parser. [...] I read from the docs that Formidable is being used to parse data.

You are right that it's using formidable internally to parse the form data.

Is there any way to force the server to use Formidable to parse body requests as well?

It does parse body requests by default, so there seems to be a bug somewhere.

[gramliu]

• What is the size of the file? I've only tried it with up to few MB

The file is only 485KB.

• What is the exact error that you are seeing? Could you copy/paste it please?

[Fri Jun 26 2020 14:43:29 GMT-0400 (Eastern Daylight Time)] ERROR PayloadTooLargeError: too many parameters
    at queryparse (/home/user/PrivacyStreams/data-centered-privacy/programmingmodel/_cloud/node_modules/body-parser/lib/types/urlencoded.js:151:13)
    at parse (/home/user/PrivacyStreams/data-centered-privacy/programmingmodel/_cloud/node_modules/body-parser/lib/types/urlencoded.js:75:9)
    at /home/user/PrivacyStreams/data-centered-privacy/programmingmodel/_cloud/node_modules/body-parser/lib/read.js:121:18
    at invokeCallback (/home/user/PrivacyStreams/data-centered-privacy/programmingmodel/_cloud/node_modules/raw-body/index.js:224:16)
    at done (/home/user/PrivacyStreams/data-centered-privacy/programmingmodel/_cloud/node_modules/raw-body/index.js:213:7)
    at IncomingMessage.onEnd (/home/user/PrivacyStreams/data-centered-privacy/programmingmodel/_cloud/node_modules/raw-body/index.js:273:7)
    at IncomingMessage.emit (events.js:315:20)
    at endReadableNT (_stream_readable.js:1224:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  type: 'parameters.too.many',
  body: '----------------------------461114968049765978419969\r\n' +
    'Content-Disposition: form-data; name="datatype"\r\n' +
    '\r\n' +
    'audio\r\n' +
    '----------------------------461114968049765978419969\r\n' +
    'Content-Disposition: form-data; name="data"\r\n' +
    'Content-Type: application/octet-stream\r\n' +
    '\r\n' +
    'RIFF$�\x07\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x02\x00D�\x00\x00
    ... (proceeds to output the binary file)
    '----------------------------461114968049765978419969--\r\n'

• Do you use a form on the front-end to send it? If so, could you share the form code please?

I'm encoding the data using the form-data library and sending the request with Axios.

const form = new FormData();
form.append("datatype", datatype);
form.append("data", data);
axios.post(serverAddress, form, { "content-type": "multipart/form-data" });

datatype is a string. data is a Buffer object containing the file bytes.

• What is your Node.js version?

v14.3.0

[franciscop]

Ah interesting, thanks. It seems fairly clear to me what the issue is, but fixing it could break things so I'll need to be testing the changes quite a bit. I'll be posting updates here.

[franciscop]

Hi @gramliu, turns out it wasn't so clear and I've been unable to reproduce this, no matter what I throw at it. Could you please try cloning and testing this repository I prepared please? Let me know if you see the same error with that code, or any other error.

Also, what is the output of console.log(datatype, data) before you do axios.post(serverAddress, form, { "content-type": "multipart/form-data" }); on the front-end? I am a bit concerned about "data is a Buffer object containing the file bytes." Is it possible for you to share some file/code to reproduce it, specially if it's in the repo I prepared?

[franciscop]

Without some follow up info here I won't be able to help

[franciscop]

Closing as inactive