Content-Transfer-Encoding for POST request on multipart message not working

romanolicker commented 1 year ago

Hola everyone,

there seems to be an issue with busboy reading the Content-Transfer-Encoding. I cannot access this information per field although it is present in the raw data.

Please note that i truncated the actual field values.

I verified its presence by outputting the raw received data on nginx level:

Transmitted data on nginx level

0.0.0.0 - - [07/Apr/2023:13:15:27 +0200] "POST /truncated HTTP/2.0" 200 27 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.62" "----------040723131528994\x0D\x0AContent-Disposition: form-data; name=\x22kndnr\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Atruncated\x0D\x0AContent-Disposition: form-data; name=\x22name_kunde\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Atruncated\x0D\x0A----------040723131528994\x0D\x0AContent-Disposition: form-data; name=\x22pw_kunde\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Atruncated\x0A----------040723131528994\x0D\x0AContent-Disposition: form-data; name=\x22version\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Atruncated—————040723131528994\x0D\x0AContent-Disposition: form-data; name=\x22action\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Atruncated\x0D\x0A----------040723131528994\x0D\x0AContent-Disposition: form-data; name=\x22hookurl\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Ahttp://127.0.0.1\x0D\x0A----------040723131528994\x0D\x0AContent-Disposition: form-data; name=\x22warenkorb\x22\x0D\x0AContent-Type: text/plain\x0D\x0AContent-Transfer-Encoding: quoted-printable\x0D\x0A\x0D\x0Atruncated\x0D\x0A----------040723131528994--\x0D\x0A"0.014 0.014 . 297

Output of test script

Field [kndnr]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc
Field [name_kunde]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc
Field [pw_kunde]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc
Field [version]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc
Field [action]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc
Field [hookurl]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc
Field [warenkorb]:
  Content-Type: undefined
  Content-Transfer-Encoding: undefined
  value: trunc

Test script

app.post('/ids', (req, res) => {
  const busboy = Busboy({ headers: req.headers });

  busboy.on('headers', (headers, fieldName, encoding, mimetype) => {
    console.log(`Headers for part [${fieldName}] with mimetype [${mimetype}]`);
    console.log(headers);
  });
    busboy.on('field', (fieldname, val, fieldnameTruncated, valTruncated, encoding, mimetype) => {
      console.log('Field [' + fieldname + ']:');
      console.log('  Content-Type: ' + mimetype);
      console.log('  Content-Transfer-Encoding: ' + encoding);
      console.log('  value: ' + val);
    });
  busboy.on('finish', () => {
    res.send('File uploaded successfully!');
  });
  req.pipe(busboy);
});

This is running on an express server. We were actually using multer to access the multipart data and knowing that busboy is the underlying lib - we tested it using the example code (with the same result).

Versions

node -v
v14.18.0

cat package.json | grep busboy
        "busboy": "^1.6.0",

mscdex commented 1 year ago

I see two issues here:

I'm not sure where you're getting this 'headers' event, there is no such thing in busboy.
You haven't migrated your code to be compatible with busboy v1.x. Please take a look at this to learn about important changes since v1.0.

romanolicker commented 1 year ago

You are right, the encoding is available with:


  const busboy = Busboy({ headers: req.headers });

    busboy.on('field', (name, value, info) => {
      console.log('Field [' + name + ']:');
      console.log('  value: ' + value);
      console.log('  Encoding: ' + info.encoding);
    });
  busboy.on('finish', () => {
    res.send('File uploaded successfully!');
  });
  req.pipe(busboy);
});```

Thanks for the super fast response. 
Then this must be an issue with multer or a version conflict there. I will further investigate.

mscdex / busboy