uNetworking / uWebSockets.js

μWebSockets for Node.js back-ends :metal:
Apache License 2.0
7.84k stars 569 forks source link

Multipart images are broken. #1084

Closed def-roth closed 1 month ago

def-roth commented 1 month ago

The following code produces results for arbitrary files exept for images. I'm running around in circles here and I bet it's just me doing something wrong. Different types of buffer allocations have been tried. Storing them in an Array and concatenating them (doesn't work) or calling buffer.toString(). At least this works as minimum reproducible I guess.


const {App : UWS_APP,} = require('uWebSockets.js');
const fs = require('fs');

const port  = 56122;

const ssApp = UWS_APP({});

ssApp
  .post('/*', (res, req) => {

    let file = Buffer.alloc(0);

    let filename = req
      .getHeader("content-disposition")
      ?.split("filename=\"")
      ?.[1]
      ?.slice(0, -1)

    res.onData((chunk, isLast) => {

      file = Buffer.concat([file, Buffer.from(chunk)]);

      if (isLast) {

        fs.writeFileSync(filename, file);
        res.end('ok done');

      }

    });

    res.onAborted(() => {

    });
  })
  .listen(port, ()=>{});

ssApp
  .options("/*", (res) => {

    res.onAborted(() => {});

    res.cork(() =>
      res.writeHeader("Access-Control-Allow-Origin", "*")
        .writeHeader("Access-Control-Allow-Methods", "*")
        .writeHeader("Access-Control-Allow-Headers", "*")
        .endWithoutBody()
    )

  });
e3dio commented 1 month ago

Multipart uploads need to be parsed, have you tried getParts

def-roth commented 1 month ago

If I read the getParts interface correct that is problematic as it returns array buffers which become memory intensive when dealing with huge files.

e3dio commented 1 month ago

There are streaming multipart parsers you would need to find one that works for you npmjs.com/search?q=multipart

def-roth commented 1 month ago

All good. This was an error on my end. I had to implement parsing myself as the requirements are pretty special, but working with the stream was a charm. Thanks for the hint though!

uNetworkingAB commented 1 month ago

getParts is not very well designed like you mention it uses in-memory buffers rather than streaming. This interface was added for a customer who wanted it like this. I would rather have streams but, yeah..

def-roth commented 1 month ago

I can add a PR to the examples as a starting point how to solve this. Yet I think there is no silver bullet here, neither memory nor disk are optimal and both have huge foot gun potential. Although most developers will never need a filesize larger than 5mb.