node-formidable / formidable

The most used, flexible, fast and streaming parser for multipart form data. Supports uploading to serverless environments, AWS S3, Azure, GCP or the filesystem. Used in production.
MIT License
7k stars 680 forks source link

How to wait for the result of writeHandler in a formidable file upload request? #977

Open itinance opened 2 months ago

itinance commented 2 months ago

I want to use formidable to upload files directly to S3. Following the official example it works just fine. However, I want to await the upload process and return the final URL of the uploaded file inside of the Http-Request to my client. And this seems to be a a challenge due to the limited nature of how fileWriteStreamHandler needs to be typed.:

const s3Client = new AWS.S3({
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_KEY,
  },
});

const uploadStream = (file) => {
  const pass = new PassThrough();
  s3Client.upload(
    {
      Bucket: 'demo-bucket',
      Key: file.newFilename,
      Body: pass,
    },
    (err, data) => {
      console.log(err, data);
    },
  );

  return pass;
};

const server = http.createServer((req, res) => {
  if (req.url === '/api/upload' && req.method.toLowerCase() === 'post') {
    // parse a file upload
    const form = formidable({
      fileWriteStreamHandler: uploadStream,
    });

    form.parse(req, () => {
      res.writeHead(200);
      res.end();
    });

    return;
  }

form.parse returns immediately, no matter how long the S3-upload process works under the hood.

I tried to use a promise and await but the function type of uploadStream must not return a promise.

Type '(file: any) => Promise<any>' is not assignable to type '(file?: VolatileFile | undefined) => Writable'.

I want to achieve something like that:

form.parse(req, (uploadedFile: string) => {
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ url: uploadedFile }, null, 2));
  res.end();
});

In order to do that, I need something like a return value from the uploadStream function where I would be able to receive the final information eventually. But since it is only returning a PassThrough-object, it seems to be very limited in that case for further approaches like mine.

Do you guys know how I can do that?

GrosSacASac commented 2 months ago

Have you tried to res.end inside the s3Client.upload callback (you need to have a ref to res there)

itinance commented 6 days ago

Have you tried to res.end inside the s3Client.upload callback (you need to have a ref to res there)

Thanks for mentioning this! As for now, in my first tests, it works as expected. I will investigate it a little more and close this ticket once this approach has been tested more