search

anacronw / multer-s3

multer storage engine for amazon s3
MIT License
660 stars 190 forks source link

Generating Hash before Upload to S3 #62

Closed manuelfink closed 3 years ago

manuelfink commented 7 years ago

Just being curious, is it anyhow possible to hook into multer-s3 before upload and access the locally cached file?

What I like to to:

  1. Hook into before upload
  2. Calculate hash
  3. Calculate phash
  4. Check with db if this file was already uploaded
  5. If yes, stop uploading to s3 and return file reference instead

I really appreciate your help to get a quick dive into multer and multer-s3!

LinusU commented 7 years ago

Actually, the file is never locally cached, it's streamed directly to S3. I would recommend looking into https://github.com/expressjs/multer/pull/399 which exposes another api that just gives you a ReadableStream per file. You could then pass this stream thru a hash-calculator, and then later on to S3...

LinusU commented 3 years ago

My recommendation would be to use Multer 3.x.

Since Multer-S3 streams the file directly we cannot really solve this here...