search

anacronw / multer-s3

multer storage engine for amazon s3
MIT License
660 stars 190 forks source link

Avoid broken files uploaded #32

Closed thebergamo closed 8 years ago

thebergamo commented 8 years ago

Have a way to prevent a broken file to be uploaded?

I see in my API some files uploaded just by the half, the other part of the image is just a gray block. Maybe some bytes are lost I guess.

How I can avoid this behavior? In other cases we use a MD5, but when we're working with streams, maybe this is a complicated think.

Any ideas?

anacronw commented 8 years ago

I'm not entirely sure of best practices, but you should be able to compute the hash on the client, and compute the hash while the file is being streamed to compare. Node's crypto library offers streaming hash updates

thebergamo commented 8 years ago

Hm... this feature maybe needs to be in the core of multer-s3not?

anacronw commented 8 years ago

Can you see if this works for you?

https://github.com/badunk/multer-s3/blob/master/index.js#L124

There should already be an eTag that is being returned

thebergamo commented 8 years ago

I'm not sure, but I need try.

In some cases my images uploaded are broken(with a half bytes not uploaded). If I need check just the ETag I need to send the checksum in the upload right?

anacronw commented 8 years ago

Yes, according to the s3 documentation, the ETag works in most cases as a checksum. The cases that it doesn't has to do with simultaneous uploads I believe from multiple sources.