Closed def-roth closed 1 month ago
If I read the getParts interface correct that is problematic as it returns array buffers which become memory intensive when dealing with huge files.
There are streaming multipart parsers you would need to find one that works for you npmjs.com/search?q=multipart
All good. This was an error on my end. I had to implement parsing myself as the requirements are pretty special, but working with the stream was a charm. Thanks for the hint though!
getParts is not very well designed like you mention it uses in-memory buffers rather than streaming. This interface was added for a customer who wanted it like this. I would rather have streams but, yeah..
I can add a PR to the examples as a starting point how to solve this. Yet I think there is no silver bullet here, neither memory nor disk are optimal and both have huge foot gun potential. Although most developers will never need a filesize larger than 5mb.
The following code produces results for arbitrary files exept for images. I'm running around in circles here and I bet it's just me doing something wrong. Different types of buffer allocations have been tried. Storing them in an Array and concatenating them (doesn't work) or calling buffer.toString(). At least this works as minimum reproducible I guess.