Open samchon opened 8 months ago
The thing is, it can accept Readable
as you mentioned, with the right configuration. While it may be a better solution, in terms of memory management, it's also a more advanced solution in terms of Node API knowledge as not everyone is well versed with streams, and it's a breaking change at the moment. We'd have to make sure to update the docs to show how the stream could be worked with, and probably be ready to answer many, many more questions on how to use the newer approach, which would inevitably be sent to Discord or StackOverflow and answered there.
While I do think that better memory management is a good thing, there is a balance to be kept between efficient and ergonomic API for developers.
Perhaps we can make a mention in the docs and the default of buffer and how to use streams if that is the desired approach and why you might want to do that
If hope to avoid breaking change, I think NestJS guide document must warn the memory problem, and give an example code converting to the streaming strategy.
Adding a mention in the docs sounds valuable
I'd tried to make concise @UploadFile()
similar decorator supports both express
and fastify
, and understood why @kamilmysliwiec and @jmcdo29 had configured of @UploadFile()
to just utilize the full Buffer
data as default. To perform streaming, have to configure storage engine of multer
, but it was not easy to determine which option to adjust as default.
On the other hand, fastify
's multipart/form-data
was much easier to handle. This is because streaming could be used right away without any additional settings. Therefore, I had developed @TypedFormData.Body()
to utillize the full data Buffer
to compatible with both express
and fastify
, and planning to apply memory storage to the multer
side by referring to fastify
's options.
Anyway, after I completed this @TypedFormData.Body()
and studied it thoroughly through various experiments, I will contribute to Nest Document. The next PR may be the next month.
@samchon Have you had any chance to contribute to NestJS documentation? :)
Have forgotten this issue for a long time due to busy.
@Shandur Thanks for reminding. It's okay you to contribute eariler.
Is there an existing issue that is already proposing this?
Is your feature request related to a problem? Please describe it
When using
@UploadFile()
decorator following the NestJS guide documents, it becomesBuffer
type.As you know, the
Buffer
type means that backend server gathers entire data of the uploaded file. If client uploads 1GB file through the API, the NestJS backend server requires at least 1GB memory about the request. If there're 100 clients uploading the 1GB file at the same time, NestJS backend server needs 100GB memory at that time.Describe the solution you'd like
I know that the default option of
multer
is using theBuffer
type instance, but it seems not proper to the full framework like NestJS. If configure storage engine of themulter
, the uploadaed file would beReadable
stream type, and I think NestJS should do it.Teachability, documentation, adoption, migration strategy
I have followed the https://docs.nestjs.com/techniques/file-upload document.
And very surprised by too much memory consumption.
What is the motivation / use case for changing the behavior?
I think NestJS should do like below:
Buffer
feature, they must configure by themselves