dotnet / AspNetCore.Docs

Documentation for ASP.NET Core
https://docs.microsoft.com/aspnet/core
Creative Commons Attribution 4.0 International
12.64k stars 25.28k forks source link

Streaming example is not streaming #30523

Open double-bee opened 1 year ago

double-bee commented 1 year ago

This example specifically talks about how streaming reduces the demands for memory or disk space when uploading large files.

The section Upload large files with streaming shows code calling FileHelpers.ProcessStreamedFile. That code creates a MemoryStream and copies the entire source stream into it. At the end it creates another copy of the stream by calling ToArray().

Maybe I did not understand the purpose of this example but it seems to do the exact opposite of what the article tries to communicate and I therefore highly recommend that this (part of the) article is corrected or removed.


Document Details

Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

guardrex commented 1 year ago

Hello @double-bee ... That's because the word "stream"/"streaming" is overloaded. Streaming in this context refers to how the file is broken up into sections and processed by the server ...

https://github.com/dotnet/AspNetCore.Docs/blob/main/aspnetcore/mvc/models/file-uploads/samples/3.x/SampleApp/Controllers/StreamingController.cs#L84

The "stream" (MemoryStream in this case) in the server's helper class is for processing each section and isn't a factor in the overall concept of "streaming" the file to the server.

Kamyk80 commented 1 year ago

I spent some time on analysing this article and I agree with @double-bee - there is no streaming in the examples.

Regarding @guardrex comment:

From my observations, browsers never split single file into multiple sections - single file from single file input seems to be always sent in one section.

First "streaming" example (database one) also assumes that - there is no code merging potential multiple file sections into one file, just last section (last assignment to streamedFileContent variable) is saved into the database.

Second "streaming" example (physical location) has potentially more sense - if there are multiple files in the request, each file will be buffered into memory and stored separately - but still, there is no streaming of large files here as well, and also, example code contains just one file input on the form, so what is the point?

Second part of the implementation:

else if (MultipartRequestHelper.HasFormDataContentDisposition(contentDisposition))

also doesn't make too much sense to me. What is the point of using StreamReader to allocate a new string with the value (as ReadToEndAsync() is called there)? What am I missing here?

The whole section Upload large files with streaming is very confusing and seems to be incorrect, which leads to over-complicated and incorrect implementations.