search

filipw / Strathweb.AspNetCore.AzureBlobFileProvider

MIT License
55 stars 22 forks source link

Out of memory exception when downloading large files from Blob #18

Open ibruynin opened 5 years ago

ibruynin commented 5 years ago

Hi

Based on application insights logs, the following method throws System.OutOfMemoryException exception when downloading files larger than 350Mb on an Azure web app running on 3.5Gb (S2 service plan)

[Edit: it's not that crazy... it's 350Mb :)]

Strathweb.AspNetCore.AzureBlobFileProvider.AzureBlobFileInfo.CreateReadStream

I'll attach the full stack trace in a separate file: fullstacktrace.txt

I'm not sure if this is intended to be used for large files, but it sure would be a great and powerful feature if it could be used like that!

ibruynin commented 5 years ago

and this is a snapshot of the heap

image

The large amount of memory is taken by a memorystream instance that seems to take twice the size of the file being downloaded...

johnayling commented 4 years ago

The way it is using a MemoryStream within the CreateReadStream method prevents it from being used for large files. Really want to return a FileStreamResult directly so that the stream acts as a pipe from storage to the client.