System out of memory exception. AsyncStreamCopier.cs

[DamianManelski]

Which service(blob, file, queue, table) does this issue concern?

This issue is concern about blob storage. It happens during calling DownloadStreamAsync from CloudBlob class.

Which version of the SDK was used?

3.0.100

Which platform are you using? (ex: .NET Core 2.1)

.NET Core 2.2

What problem was encountered?

System out of memory exception. We got your library included in our code to provide full stack trace. This is it:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at Microsoft.Azure.Storage.Core.Util.AsyncStreamCopier1.StartCopyStreamAsyncHelper(Nullable1 copyLength, Nullable1 maxLength, CancellationToken token) at Microsoft.Azure.Storage.Core.Util.AsyncStreamCopier1.StartCopyStreamAsync(Nullable1 copyLength, Nullable1 maxLength, CancellationToken cancellationToken) at Microsoft.Azure.Storage.Core.Executor.Executor.ExecuteAsync[T](RESTCommand1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token) --- End of inner exception stack trace --- at Microsoft.Azure.Storage.Core.Executor.Executor.ExecuteAsync[T](RESTCommand1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token) at Common.AzureStorage.AzureBlobStorageService.GetFileAsync(String fileId, Boolean downloadFileContent, String customContainerName) in D:\a\1\s\src\Common\AzureStorage\AzureBlobStorageService.cs:line 129 at Common.AzureStorage.AzureStorageClient.ReadFilesAsync(IList1 filesIds, String directoryName) in D:\a\1\s\src\Common\AzureStorage\AzureStorageClient.cs:line 73 --- End of inner exception stack trace --- at Common.AzureStorage.AzureStorageClient.ReadFilesAsync(IList1 filesIds, String directoryName) in D:\a\1\s\src\Common\AzureStorage\AzureStorageClient.cs:line 94

As you can see at the end is part of our code in a stack trace.

How can we reproduce the problem in the simplest way?

It happens occasionally when we trying to read file from azure storage. The file is not big and it seems like it doesn't depend of file size.

Have you found a mitigation/solution?

No but the we have found that the exception is being thrown from line 304 which is await writeTask.ConfigureAwait(false); into this method: private async Task StartCopyStreamAsyncHelper(ILogger logger, long? copyLength, long? maxLength, CancellationToken token)

[amnguye]

Hi Damien,

Could you provide the exact lines of code you are writing to reproduce this issue. Also I am concerned you're using such an old version of the SDK. The issue where you have posted the issue is on v11 of the .NET Storage SDK. We recommend you upgrade to the latest version of the sdk which is v12. If you have any issues with v12 of the SDK please post them to this repo http://github.com/Azure/azure-sdk-for-net/ That way the right people can address your issues or questions.

[dm1988air]

We see that the exception is being thrown from DownloadToStreamAsync from ICloudBlob.cs interface. If you want to look deeper please follow stack trace information i provided in my previous message. I use v11 (we will check if this happens also in v12). What is SDK version you rocommend ?

[amnguye]

We recommend v12 as we have fixed many issues in that version and it is the current version as well.

[amnguye]

Could you also provide the exact lines of code you are writing to reproduce this issue.

You say you see the exception being thrown from DownloadToStreamAsync but the stack trace shows the exception is being thrown from StartCopyStreamAsyncHelper. DownloadToStreamAsync and StartCopyStreamAsync do not call each other.

[dm1988air]

We recommend v12 as we have fixed many issues in that version and it is the current version as well.

Hi.

Could you confirm that newest version of this package is 12 ? I can see any upgrade available for this package:

[amnguye]

Yes it is Azure.Storage.Blobs