[BUG]Memory usage increasing over time while uploading files to a storage account from an IOT Edge module

Library name and version

Azure.Storage.Blobs:12.15.0

Describe the bug

We have an IOT Edge module that is used for uploading files, up to 50 MB in size to a storage account in Azure.

The connection to the storage account in Azure is not reliable and very likely to drop from time to time. The program uses a retry logic to upload in case the blob upload fails.

The storage account is associated with the IOT Hub in Azure and the files are uploaded using the following code. After a while though, the memory used by this module continues to increase over time. We have currently added a limit to how much memory can be used by this module and that seems to have fixed the problem for now.

We are using Azure.Storage.Blob: 12.15.0 for upload.

Expected behavior

Memory to stay stable over time

Actual behavior

Memory rises over time.

Reproduction Steps

Here is the method that uploads the file to the storage account:

public async Task<string> PersistFileToBlobStorage(string filepath)
{
    try
    {
        if (!File.Exists(filepath))
        {
            throw new FileNotFoundException(filepath);
        }
        _logger.LogInformation($"File exists: [{filepath}], attempting to persist to IoT Hub.");
        using (FileStream fs = File.OpenRead(filepath))
        {
            _logger.LogInformation($"FileStream opened.\tLength: [{fs.Length}]");
            string filename = Path.GetFileName(filepath);
            _logger.LogInformation($"Attempting to upload blob:\tFilename: [{filename}]");
            FileUploadSasUriRequest fileUploadSasUriRequest = new FileUploadSasUriRequest
            {
                BlobName = filename
            };
            FileUploadSasUriResponse sasUri = await _deviceClient.GetFileUploadSasUriAsync(fileUploadSasUriRequest);
            Uri uploadUri = sasUri.GetBlobUri();
            BlockBlobClient blockBlobClient = new BlockBlobClient(uploadUri);

            try
            {
                var response = await blockBlobClient.UploadAsync(fs, new BlobUploadOptions());
            }
            catch (Exception ex)
            {
                _logger.LogWarning($"Notifying IoT Hub of the following error: ", ex.Message);
                // If file upload fails, release resources from the IoT Hub to avoid too many concurrent uploads registered
                var failedFileUploadCompletionNotification = new FileUploadCompletionNotification
                {
                    CorrelationId = sasUri.CorrelationId,
                    IsSuccess = false,
                    StatusCode = 500,
                    StatusDescription = "Failure"
                };

                await _deviceClient.CompleteFileUploadAsync(failedFileUploadCompletionNotification);
                throw;
            }

            var successfulFileUploadCompletionNotification = new FileUploadCompletionNotification
            {
                CorrelationId = sasUri.CorrelationId,
                IsSuccess = true,
                StatusCode = 200,
                StatusDescription = "Success"
            };

            await _deviceClient.CompleteFileUploadAsync(successfulFileUploadCompletionNotification);
            // Get the URL for the blob in storage (credentials removed)

            string uploadUrl = sasUri.GetBlobUri().GetLeftPart(UriPartial.Path).Replace("%2F", "/");
            return uploadUrl;
        }
    }
    catch (Exception ex)
    {
        _logger.LogError($"Exception persisting file to IoT Hub: [{filepath}]", ex);
        throw;
    }
}

Environment

RHEL 9 Azure.Storage.Blobs 12.15.0 .net 6.0 IOT Edge Runtime 1.4

Azure / azure-sdk-for-net