Invalid behavior when doing full upload after non-committed multipart upload

[paveliak]

Which service(blob, file, queue, table) does this issue concern?

Blob

Which version of the Azurite was used?

3.8.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

We use docker container from mcr.microsoft.com/azure-storage/azurite@sha256:f1fdee4a45226659c5afa30b8d8d3a479037c701826d32448078f22a7cd011a4

What's the Node.js version?

Whatever comes in the container

What problem was encountered?

There is a discrepancy in how Azurite and Azure Blob store handle full blob upload that happens after multi part upload if multipart upload was not committed and full blob size exceeds 4Mb. Azure Blob throws Microsoft.Azure.Storage.StorageException: 'The specified blob or block content is invalid.' while Azurite performs upload with no error. See repro code below.

Note that full blob upload must be over 4Mb. If you set it to 4Mb or smaller then Azure Blob Storage would not throw. And Azurite would not throw either.

Steps to reproduce the issue?

CloudBlobContainer c = blobClient.GetContainerReference(Guid.NewGuid().ToString("N"));
await c.CreateIfNotExistsAsync();
CloudBlockBlob b = c.GetBlockBlobReference(Guid.NewGuid().ToString("N"));
string bid = Convert.ToBase64String(Encoding.UTF8.GetBytes("00000"));
await b.PutBlockAsync(bid, new MemoryStream(new byte[10]), contentMD5: null, accessCondition: default, options: null, operationContext: null);
await b.UploadFromStreamAsync(new MemoryStream(new byte[1024 * 1024 * 8])); <--- Azure Blob Store throws here

Have you found a mitigation/solution?

No

[XiaoningLiu]

Thanks @paveliak Believe Azure Blob Storage has additional validation against blocks length or block ID while Azurite doesn't. Azurite currently trys to align Azure Blob Storage features and behaviors while are well documented. Will check if it's another undocumented negative check and update accordingly.

[paveliak]

Thanks @XiaoningLiu Here is unit test for this issue. Hope it would be helpful:

using System;
using System.IO;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace AzuriteTest
{
    [TestClass]
    public class Issue531Test
    {
        [TestMethod]
        public async Task CantUploadBlobAfterBlockUpload()
        {
            string connectionString = "<STORAGE_ACCOUNT_CONNECTION_STRING>";
            CloudBlobClient blobClient = CloudStorageAccount.Parse(connectionString).CreateCloudBlobClient();
            CloudBlobContainer blobContainer = blobClient.GetContainerReference(Guid.NewGuid().ToString("N"));
            await blobContainer.CreateIfNotExistsAsync();
            CloudBlockBlob blockBlob = blobContainer.GetBlockBlobReference(Guid.NewGuid().ToString("N"));
            string blockId = Convert.ToBase64String(Encoding.UTF8.GetBytes("00000"));
            await blockBlob.PutBlockAsync(blockId, new MemoryStream(new byte[10]), contentMD5: null, accessCondition: default, options: null, operationContext: null);
            await Assert.ThrowsExceptionAsync<StorageException>(() => blockBlob.UploadFromStreamAsync(new MemoryStream(new byte[1024 * 1024 * 8])));
        }
    }
}