search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.8k stars 319 forks source link

StageBlock/UploadPages: not check the content aligned with "Content-MD5" header #303

Open blueww opened 4 years ago

blueww commented 4 years ago

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.3.0-preview

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

VS code

What's the Node.js version?

v10.15.3

What problem was encountered?

StageBlock/UploadPages: not check the content aligned with "Content-MD5" header So the request will success, even input a wrong "Content-MD5" header

Steps to reproduce the issue?

StageBlock/UploadPages with a wrong a wrong "Content-MD5" header

Have you found a mitigation/solution?

No

Rest doc:

Content-MD5 | Optional. An MD5 hash of the page content. This hash is used to verify the integrity of the page during transport. When this header is specified, the storage service compares the hash of the content that has arrived with this header value.

Test case info:

Test Name: ExtendedErrorInfoVerifyXmlWithAdditionalDetails Test FullName: Microsoft.Azure.Storage.StorageExceptionTests.ExtendedErrorInfoVerifyXmlWithAdditionalDetails Test Source: C:\code\azure-storage-net\Test\ClassLibraryCommon\StorageExceptionTests.cs : line 137 Test Outcome: Failed Test Duration: 0:00:01.6398374

Result StackTrace:
at Microsoft.Azure.Storage.TestHelper.ExpectedException[T](Action operation, String operationDescription, String expectedMessage) in C:\code\azure-storage-net\Test\Common\TestHelper.Common.cs:line 85 at Microsoft.Azure.Storage.StorageExceptionTests.ExtendedErrorInfoVerifyXmlWithAdditionalDetails() in C:\code\azure-storage-net\Test\ClassLibraryCommon\StorageExceptionTests.cs:line 164 Result Message: Assert.Fail failed. No exception received while expecting Microsoft.Azure.Storage.StorageException: Invalid MD5 should fail with mismatch

torbjornvatn commented 4 years ago

I also experienced this @blueww when my tests relying on Azurite started failing. I use the https://godoc.org/github.com/Azure/azure-storage-blob-go/azblob#UploadStreamToBlockBlob function from https://github.com/Azure/azure-storage-blob-go

blueww commented 4 years ago

The check is still not implemented in Azurite. We will priority it with all other feature requirements.

dennisdietrich commented 4 years ago

The check is still not implemented in Azurite. We will priority it with all other feature requirements.

Please increase the priority of this issue. Microsoft is pushing Azurite as the official replacement for the Azure Storage Emulator. Yet, here we have something - verifying that I set the correct MD5 - that's impossible to test with Azurite at the moment. I can't think of anything more important for file/blob operations than data integrity so this is something I absolutely have to get right in my client but I can only test it against the Azure live site.

wayne-li2 commented 3 years ago

I've also been getting this problem as well! Just leaving a comment here to raise awareness, we are pushing heavily to use Azurite and this is our last blocker.

If I have time I can maybe put in a PR.

wayne-li2 commented 3 years ago

Okay, I think I've patched BlockBlob_StageBlock. I don't think I have the time to tackle UploadPages but maybe someone can take my PR as a template and apply something similar there. PR is #689