search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.81k stars 321 forks source link

Blob_SetMetadata： Fail with 403 when use HttpRequestHandler.SetHeader in request OperationContext #331

Open blueww opened 4 years ago

blueww commented 4 years ago

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.3.0-preview

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

VS code

What's the Node.js version?

v10.15.3

What problem was encountered?

Blob_SetMetadata： Fail with 403 when use HttpRequestHandler.SetHeader in request OperationContext

Steps to reproduce the issue?

          var blobName = GetRandomBlobName();
                CloudPageBlob blob = container.GetPageBlobReference(blobName);
                CloudPageBlob blob2 = container.GetPageBlobReference(blobName);
                blob.Create(1024);
                blob.Metadata["key1"] = "value1";

                OperationContext context = new OperationContext();
                context.SendingRequest += (sender, e) =>
                {
                    HttpRequestHandler.SetHeader(e.Request, "x-ms-meta-key1", string.Empty);
                };

                blob.SetMetadata(operationContext: context);

Have you found a mitigation/solution?

No

Debug Log:

2019-12-18T03:05:04.022Z 4924342c-6234-433f-86df-ad99296838ac info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://127.0.0.1/devstoreaccount1/testcb6019b7f8e4b4b1baeb044b072c5e915/testb63b649aca6764f039677b52ca75e459c?comp=metadata RequestHeaders:{"x-ms-meta-key1":"value1, ","x-ms-client-request-id":"9a3ffec7-23e5-45f9-9716-75749ee73d30","user-agent":"Azure-Storage/11.1.1 (.NET Core; Win32NT 6.2.9200.0)","x-ms-version":"2019-02-02","x-ms-date":"Wed, 18 Dec 2019 03:05:04 GMT","authorization":"SharedKey devstoreaccount1:8Xoa5/7XN8DK4RB1yAkVIYDW5vmbdXGVdVKrIYKiO+E=","content-length":"0","host":"127.0.0.1:10000"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2019-12-18T03:05:04.023Z 4924342c-6234-433f-86df-ad99296838ac info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=testcb6019b7f8e4b4b1baeb044b072c5e915 Blob=testb63b649aca6764f039677b52ca75e459c
2019-12-18T03:05:04.023Z 4924342c-6234-433f-86df-ad99296838ac verbose: DispatchMiddleware: Dispatching request...
2019-12-18T03:05:04.025Z 4924342c-6234-433f-86df-ad99296838ac info: DispatchMiddleware: Operation=Blob_SetMetadata
2019-12-18T03:05:04.026Z 4924342c-6234-433f-86df-ad99296838ac verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2019-12-18T03:05:04.026Z 4924342c-6234-433f-86df-ad99296838ac info: PublicAccessAuthenticator:validate() Start validation against public access.
2019-12-18T03:05:04.027Z 4924342c-6234-433f-86df-ad99296838ac debug: PublicAccessAuthenticator:validate() Getting account properties...
2019-12-18T03:05:04.027Z 4924342c-6234-433f-86df-ad99296838ac debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: testcb6019b7f8e4b4b1baeb044b072c5e915, blob: testb63b649aca6764f039677b52ca75e459c
2019-12-18T03:05:04.033Z 4924342c-6234-433f-86df-ad99296838ac debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container testcb6019b7f8e4b4b1baeb044b072c5e915
2019-12-18T03:05:04.033Z 4924342c-6234-433f-86df-ad99296838ac info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2019-12-18T03:05:04.034Z 4924342c-6234-433f-86df-ad99296838ac info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:9a3ffec7-23e5-45f9-9716-75749ee73d30\nx-ms-date:Wed, 18 Dec 2019 03:05:04 GMT\nx-ms-meta-key1:value1, \nx-ms-version:2019-02-02\n/devstoreaccount1/devstoreaccount1/testcb6019b7f8e4b4b1baeb044b072c5e915/testb63b649aca6764f039677b52ca75e459c\ncomp:metadata"
2019-12-18T03:05:04.034Z 4924342c-6234-433f-86df-ad99296838ac info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:npLSWTGh4RIIlLC6cNf2u5SqkZG6VBbThPBZm8nzh1s=
2019-12-18T03:05:04.034Z 4924342c-6234-433f-86df-ad99296838ac info: BlobSharedKeyAuthenticator:validate() Validation failed.
2019-12-18T03:05:04.035Z 4924342c-6234-433f-86df-ad99296838ac info: AccountSASAuthenticator:validate() Start validation against account Shared Access Signature pattern.
2019-12-18T03:05:04.035Z 4924342c-6234-433f-86df-ad99296838ac debug: AccountSASAuthenticator:validate() Getting account properties...
2019-12-18T03:05:04.035Z 4924342c-6234-433f-86df-ad99296838ac debug: AccountSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: testcb6019b7f8e4b4b1baeb044b072c5e915, blob: testb63b649aca6764f039677b52ca75e459c
2019-12-18T03:05:04.035Z 4924342c-6234-433f-86df-ad99296838ac debug: AccountSASAuthenticator:validate() Got account properties successfully.
2019-12-18T03:05:04.036Z 4924342c-6234-433f-86df-ad99296838ac debug: AccountSASAuthenticator:validate() Retrieved signature from URL parameter sig: undefined
2019-12-18T03:05:04.036Z 4924342c-6234-433f-86df-ad99296838ac info: AccountSASAuthenticator:validate() Failed to get valid account SAS values from request.
2019-12-18T03:05:04.037Z 4924342c-6234-433f-86df-ad99296838ac info: BlobSASAuthenticator:validate() Start validation against blob service Shared Access Signature pattern.
2019-12-18T03:05:04.037Z 4924342c-6234-433f-86df-ad99296838ac debug: BlobSASAuthenticator:validate() Getting account properties...
2019-12-18T03:05:04.037Z 4924342c-6234-433f-86df-ad99296838ac debug: BlobSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: testcb6019b7f8e4b4b1baeb044b072c5e915, blob: testb63b649aca6764f039677b52ca75e459c
2019-12-18T03:05:04.037Z 4924342c-6234-433f-86df-ad99296838ac debug: BlobSASAuthenticator:validate() Got account properties successfully.
2019-12-18T03:05:04.037Z 4924342c-6234-433f-86df-ad99296838ac debug: BlobSASAuthenticator:validate() Retrieved signature from URL parameter sig: undefined
2019-12-18T03:05:04.038Z 4924342c-6234-433f-86df-ad99296838ac debug: BlobSASAuthenticator:validate() No signature found in request. Skip blob service SAS validation.
2019-12-18T03:05:04.039Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2019-12-18T03:05:04.040Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.  ErrorHTTPStatusCode=403 ErrorHTTPStatusMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. ErrorHTTPHeaders={"x-ms-error-code":"AuthorizationFailure","x-ms-request-id":"4924342c-6234-433f-86df-ad99296838ac"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>AuthorizationFailure</Code>\n  <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:4924342c-6234-433f-86df-ad99296838ac\nTime:2019-12-18T03:05:04.038Z</Message>\n</Error>" ErrorStack="StorageError: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\n    at Function.getAuthorizationFailure (C:\\code\\azurite\\src\\blob\\errors\\StorageErrorFactory.ts:253:12)\n    at authenticate.then.pass (C:\\code\\azurite\\src\\blob\\authentication\\AuthenticationMiddlewareFactory.ts:25:35)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"
2019-12-18T03:05:04.041Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Set HTTP code: 403
2019-12-18T03:05:04.041Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Set HTTP status message: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.
2019-12-18T03:05:04.041Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=AuthorizationFailure
2019-12-18T03:05:04.042Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=4924342c-6234-433f-86df-ad99296838ac
2019-12-18T03:05:04.042Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Set content type: application/xml
2019-12-18T03:05:04.042Z 4924342c-6234-433f-86df-ad99296838ac error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>AuthorizationFailure</Code>\n  <Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.\nRequestId:4924342c-6234-433f-86df-ad99296838ac\nTime:2019-12-18T03:05:04.038Z</Message>\n</Error>"
2019-12-18T03:05:04.043Z 4924342c-6234-433f-86df-ad99296838ac info: EndMiddleware: End response. TotalTimeInMS=21 StatusCode=403 StatusMessage=Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature. Headers={"server":"Azurite-Blob/3.3.0-preview","x-ms-error-code":"AuthorizationFailure","x-ms-request-id":"4924342c-6234-433f-86df-ad99296838ac","content-type":"application/xml"}

Test Case:

Test Name:  CloudPageBlobGetEmptyMetadata
Test FullName:  Microsoft.Azure.Storage.Blob.CloudPageBlobTest.CloudPageBlobGetEmptyMetadata
Test Source:    C:\code\azure-storage-net\Test\ClassLibraryCommon\Blob\CloudPageBlobTest.cs : line 1523
Test Outcome:   Failed
Test Duration:  0:00:00.3238956

Result StackTrace:  
at Microsoft.Azure.Storage.Core.Executor.Executor.ExecuteAsync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token) in C:\code\azure-storage-net\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 274
   at Microsoft.Azure.Storage.Core.Executor.Executor.<>c__DisplayClass0_0`1.<ExecuteSync>b__0() in C:\code\azure-storage-net\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 41
   at Microsoft.Azure.Storage.Core.Util.CommonUtility.RunWithoutSynchronizationContext[T](Func`1 actionToRun) in C:\code\azure-storage-net\Lib\Common\Core\Util\CommonUtility.cs:line 378
   at Microsoft.Azure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext) in C:\code\azure-storage-net\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 40
   at Microsoft.Azure.Storage.Blob.CloudBlob.SetMetadata(AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext) in C:\code\azure-storage-net\Lib\ClassLibraryCommon\Blob\CloudBlob.cs:line 1658
   at Microsoft.Azure.Storage.Blob.CloudPageBlobTest.CloudPageBlobGetEmptyMetadata() in C:\code\azure-storage-net\Test\ClassLibraryCommon\Blob\CloudPageBlobTest.cs:line 1542
Result Message: 
Test method Microsoft.Azure.Storage.Blob.CloudPageBlobTest.CloudPageBlobGetEmptyMetadata threw exception: 
Microsoft.Azure.Storage.StorageException: Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.
XiaoningLiu commented 4 years ago

Undocumented behavior for Constructing the canonicalized headers string https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key#constructing-the-canonicalized-headers-string

Marker as post ga item

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.