search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.83k stars 325 forks source link

Empty blob access error using Java SDK #2458

Open cnaj opened 2 months ago

cnaj commented 2 months ago

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.32.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

Docker

What's the Node.js version?

n/a

What problem was encountered?

Downloading an empty (0 bytes) blob from Azurite gives a 416 error response, while Azure works as expected. With version 3.31.0, both Azurite and Azure work as expected without an error.

This seems to contradict #2410, which reported the opposite behavior for an earlier Azurite version. As a consequence, #2417 was implemented.

Steps to reproduce the issue?

See https://github.com/cnaj/azurite-empty-blob-issue-demo for a reproduction of the issue.

Have you found a mitigation/solution?

no

blueww commented 2 months ago

@cnaj For the download request, do you add header "x-ms-rage" with a range end bigger than the real blob size 0?

cnaj commented 2 months ago

Indeed, the Azure server responds with 416, too, but it was hidden by the Java Azure SDK. However, there seems to be a difference in the 416 response: The Azure server adds a "Content-Range: bytes */0" header in the response, which Azurite doesn't. Apparently, this triggers the Java SDK to retry the download with the correct range.

Logs for the public Azure server:

[Test worker] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"b22d6b28-5c86-417e-8696-e3fdf9d931b9","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":416,"url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":20,"content-length":249,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-version":"2024-08-04","Content-Range":"bytes */0","x-ms-error-code":"InvalidRange","Content-Type":"application/xml","x-ms-request-id":"4fedb6ba-101e-0000-291d-fdff97000000","x-ms-client-request-id":"b22d6b28-5c86-417e-8696-e3fdf9d931b9","content-length":249}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0--1","x-ms-client-request-id":"8ef21149-1b2b-46cc-a0e8-868a2ae5d5c3","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":200,"url":"https://xxxx.blob.core.windows.net/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":71,"content-length":0,"Date":"Mon, 02 Sep 2024 09:52:34 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-lease-status":"unlocked","x-ms-version":"2024-08-04","x-ms-lease-state":"available","x-ms-blob-type":"BlockBlob","x-ms-server-encrypted":"true","Last-Modified":"Mon, 02 Sep 2024 09:52:34 GMT","Content-MD5":"1B2M2Y8AsgTpgAmY7PhCfg==","x-ms-creation-time":"Mon, 02 Sep 2024 09:52:34 GMT","ETag":"0x8DCCB34F85C7630","Content-Type":"application/octet-stream","Accept-Ranges":"bytes","x-ms-request-id":"c925b91e-b01e-0019-441d-fd7f2c000000","x-ms-client-request-id":"8ef21149-1b2b-46cc-a0e8-868a2ae5d5c3","content-length":0}

Logs for Azurite:

[Test worker] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Mon, 02 Sep 2024 09:53:35 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"9abe0988-16f7-45e9-b6a3-cee54657deea","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":416,"url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":3,"Date":"Mon, 02 Sep 2024 09:53:35 GMT","Server":"Azurite-Blob/3.32.0","Transfer-Encoding":"chunked","Connection":"keep-alive","content-type":"application/xml","x-ms-error-code":"InvalidRange","x-ms-request-id":"387ca0c3-1fc3-46a9-91b8-8b48cc2441c5","redactedHeaders":"Keep-Alive"}

No further GET is being performed for Azurite by the SDK.

Logs were acquired by setting the environment variable AZURE_HTTP_LOG_DETAIL_LEVEL=HEADERS, as documented at https://learn.microsoft.com/en-us/azure/developer/java/sdk/logging-overview

blueww commented 2 months ago

@cnaj

A Fix PR is already raised, please check if it works for you. https://github.com/Azure/Azurite/pull/2461

cnaj commented 2 months ago

Thanks for the PR! I've tested it with my Java reproduction test, but now the followup request generates a 500 error.

Test logs (as before):

[Test worker] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"71c01aca-2873-4884-98d1-51e9b148fb5f","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":416,"url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":12,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","Server":"Azurite-Blob/3.32.0","Transfer-Encoding":"chunked","Content-Range":"bytes */0","Connection":"keep-alive","content-type":"application/xml","x-ms-error-code":"InvalidRange","x-ms-request-id":"0f138ca1-55d0-428f-b8f6-66783c4805a8","redactedHeaders":"Keep-Alive"}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","tryCount":1,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","x-ms-version":"2024-08-04","x-ms-range":"bytes=0--1","x-ms-client-request-id":"feca516f-681d-4eed-acda-c8cd80cdab5f","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","redactedHeaders":"Authorization","content-length":0}
[reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":500,"url":"http://127.0.0.1:10000/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt","durationMs":3,"content-length":0,"Date":"Tue, 03 Sep 2024 10:46:02 GMT","Server":"Azurite-Blob/3.32.0","Connection":"keep-alive","redactedHeaders":"Keep-Alive","content-length":0}

Azurite debug log:

2024-09-03T10:46:02.481Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt RequestHeaders:{"host":"127.0.0.1:10000","date":"Tue, 03 Sep 2024 10:46:02 GMT","authorization":"SharedKey devstoreaccount1:F4noZ3BQJwqCiercZUEdxB4Pwui33GtaDnzlrdTI6/o=","x-ms-version":"2024-08-04","x-ms-range":"bytes=0-4194303","x-ms-client-request-id":"71c01aca-2873-4884-98d1-51e9b148fb5f","accept":"application/xml","user-agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","content-length":"0"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2024-09-03T10:46:02.482Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=test-42b8bbde-5e85-4359-9130-64822da82035 Blob=empty-file.txt
2024-09-03T10:46:02.482Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 verbose: DispatchMiddleware: Dispatching request...
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: DispatchMiddleware: Operation=Blob_Download
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-09-03T10:46:02.483Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: test-42b8bbde-5e85-4359-9130-64822da82035, blob: empty-file.txt
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container test-42b8bbde-5e85-4359-9130-64822da82035
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\nTue, 03 Sep 2024 10:46:02 GMT\n\n\n\n\n\nx-ms-client-request-id:71c01aca-2873-4884-98d1-51e9b148fb5f\nx-ms-range:bytes=0-4194303\nx-ms-version:2024-08-04\n/devstoreaccount1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt"
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:F4noZ3BQJwqCiercZUEdxB4Pwui33GtaDnzlrdTI6/o=
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 verbose: DeserializerMiddleware: Start deserializing...
2024-09-03T10:46:02.484Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: HandlerMiddleware: DeserializedParameters={"options":{"range":"bytes=0-4194303","requestId":"71c01aca-2873-4884-98d1-51e9b148fb5f","leaseAccessConditions":{},"cpkInfo":{},"modifiedAccessConditions":{}},"version":"2024-08-04"}
2024-09-03T10:46:02.486Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The range specified is invalid for the current size of the resource.  ErrorHTTPStatusCode=416 ErrorHTTPStatusMessage=The range specified is invalid for the current size of the resource. ErrorHTTPHeaders={"x-ms-error-code":"InvalidRange","x-ms-request-id":"0f138ca1-55d0-428f-b8f6-66783c4805a8","Content-Range":"bytes */0"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>InvalidRange</Code>\n  <Message>The range specified is invalid for the current size of the resource.\nRequestId:0f138ca1-55d0-428f-b8f6-66783c4805a8\nTime:2024-09-03T10:46:02.486Z</Message>\n</Error>" ErrorStack="StorageError: The range specified is invalid for the current size of the resource.\n    at StorageErrorFactory.getInvalidPageRange2 (/Users/cernaj/github/Azure/Azurite/dist/src/blob/errors/StorageErrorFactory.js:79:27)\n    at BlobHandler.downloadBlockBlobOrAppendBlob (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:649:53)\n    at BlobHandler.download (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:45:25)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP code: 416
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP status message: The range specified is invalid for the current size of the resource.
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=InvalidRange
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=0f138ca1-55d0-428f-b8f6-66783c4805a8
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP Header: Content-Range=bytes */0
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set content type: application/xml
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>InvalidRange</Code>\n  <Message>The range specified is invalid for the current size of the resource.\nRequestId:0f138ca1-55d0-428f-b8f6-66783c4805a8\nTime:2024-09-03T10:46:02.486Z</Message>\n</Error>"
2024-09-03T10:46:02.487Z 0f138ca1-55d0-428f-b8f6-66783c4805a8 info: EndMiddleware: End response. TotalTimeInMS=6 StatusCode=416 StatusMessage=The range specified is invalid for the current size of the resource. Headers={"server":"Azurite-Blob/3.32.0","x-ms-error-code":"InvalidRange","x-ms-request-id":"0f138ca1-55d0-428f-b8f6-66783c4805a8","content-range":"bytes */0","content-type":"application/xml"}
2024-09-03T10:46:02.492Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt RequestHeaders:{"host":"127.0.0.1:10000","date":"Tue, 03 Sep 2024 10:46:02 GMT","authorization":"SharedKey devstoreaccount1:PWXMvUmglG/3azeGiPoTLvQIuiXesM4WNIYPFhNVo3o=","x-ms-version":"2024-08-04","x-ms-range":"bytes=0--1","x-ms-client-request-id":"feca516f-681d-4eed-acda-c8cd80cdab5f","accept":"application/xml","user-agent":"azsdk-java-azure-storage-blob/12.27.1 (17.0.12; Mac OS X; 14.6.1)","content-length":"0"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2024-09-03T10:46:02.492Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=test-42b8bbde-5e85-4359-9130-64822da82035 Blob=empty-file.txt
2024-09-03T10:46:02.492Z dbf3ca32-86b7-4a6b-8228-21bab486b27c verbose: DispatchMiddleware: Dispatching request...
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: DispatchMiddleware: Operation=Blob_Download
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: PublicAccessAuthenticator:validate() Start validation against public access.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c debug: PublicAccessAuthenticator:validate() Getting account properties...
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: test-42b8bbde-5e85-4359-9130-64822da82035, blob: empty-file.txt
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container test-42b8bbde-5e85-4359-9130-64822da82035
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\nTue, 03 Sep 2024 10:46:02 GMT\n\n\n\n\n\nx-ms-client-request-id:feca516f-681d-4eed-acda-c8cd80cdab5f\nx-ms-range:bytes=0--1\nx-ms-version:2024-08-04\n/devstoreaccount1/devstoreaccount1/test-42b8bbde-5e85-4359-9130-64822da82035/empty-file.txt"
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:PWXMvUmglG/3azeGiPoTLvQIuiXesM4WNIYPFhNVo3o=
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c verbose: DeserializerMiddleware: Start deserializing...
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: HandlerMiddleware: DeserializedParameters={"options":{"range":"bytes=0--1","requestId":"feca516f-681d-4eed-acda-c8cd80cdab5f","leaseAccessConditions":{},"cpkInfo":{},"modifiedAccessConditions":{}},"version":"2024-08-04"}
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c error: ErrorMiddleware: Received an error, fill error information to HTTP response
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c error: ErrorMiddleware: ErrorName=RangeError ErrorMessage=deserializeRangeHeader: raw range value bytes=0--1 is wrong. ErrorStack="RangeError: deserializeRangeHeader: raw range value bytes=0--1 is wrong.\n    at deserializeRangeHeader (/Users/cernaj/github/Azure/Azurite/dist/src/blob/utils/utils.js:46:15)\n    at BlobHandler.downloadBlockBlobOrAppendBlob (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:638:64)\n    at BlobHandler.download (/Users/cernaj/github/Azure/Azurite/dist/src/blob/handlers/BlobHandler.js:45:25)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c error: ErrorMiddleware: Set HTTP code: 500
2024-09-03T10:46:02.493Z dbf3ca32-86b7-4a6b-8228-21bab486b27c info: EndMiddleware: End response. TotalTimeInMS=1 StatusCode=500 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.32.0"}

Looks like the Azure SDK sends x-ms-range: bytes=0--1, which I would interpret as "from 0 to -1", i.e. until the end. Azure responds with 200 (see previous my comment).

blueww commented 2 months ago

@cnaj

I tried to search x-ms-range: bytes=0--1 expected behavior from rest API doc, but not find anything. https://learn.microsoft.com/en-us/rest/api/storageservices/get-blob?tabs=microsoft-entra-id https://learn.microsoft.com/en-us/rest/api/storageservices/specifying-the-range-header-for-blob-service-operations

I also tried to send it with storage .net SDK, and get error "Specified argument was out of the range of valid values." from SDK side.

x-ms-range: bytes=0--1 looks is a none documented behavior. Azurite is based on rest API doc/swagger, we will try to get aligned behavior as Azure server, but cover all the corner cases is not our first priority.

Azurite welcome contribution! It would be great if you could raise a PR to fix this issue!

The change should be in the function: https://github.com/Azure/Azurite/blob/76f626284e4b4b58b95065bb3c92351f30af7f3d/src/blob/utils/utils.ts#L40 Please make sure change log is updated, and enough test coverage in the PR.

cnaj commented 2 months ago

So it's actually the Java SDK's fault because it sends an invalid header.

Nevertheless, it looks like Azure chooses to ignore the Range header (see https://httpwg.org/specs/rfc9110.html#field.range), either because of invalid range or because the content is empty.

From the RFC:

A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (...)

and

A server that supports range requests MAY ignore a Range header field when the selected representation has no content (i.e., the selected representation's data is of zero length).

We can see this from the fact that Azure reponds with 200 OK, and not with 206 Partial Content.

In this case, I think that Azurite should implement the same behavior, and not accept the invalid range request, which would also fix my use case (and be usable with the Java SDK). I'd be glad to help (time permitting), but TypeScript is not my main strength...

blueww commented 2 months ago

So it's actually the Java SDK's fault because it sends an invalid header.

Nevertheless, it looks like Azure chooses to ignore the Range header (see https://httpwg.org/specs/rfc9110.html#field.range), either because of invalid range or because the content is empty.

From the RFC:

A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (...)

and

A server that supports range requests MAY ignore a Range header field when the selected representation has no content (i.e., the selected representation's data is of zero length).

We can see this from the fact that Azure reponds with 200 OK, and not with 206 Partial Content.

In this case, I think that Azurite should implement the same behavior, and not accept the invalid range request, which would also fix my use case (and be usable with the Java SDK). I'd be glad to help (time permitting), but TypeScript is not my main strength...

@cnaj

Thanks for the reply! As you indicate, A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (...)

Current Azurite reject the invalid range, this is also aligned with RFC. However, Azure server choose ignore it, which is also aligned with RFC. As this is a corner case, we won't take it as our recent priority. Looking forward for your fixing PR.