search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.83k stars 325 forks source link

com.azure.storage.blob.models.BlobStorageException: Status code 400, (empty body) #454

Open snodnipper opened 4 years ago

snodnipper commented 4 years ago

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.7.0 (mcr.microsoft.com/azure-storage/azurite:3.7.0) I also used azurite:latest.

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

DockerHub

What's the Node.js version?

na (FROM node:10-alpine in Dockerfile)

What problem was encountered?

When using com.azure:azure-storage-blob:12.6.1 and using listBlobs an exception is thrown: com.azure.storage.blob.models.BlobStorageException: Status code 400, (empty body)

^ this is inconsistent with the production Azure blob storage service. This issue is documented here: https://github.com/Azure/azure-sdk-for-java/issues/6945. Azurite should have the same behaviour as the production service.

Steps to reproduce the issue?

//String connectionString = realConnectionString;
String connectionString = fakeConnectionString;
System.out.println("Using the Connection String: " + connectionString);

String containerName = "somecontainer";
StorageSharedKeyCredential credential = StorageSharedKeyCredential.fromConnectionString(connectionString);

BlobContainerClient blobContainerClient = new BlobContainerClientBuilder()
         .connectionString(connectionString)
         .containerName(containerName)
        .credential(credential)
        .buildClient();

// create the container if it doesn't already exist
if (!blobContainerClient.exists()) {
    blobContainerClient.create();
}

String location = "testing";

// As documented by https://github.com/Azure/azure-sdk-for-java/issues/6945
ListBlobsOptions options = new ListBlobsOptions().setDetails(new BlobListDetails().setRetrieveMetadata(true));

// this passes ✅
//PagedIterable<BlobItem> blobs = blobContainerClient.listBlobsByHierarchy(location, options, Duration.ofSeconds(15));
// this fails ❌
PagedIterable<BlobItem> blobs = blobContainerClient.listBlobsByHierarchy(location);
        assertFalse(blobs.iterator().hasNext());

Have you found a mitigation/solution?

ListBlobsOptions options = new ListBlobsOptions().setDetails(new BlobListDetails().setRetrieveMetadata(true));
blobContainerClient.listBlobsByHierarchy(location, options, Duration.ofSeconds(15));
blueww commented 4 years ago

@snodnipper

Would you please give the Azurite debug.log for this failed request?

I tried to repro it with .net SDK, and seems .net SDK works well on this request. See the following Debug.log on my machine for this request.

2020-06-10T03:28:17.894Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://localhost./devstoreaccount1/weitest1?restype=container&comp=list&prefix=Dir&delimiter= RequestHeaders:{"x-ms-version":"2019-07-07","x-ms-client-request-id":"7ed60104-5df9-441d-834f-6d0475ffdaf4","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Blobs/12.4.0 (.NET Framework 4.8.4180.0; Microsoft Windows 10.0.18363 )","x-ms-date":"Wed, 10 Jun 2020 03:28:17 GMT","authorization":"SharedKey devstoreaccount1:q22UGoOPHhARpmlmNKXSlv+ttiHURtY+T9pbbx39rGI=","host":"localhost.:10000"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2020-06-10T03:28:17.894Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=weitest1 Blob=
2020-06-10T03:28:17.895Z aec4f573-64d8-4620-a5dc-3ad06f470f2c verbose: DispatchMiddleware: Dispatching request...
2020-06-10T03:28:17.895Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: DispatchMiddleware: Operation=Container_ListBlobHierarchySegment
2020-06-10T03:28:17.896Z aec4f573-64d8-4620-a5dc-3ad06f470f2c verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2020-06-10T03:28:17.896Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: PublicAccessAuthenticator:validate() Start validation against public access.
2020-06-10T03:28:17.896Z aec4f573-64d8-4620-a5dc-3ad06f470f2c debug: PublicAccessAuthenticator:validate() Getting account properties...
2020-06-10T03:28:17.896Z aec4f573-64d8-4620-a5dc-3ad06f470f2c debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: weitest1, blob: 
2020-06-10T03:28:17.897Z aec4f573-64d8-4620-a5dc-3ad06f470f2c debug: PublicAccessAuthenticator:validate() Public access type for container weitest1 is "container"
2020-06-10T03:28:17.897Z aec4f573-64d8-4620-a5dc-3ad06f470f2c debug: PublicAccessAuthenticator:validate() Operation Container_ListBlobHierarchySegment is in container level public access list. Validation passed.
2020-06-10T03:28:17.898Z aec4f573-64d8-4620-a5dc-3ad06f470f2c verbose: DeserializerMiddleware: Start deserializing...
2020-06-10T03:28:17.898Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: HandlerMiddleware: DeserializedParameters={"options":{"prefix":"Dir","include":[],"requestId":"7ed60104-5df9-441d-834f-6d0475ffdaf4"},"delimiter":"","restype":"container","comp":"list","version":"2019-07-07"}
2020-06-10T03:28:17.899Z aec4f573-64d8-4620-a5dc-3ad06f470f2c verbose: SerializerMiddleware: Start serializing...
2020-06-10T03:28:17.899Z aec4f573-64d8-4620-a5dc-3ad06f470f2c debug: Serializer: Raw response body string is <?xml version="1.0" encoding="UTF-8" standalone="yes"?><EnumerationResults ServiceEndpoint="http://localhost.:10000/devstoreaccount1" ContainerName="weitest1"><Prefix>Dir</Prefix><Marker/><MaxResults>5000</MaxResults><Delimiter>/</Delimiter><Blobs/><NextMarker/></EnumerationResults>
2020-06-10T03:28:17.899Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: Serializer: Start returning stream body.
2020-06-10T03:28:17.900Z aec4f573-64d8-4620-a5dc-3ad06f470f2c info: EndMiddleware: End response. TotalTimeInMS=6 StatusCode=200 StatusMessage=OK Headers={"server":"Azurite-Blob/3.7.0","content-type":"application/xml","x-ms-client-request-id":"7ed60104-5df9-441d-834f-6d0475ffdaf4","x-ms-request-id":"aec4f573-64d8-4620-a5dc-3ad06f470f2c","x-ms-version":"2019-07-07","date":"Wed, 10 Jun 2020 03:28:17 GMT"}
snodnipper commented 4 years ago

Hopefully the following is helpful:

2020-06-10T03:49:31.946Z     info: Azurite Blob service is starting on 0.0.0.0:10000
2020-06-10T03:49:31.950Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2020-06-10T03:49:31.950Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2020-06-10T03:49:31.990Z     info: BlobGCManager:start() Starting BlobGCManager. Set status to Initializing.
2020-06-10T03:49:31.991Z     info: BlobGCManager:start() Trigger mark and sweep loop. Set status to Running.
2020-06-10T03:49:31.991Z     info: BlobGCManager:markSweepLoop() Start next mark and sweep.
2020-06-10T03:49:31.991Z     info: BlobGCManager:markSweep() Get all extents.
2020-06-10T03:49:31.994Z     info: BlobGCManager:start() BlobGCManager successfully started.
2020-06-10T03:49:31.998Z     info: BlobGCManager:markSweep() Got 4 extents.
2020-06-10T03:49:31.998Z     info: BlobGCManager:markSweep() Get referred extents.
2020-06-10T03:49:32.001Z     info: BlobGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2020-06-10T03:49:32.001Z     info: BlobGCManager:markSweepLoop() Mark and sweep finished, taken 10ms.
2020-06-10T03:49:32.001Z     info: BlobGCManager:markSweepLoop() Sleep for 600000ms.
2020-06-10T03:49:32.005Z     info: Azurite Blob service successfully listens on http://0.0.0.0:10000
2020-06-10T03:49:32.006Z     info: Azurite Queue service is starting on 0.0.0.0:10001
2020-06-10T03:49:32.006Z     info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2020-06-10T03:49:32.006Z     info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2020-06-10T03:49:32.027Z     info: QueueGCManager:start() Starting QueueGCManager, set status to Initializing
2020-06-10T03:49:32.027Z     info: QueueGCManager:start() Trigger mark and sweep loop, set status to Running.
2020-06-10T03:49:32.028Z     info: QueueGCManager:markSweepLoop() Start new mark and sweep.
2020-06-10T03:49:32.028Z     info: QueueGCManger:markSweep() Get all extents.
2020-06-10T03:49:32.028Z     info: QueueGCManager:start() QueueGCManager successfully started.
2020-06-10T03:49:32.029Z     info: QueueGCManager:marksweep() Get 0 extents.
2020-06-10T03:49:32.029Z     info: QueueGCManager:markSweep() Get referred extents, then remove from allExtents.
2020-06-10T03:49:32.029Z     info: QueueGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2020-06-10T03:49:32.029Z     info: QueueGCManager:markSweepLoop() Mark and sweep finished, take 1ms.
2020-06-10T03:49:32.029Z     info: QueueGCManager:markSweepLoop() Sleep for 60000
2020-06-10T03:49:32.030Z     info: Azurite Queue service successfully listens on http://0.0.0.0:10001
2020-06-10T03:49:33.343Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://localhost/devstoreaccount1/somecontainer?restype=container RequestHeaders:{"host":"localhost:32948","accept":"*/*","date":"Wed, 10 Jun 2020 03:49:33 GMT","authorization":"SharedKey devstoreaccount1:u74GruVbRYkGqxY8TPZRHo5wOQ8Ujwr9K+Kq4QiP07s=","x-ms-version":"2019-02-02","x-ms-client-request-id":"4139f388-48ef-4376-8c99-5618e0e9ad50","user-agent":"azsdk-java-azure-storage-blob/12.3.0 (11.0.7; Mac OS X 10.15.5)","content-length":"0"} ClientIP=172.17.0.1 Protocol=http HTTPVersion=1.1
2020-06-10T03:49:33.344Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=somecontainer Blob=
2020-06-10T03:49:33.345Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 verbose: DispatchMiddleware: Dispatching request...
2020-06-10T03:49:33.348Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: DispatchMiddleware: Operation=Container_GetProperties
2020-06-10T03:49:33.350Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2020-06-10T03:49:33.350Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: PublicAccessAuthenticator:validate() Start validation against public access.
2020-06-10T03:49:33.350Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 debug: PublicAccessAuthenticator:validate() Getting account properties...
2020-06-10T03:49:33.350Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: somecontainer, blob: 
2020-06-10T03:49:33.362Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container somecontainer
2020-06-10T03:49:33.363Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2020-06-10T03:49:33.369Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\nWed, 10 Jun 2020 03:49:33 GMT\n\n\n\n\n\nx-ms-client-request-id:4139f388-48ef-4376-8c99-5618e0e9ad50\nx-ms-version:2019-02-02\n/devstoreaccount1/devstoreaccount1/somecontainer\nrestype:container"
2020-06-10T03:49:33.371Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:u74GruVbRYkGqxY8TPZRHo5wOQ8Ujwr9K+Kq4QiP07s=
2020-06-10T03:49:33.371Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2020-06-10T03:49:33.371Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 verbose: DeserializerMiddleware: Start deserializing...
2020-06-10T03:49:33.374Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: HandlerMiddleware: DeserializedParameters={"options":{"requestId":"4139f388-48ef-4376-8c99-5618e0e9ad50","leaseAccessConditions":{}},"restype":"container","version":"2019-02-02"}
2020-06-10T03:49:33.376Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2020-06-10T03:49:33.377Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The specified container does not exist.  ErrorHTTPStatusCode=404 ErrorHTTPStatusMessage=The specified container does not exist. ErrorHTTPHeaders={"x-ms-error-code":"ContainerNotFound","x-ms-request-id":"7f741d2c-90fa-4767-b33d-e77bccb004c8"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>ContainerNotFound</Code>\n  <Message>The specified container does not exist.\nRequestId:7f741d2c-90fa-4767-b33d-e77bccb004c8\nTime:2020-06-10T03:49:33.374Z</Message>\n</Error>" ErrorStack="StorageError: The specified container does not exist.\n    at Function.getContainerNotFound (/opt/azurite/dist/src/blob/errors/StorageErrorFactory.js:14:16)\n    at LokiBlobMetadataStore.getContainerWithLeaseUpdated (/opt/azurite/dist/src/blob/persistence/LokiBlobMetadataStore.js:1864:53)\n    at LokiBlobMetadataStore.getContainerProperties (/opt/azurite/dist/src/blob/persistence/LokiBlobMetadataStore.js:307:32)\n    at ContainerHandler.getProperties (/opt/azurite/dist/src/blob/handlers/ContainerHandler.js:71:62)\n    at /opt/azurite/dist/src/blob/generated/middleware/HandlerMiddlewareFactory.js:58:18\n    at /opt/azurite/dist/src/blob/generated/ExpressMiddlewareFactory.js:77:63\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:317:13)\n    at /opt/azurite/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:335:12)\n    at next (/opt/azurite/node_modules/express/lib/router/index.js:275:10)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"
2020-06-10T03:49:33.377Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Set HTTP code: 404
2020-06-10T03:49:33.377Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Set HTTP status message: The specified container does not exist.
2020-06-10T03:49:33.377Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=ContainerNotFound
2020-06-10T03:49:33.377Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=7f741d2c-90fa-4767-b33d-e77bccb004c8
2020-06-10T03:49:33.377Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Set content type: application/xml
2020-06-10T03:49:33.378Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>ContainerNotFound</Code>\n  <Message>The specified container does not exist.\nRequestId:7f741d2c-90fa-4767-b33d-e77bccb004c8\nTime:2020-06-10T03:49:33.374Z</Message>\n</Error>"
2020-06-10T03:49:33.380Z 7f741d2c-90fa-4767-b33d-e77bccb004c8 info: EndMiddleware: End response. TotalTimeInMS=38 StatusCode=404 StatusMessage=The specified container does not exist. Headers={"server":"Azurite-Blob/3.7.0","x-ms-error-code":"ContainerNotFound","x-ms-request-id":"7f741d2c-90fa-4767-b33d-e77bccb004c8","content-type":"application/xml"}
2020-06-10T03:49:33.492Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://localhost/devstoreaccount1/somecontainer?restype=container RequestHeaders:{"host":"localhost:32948","accept":"*/*","date":"Wed, 10 Jun 2020 03:49:33 GMT","authorization":"SharedKey devstoreaccount1:RarLeXVTdRu5urUYS/cxK7t7MLEPsw3V6h36tZ1QuIw=","x-ms-version":"2019-02-02","x-ms-client-request-id":"a02b46f1-2b60-4b8d-90c2-768866c55aa6","user-agent":"azsdk-java-azure-storage-blob/12.3.0 (11.0.7; Mac OS X 10.15.5)","content-length":"0"} ClientIP=172.17.0.1 Protocol=http HTTPVersion=1.1
2020-06-10T03:49:33.492Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=somecontainer Blob=
2020-06-10T03:49:33.493Z 85c2a3f1-4735-4465-a698-b2d7822de01c verbose: DispatchMiddleware: Dispatching request...
2020-06-10T03:49:33.495Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: DispatchMiddleware: Operation=Container_Create
2020-06-10T03:49:33.495Z 85c2a3f1-4735-4465-a698-b2d7822de01c verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2020-06-10T03:49:33.495Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: PublicAccessAuthenticator:validate() Start validation against public access.
2020-06-10T03:49:33.495Z 85c2a3f1-4735-4465-a698-b2d7822de01c debug: PublicAccessAuthenticator:validate() Getting account properties...
2020-06-10T03:49:33.495Z 85c2a3f1-4735-4465-a698-b2d7822de01c debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: somecontainer, blob: 
2020-06-10T03:49:33.496Z 85c2a3f1-4735-4465-a698-b2d7822de01c debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container somecontainer
2020-06-10T03:49:33.496Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2020-06-10T03:49:33.496Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\n\n\n\n\nWed, 10 Jun 2020 03:49:33 GMT\n\n\n\n\n\nx-ms-client-request-id:a02b46f1-2b60-4b8d-90c2-768866c55aa6\nx-ms-version:2019-02-02\n/devstoreaccount1/devstoreaccount1/somecontainer\nrestype:container"
2020-06-10T03:49:33.496Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:RarLeXVTdRu5urUYS/cxK7t7MLEPsw3V6h36tZ1QuIw=
2020-06-10T03:49:33.496Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2020-06-10T03:49:33.496Z 85c2a3f1-4735-4465-a698-b2d7822de01c verbose: DeserializerMiddleware: Start deserializing...
2020-06-10T03:49:33.497Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: HandlerMiddleware: DeserializedParameters={"options":{"metadata":{},"requestId":"a02b46f1-2b60-4b8d-90c2-768866c55aa6"},"restype":"container","version":"2019-02-02"}
2020-06-10T03:49:33.500Z 85c2a3f1-4735-4465-a698-b2d7822de01c verbose: SerializerMiddleware: Start serializing...
2020-06-10T03:49:33.501Z 85c2a3f1-4735-4465-a698-b2d7822de01c info: EndMiddleware: End response. TotalTimeInMS=9 StatusCode=201 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.7.0","etag":"\"0x1C2700E06858820\"","last-modified":"Wed, 10 Jun 2020 03:49:33 GMT","x-ms-client-request-id":"a02b46f1-2b60-4b8d-90c2-768866c55aa6","x-ms-request-id":"85c2a3f1-4735-4465-a698-b2d7822de01c","x-ms-version":"2019-07-07"}
2020-06-10T03:49:33.585Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://localhost/devstoreaccount1/somecontainer?prefix=test&delimiter=/&include=&restype=container&comp=list RequestHeaders:{"host":"localhost:32948","accept":"*/*","date":"Wed, 10 Jun 2020 03:49:33 GMT","authorization":"SharedKey devstoreaccount1:1aNUjiONeG+w5DIau0joDsORCT2NF8tXTPoB5pMtWpg=","x-ms-version":"2019-02-02","x-ms-client-request-id":"504370d8-2af2-4da9-8569-6cce9348dc23","user-agent":"azsdk-java-azure-storage-blob/12.3.0 (11.0.7; Mac OS X 10.15.5)","content-length":"0"} ClientIP=172.17.0.1 Protocol=http HTTPVersion=1.1
2020-06-10T03:49:33.585Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=somecontainer Blob=
2020-06-10T03:49:33.585Z f50f18b7-715d-4643-acbf-62c89fcd2a4a verbose: DispatchMiddleware: Dispatching request...
2020-06-10T03:49:33.586Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: DispatchMiddleware: Operation=Container_ListBlobHierarchySegment
2020-06-10T03:49:33.586Z f50f18b7-715d-4643-acbf-62c89fcd2a4a verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2020-06-10T03:49:33.586Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: PublicAccessAuthenticator:validate() Start validation against public access.
2020-06-10T03:49:33.586Z f50f18b7-715d-4643-acbf-62c89fcd2a4a debug: PublicAccessAuthenticator:validate() Getting account properties...
2020-06-10T03:49:33.587Z f50f18b7-715d-4643-acbf-62c89fcd2a4a debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: somecontainer, blob: 
2020-06-10T03:49:33.588Z f50f18b7-715d-4643-acbf-62c89fcd2a4a debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container somecontainer
2020-06-10T03:49:33.588Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2020-06-10T03:49:33.588Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\nWed, 10 Jun 2020 03:49:33 GMT\n\n\n\n\n\nx-ms-client-request-id:504370d8-2af2-4da9-8569-6cce9348dc23\nx-ms-version:2019-02-02\n/devstoreaccount1/devstoreaccount1/somecontainer\ncomp:list\ndelimiter:/\ninclude:\nprefix:test\nrestype:container"
2020-06-10T03:49:33.588Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:1aNUjiONeG+w5DIau0joDsORCT2NF8tXTPoB5pMtWpg=
2020-06-10T03:49:33.588Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2020-06-10T03:49:33.588Z f50f18b7-715d-4643-acbf-62c89fcd2a4a verbose: DeserializerMiddleware: Start deserializing...
2020-06-10T03:49:33.590Z f50f18b7-715d-4643-acbf-62c89fcd2a4a error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2020-06-10T03:49:33.590Z f50f18b7-715d-4643-acbf-62c89fcd2a4a error: ErrorMiddleware: ErrorName=DeserializationError ErrorMessage= is not a valid value for include. The valid values are: ["copy","deleted","metadata","snapshots","uncommittedblobs"].  ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=undefined ErrorHTTPHeaders=undefined ErrorHTTPBody=undefined ErrorStack="Error:  is not a valid value for include. The valid values are: [\"copy\",\"deleted\",\"metadata\",\"snapshots\",\"uncommittedblobs\"].\n    at serializeEnumType (/opt/azurite/node_modules/@azure/ms-rest-js/dist/msRest.node.js:747:15)\n    at Serializer.serialize (/opt/azurite/node_modules/@azure/ms-rest-js/dist/msRest.node.js:527:27)\n    at serializeSequenceType (/opt/azurite/node_modules/@azure/ms-rest-js/dist/msRest.node.js:820:35)\n    at Serializer.serialize (/opt/azurite/node_modules/@azure/ms-rest-js/dist/msRest.node.js:539:27)\n    at Object.deserialize (/opt/azurite/dist/src/blob/generated/utils/serializer.js:25:25)\n    at Object.deserializerMiddleware [as default] (/opt/azurite/dist/src/blob/generated/middleware/deserializer.middleware.js:29:18)\n    at /opt/azurite/dist/src/blob/generated/ExpressMiddlewareFactory.js:62:46\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:317:13)\n    at /opt/azurite/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:335:12)\n    at next (/opt/azurite/node_modules/express/lib/router/index.js:275:10)\n    at authenticate.then.pass (/opt/azurite/dist/src/blob/middlewares/AuthenticationMiddlewareFactory.js:19:21)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"
2020-06-10T03:49:33.590Z f50f18b7-715d-4643-acbf-62c89fcd2a4a error: ErrorMiddleware: Set HTTP code: 400
2020-06-10T03:49:33.591Z f50f18b7-715d-4643-acbf-62c89fcd2a4a error: ErrorMiddleware: Set HTTP body: undefined
2020-06-10T03:49:33.591Z f50f18b7-715d-4643-acbf-62c89fcd2a4a info: EndMiddleware: End response. TotalTimeInMS=6 StatusCode=400 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.7.0"}
blueww commented 4 years ago

@snodnipper

Thanks for the info! I get the failure happens when "include=" don't have any value. As I see, Java SDK has fixed the issue, and I don't see other SDK (like .net) has this issue. So seems the issue is not blocking currently.

We will discuss how to fix it, and update when get a decision.

XiaoningLiu commented 4 years ago

It's another undocumented behavior for Azure Storage production, not an issue of Azurite.

snodnipper commented 4 years ago

Sure - perhaps there is opportunity to share test suites between production and Azurite.

We have updated the dependencies across our services and practically this is no longer an issue.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.