Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.83k stars 325 forks source link

Internal Server Error Page Blobs #286

Open galvesribeiro opened 4 years ago

galvesribeiro commented 4 years ago

Hello folks!

Using the latest Nuget: <PackageReference Include="Microsoft.Azure.Storage.Blob" Version="11.1.1" />

With Azurite on version: 3.3.0-preview (installed from npm)

And using the following code block (try/catch just to simplify the exception): image

I have this message on the output: 127.0.0.1 - - [26/Nov/2019:13:23:37 +0000] "GET /devstoreaccount1/events/TestStream1.ewh HTTP/1.1" 500 -

Debug log file shows this:

2019-11-26T13:34:54.712Z 0bf6b72b-7961-45cb-8f28-22d06f02c9d3 error: ErrorMiddleware: Received an error, fill error information to HTTP response
2019-11-26T13:34:54.712Z 0bf6b72b-7961-45cb-8f28-22d06f02c9d3 error: ErrorMiddleware: ErrorName=Error ErrorMessage=ENOENT: no such file or directory, open '__blobstorage__/4992e7e3-b3a2-4819-93e8-ec5060c570bf' ErrorStack="Error: ENOENT: no such file or directory, open '__blobstorage__/4992e7e3-b3a2-4819-93e8-ec5060c570bf'"
2019-11-26T13:34:54.712Z 0bf6b72b-7961-45cb-8f28-22d06f02c9d3 error: ErrorMiddleware: Set HTTP code: 500
2019-11-26T13:34:54.712Z 0bf6b72b-7961-45cb-8f28-22d06f02c9d3 info: EndMiddleware: End response. TotalTimeInMS=3 StatusCode=500 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.3.0-preview"}

However, I can clearly see the file:

image

But it doesn't download from storage explorer:

2019/11/26 13:37:45 ==> REQUEST/RESPONSE (Try=1/430.549776ms, OpTime=434.560191ms) -- RESPONSE STATUS CODE ERROR
   GET http://127.0.0.1:10000/devstoreaccount1/events/TestStream1.ewh?se=2019-11-29t13%3A37%3A45z&sig=-REDACTED-&sp=rl&sr=c&sv=2018-03-28&timeout=901
   If-Unmodified-Since: [Tue, 26 Nov 2019 13:16:36 GMT]
   User-Agent: [AzCopy/10.2.1 Azure-Storage/0.7 (go1.12; darwin)]
   X-Ms-Client-Request-Id: [012565b4-f481-4957-5f58-89914f770040]
   X-Ms-Range: [bytes=0-8388607]
   X-Ms-Version: [2018-03-28]
   --------------------------------------------------------------------------------
   RESPONSE Status: 500 Internal Server Error
   Connection: [keep-alive]
   Content-Length: [8388608]
   Content-Type: [application/octet-stream]
   Date: [Tue, 26 Nov 2019 13:37:45 GMT]
   Etag: ["d-at/07MNbfWpq9D3ySVd0zfVExCI"]
   Last-Modified: [Tue, 26 Nov 2019 13:16:36 GMT]
   Server: [Azurite-Blob/3.3.0-preview]
   X-Ms-Blob-Type: [PageBlob]
   X-Ms-Client-Request-Id: [012565b4-f481-4957-5f58-89914f770040]
   X-Ms-Lease-State: [available]
   X-Ms-Lease-Status: [unlocked]
   X-Ms-Request-Id: [f2fa3ebb-207e-46a3-92cf-29ad3b071cf6]
   X-Ms-Server-Encrypted: [true]
   X-Ms-Version: [2019-02-02]

2019/11/26 13:37:46 INFO: [P#0-T#0] Page blob throughput tuner: Target Mbps 4006
2019/11/26 13:37:47 INFO: [P#0-T#0] Page blob throughput tuner: Target Mbps 4012
2019/11/26 13:37:47 PERF: primary performance constraint is Unknown. States: R:  0, W:  0, F:  0, H:  1, B:  0, S:  0, P:  7, Q:  0, D:  0, T:  8
2019/11/26 13:37:48 INFO: [P#0-T#0] Page blob throughput tuner: Target Mbps 4018
2019/11/26 13:37:49 PERF: primary performance constraint is Unknown. States: R:  0, W:  0, F:  0, H:  1, B:  0, S:  0, P:  7, Q:  0, D:  0, T:  8
2019/11/26 13:37:49 INFO: [P#0-T#0] Page blob throughput tuner: Target Mbps 4024
2019/11/26 13:37:50 INFO: [P#0-T#0] Page blob throughput tuner: Target Mbps 4030
2019/11/26 13:37:50 ERR: [P#0-T#0] DOWNLOADFAILED: http://127.0.0.1:10000/devstoreaccount1/events/TestStream1.ewh?se=2019-11-29t13%3A37%3A45z&sig=-REDACTED-&sp=rl&sr=c&sv=2018-03-28 : 000 : unexpected EOF. When Downloading response body. X-Ms-Request-Id: 

   Dst: /Users/guto/Downloads/TestStream1.ewh
2019/11/26 13:37:50 JobID=71f2ecb0-266b-3d45-4030-d4099b81438a, Part#=0, TransfersDone=1 of 1
2019/11/26 13:37:50 all parts of Job 71f2ecb0-266b-3d45-4030-d4099b81438a successfully completed, cancelled or paused
2019/11/26 13:37:51 PERF: primary performance constraint is Unknown. States: R:  0, W:  0, F:  0, H:  1, B:  0, S:  0, P:  7, Q:  0, D:  0, T:  8

That problem doesn't happen on a real storage account.

Can someone shed a light on what is going on?

Thanks!

XiaoningLiu commented 4 years ago

Hi @galvesribeiro The error for your debug log shows the persisted data for the page blob is missing from your local file system. Some of the data of the page blob is persisted in __blobstorage__/4992e7e3-b3a2-4819-93e8-ec5060c570bf, but it's missing.

Can you share the full debug.log here? Including the page blob upload until to the error happens?

galvesribeiro commented 4 years ago

debug.log

Yeah, looks like I've found the cause. That __blobstorage__ was being created in a directory that wasn't where I targeted the -l or -d parameters and was keeping being deleted.

So, either the -l must be respected or the docs should be updated to mention that __blobstorage__ directory will be created on the directory you invoked the CLI.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.