search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.8k stars 320 forks source link

Blob Service: ListBlobsHierarchy doesn't return an error when invalid marker is passed in #1494

Open emarnadi-instabase opened 2 years ago

emarnadi-instabase commented 2 years ago

Which service(blob, file, queue, table) does this issue concern?

Blob Service

Which version of the Azurite was used?

1.16.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

Visual Studio Code and DockerHub

What's the Node.js version?

6.14.6

What problem was encountered?

When passing in an invalid marker into ListBlobsHierarchy with the Azure Blob Storage, a 400 error is returned, but when passing the same invalid marker with the same 'folder' structure into Azurite, no error is returned.

Steps to reproduce the issue?

Call ListBlobsHierarchy and input a marker that is invalid.

Please be sure to remove any PII or sensitive information before sharing!
The debug log will log raw request headers and bodies, so that we can replay these against Azurite using REST and create tests to validate resolution.

Have you found a mitigation/solution?

NO

blueww commented 2 years ago

@emarnadi-instabase Would you please share the details for:

  1. which invalid marker you have input?
  2. What's the server error from Azure Blob storage ?
  3. Azurite debug log
emarnadi-instabase commented 2 years ago

Sorry about the late reply

  1. Invalid marker: "invalid", or any other random string that shouldn't work
  2. With the old Go SDK, there was an error thrown. With the new Go SDK, no results are returned with an invalid marker. However, Azurite treats an invalid marker as an empty marker, and still returns results.
blueww commented 2 years ago

@emarnadi-instabase

I still don't get what you mean by "marker", do you mean list blob delimiter or prefix, or something else? Please help to provide the debug log for azurite. (better with the program you use to repro the issue.)

emarnadi-instabase commented 2 years ago

The marker's the continuation token

blueww commented 2 years ago

@emarnadi-instabase Thanks for the reply! I can repro this issue in Azurite.

It looks this is caused by Azurite return different continuation token than server on list blob. Azurite just return the last blob name, but server will return a more complex format include the base64 encoded blob name and time. So currently Azurite can accept any string as continuation token since it might be a blob name.

A possible fix might be to base64 encode the continuation token from Azurite, then not all string can be accept.

The related API code in Azurite is https://github.com/Azure/Azurite/blob/da13df9115d9deb904e0fb0837dcd17f87c06d57/src/blob/handlers/ContainerHandler.ts#L616

Azurite welcome contribution! It would be great if you can raise a PR to fix this.