search

Azure / azure-storage-python

Microsoft Azure Storage Library for Python
https://azure-storage.readthedocs.io
MIT License
339 stars 241 forks source link

list Azure Blobs by pattern #457

Open JWDobken opened 6 years ago

JWDobken commented 6 years ago

With the Python Azure API I would like to query blobs by a pattern.

For example, get all report pdf's (with the Unix style path name pattern expansion):

block_blob_service.list_blobs('mycontainer', '**/reports/*.pdf')
zezha-msft commented 6 years ago

Hi @JWDobken, thanks for reaching out!

I will certainly discuss your feature request with the team, but I think it's probably not a feature that we want to support at the moment, even though it could be very useful. The main reason is that this kind of search could be extremely expensive, and that is because the service only supports prefix searches. So for a search like '**/reports/*.pdf', we basically have to list every single blob on that container (could be millions of blobs), and check on the client side if the blob name matches the pattern. If we provided this API, I'm worried that users might not realize the cost associated with the operation (you're billed by the number of I/Os after all). Nevertheless, if the service provided additional indexing/searching, then this feature is still a possibility.

I'll updated this thread when I have news. Thanks!

eliisiita1 commented 1 year ago

Hi, has this been updated? Regards.