search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.54k stars 2.77k forks source link

Gen2 storage helper fails without container creation permissions #30362

Open tomasvanpottelbergh opened 1 year ago

tomasvanpottelbergh commented 1 year ago

I am using the azure.ai.ml._artifacts._artifact_utilities.download_artifact_from_aml_uri function as a workaround to download Azure ML data assets locally (see #26213 for more details).

However, this method has stopped working in more recent versions of azure-ai-ml, giving AuthorizationFailure errors. I have managed to trace the problem down to these lines of code which try to create a new storage container. The problem is that the except ResourceExistsError doesn't catch the authorization error, which occurs because I only have read permissions.

Would it be possible to change the Gen2StorageClient in a way that also works when the user has read-only access to the storage account?

github-actions[bot] commented 1 year ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github @Azure/azure-ml-sdk.

luigiw commented 1 year ago

@diondrapeck , can you share your suggestion on this?

diondrapeck commented 1 year ago

@tomasvanpottelbergh Downloading data assets locally isn't supported, so this is expected behavior. Your workaround was a loophole via a private function which is no longer working. We can evaluate the value of adding a download method for individual assets (right now Job download is what we offer), but it's not something we currently plan to implement.

You could use the AzureML Storage Blob SDK to directly download the assets. That's what we're using in the backend; however, I'm not sure if your read-only permissions will block you. That's not something we have control over.

tomasvanpottelbergh commented 1 year ago

Thanks for your reply @diondrapeck. I fully understand that I was using a private function, but since this function is used throughout the SDK, the error will occur whenever a user with read-only access tries to download something from a Blob datastore.

Adding an option to download individual assets (as existed in SDK v1) would be very welcome! The azureml-fsspec goes some way, but it needs to be combined with the SDK to get the Azure ML URI first. Is there a reason for not adding the download option to the SDK directly, even though it already contains the necessary helpers?