search

fsspec / adlfs

fsspec-compatible Azure Datake and Azure Blob Storage access
BSD 3-Clause "New" or "Revised" License
174 stars 104 forks source link

Using dfs endpoint #169

Open wielrenner opened 3 years ago

wielrenner commented 3 years ago

For my ADLS2 blob I only have a dfs endpoint configured as a private endpoint.

When trying to list folder/files in a blob I get a HttpResponseError.

HttpResponseError: This request is not authorized to perform this operation. RequestId:00f7171a-201e-0039-6842-f5ecb8000000 Time:2021-01-28T06:54:33.4368226Z ErrorCode:AuthorizationFailure Error:None

Trying to set fs.account_url = "https://{}.dfs.core.windows.net".format(os.environ['AZURE_ACCOUNT_NAME']) also does not result in communication via the dfs endpoint. If I allow access from all networks on my storage account I'm able to connect.

I'm making the connection via a service principal. fs = AzureBlobFileSystem(account_name=xxxx, tenant_id=xxxx, client_id=xxxx, client_secret=xxxx)

wielrenner commented 3 years ago

Any thoughts?

sdebruyn commented 2 years ago

@wielrenner How did you solve this or did you create a workaround?