fsspec / adlfs

fsspec-compatible Azure Datake and Azure Blob Storage access
BSD 3-Clause "New" or "Revised" License
178 stars 104 forks source link

[feature request] support making directories beyond top-level containers #318

Open matt-erhart opened 2 years ago

matt-erhart commented 2 years ago

Current state from the doc for mkdir(): Mkdir is a no-op for creating anything except top-level containers. This aligns to the Azure Blob Filesystem flat hierarchy"

Datalake handles a nested dir structure with https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-namespace Also see: https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-python

Feature request is to support making directories beyond top-level containers.

matt-erhart commented 2 years ago

One way to do it:

import os, uuid, sys
from azure.storage.filedatalake import DataLakeServiceClient
from azure.core._match_conditions import MatchConditions
from azure.storage.filedatalake._models import ContentSettings
storage_account_name = 'testname'
service_client = DataLakeServiceClient(account_url="https://{storage_account_name}.dfs.core.windows.net")\
    .from_connection_string(conn_str=connection_string)

container_name = 'testcontainer'
file_system_client = service_client.get_file_system_client(file_system=container_name)
file_system_client.create_directory("my_directory/level1/level2")